Natasha Oza

Coinbase Pro: Future of the Portfolio Feature

GOAL: Understand Coinbase Pro user sentiment and behavior to determine whether the portfolios feature should be built on the new Retail Advanced Trade platform (RAT) or not.

 

SITUATION: Cross-functional collaboration with the product team, engineering team, and senior leadership, culminating in an all-hands presentation.

 

IMPACT: Initial survey results were so compelling, the team asked for a research scope extension.

The final presented research insights made portfolios top of mind for senior leadership. Instead of being slated for automatic deprecation, building the portfolios on RAT (or proposed substitutes) was under consideration.

Company: Coinbase

Methods: Literature review, Interviews, Survey, Kano analysis

Role: User Researcher (intern)

Team: 2 Product managers, Data scientist, Product designer

Duration: 8 weeks

BACKGROUND AND PROBLEM

Portfolios are a common feature across fintech products, including Coinbase Pro, that help users manage their investment strategies by separating their assets into “folders”. ~5 million active Coinbase Pro users use portfolios.

 

With the deprecation of Coinbase Pro, all its users were being migrated to Retail Advanced Trade (“RAT”). RAT had all the features of Coinbase Pro, and more, except for one: portfolios. Building portfolios in RAT would be a huge time and resource intensive feat. However, would the consequences of not having portfolios on RAT be worth it?

ROLE AND CONTRIBUTION

I was the user researcher in charge of this project. I worked on it end-to-end, which included:

  • Creating a research plan
  • Building and shipping the survey
  • Kano and qualitative analysis on the survey results to present to the team
  • Planning and creating interview guides
  • Recruiting participants
  • Conducting interviews
  • Synthesizing insights to build product strategy recommendations.

Given the business impact of the portfolios feature, this research was crucial and required cross-functional collaboration with data science, product managers, engineers, and senior leadership.

Research Question

How might we understand the importance of portfolios to the trading strategy of Coinbase Pro users?

Research goals:

  • Determine the importance of portfolios for different users (API vs non-API users)
    • How does perceived importance of portfolios differ for low vs. high-volume traders?
  • Understand what purpose portfolios serve for users (current jobs being served)
  • Identify risk of users migrating to competitor platforms if RAT doesn’t have portfolios

IN-DEPTH PROCESS

Phase 0 (stakeholder interviews)

My first challenge was ramping up on all things crypto, Coinbase Pro, portfolios, and Retail Advanced Trade (RAT).

 

I scheduled multiple 1:1 stakeholder conversations to understand from different teams what the user experience is and what they envision it to be. Some questions I addressed through this activity were:

  • Why is Coinbase Pro being deprecated and users migrated to RAT?
  • Who are current Coinbase Pro users?
    • Who are portfolio users?
  • What are the assumptions the team has about portfolio usage?
  • What are limitations to building portfolios on RAT?
  • What are the decisions the team wants this research to inform?

⚒️ Challenge: The week I joined, Coinbase reduced its workforce by 18%. Suddenly, my project seemed inconsequential. Stakeholders I’d set up calls with had lost their jobs, had their colleagues affected, and were navigating incredible uncertainty. I offered to reschedule or, if they chose not to, used the time to give them a space to talk. While there are situations that demand pushing harder, this one deserved empathy. As the project took off, I felt the team was more willing to support me because of the relationship we established in these initial interactions.

Phase 1 (data science collaboration and surveys)

Identifying users

I wanted our sample to consist only of active portfolio users, i.e., those who have at least 2 portfolios with >$1 balance in each.

 

Addressing the research question

I leveraged the Kano model to asses Coinbase Pro user sentiment on portfolios as a function of:

  • Satisfaction (where the user lies on the spectrum of delighted to frustrated with portfolios)
  • Functionality (how well the user thinks we’ve implemented and developed portfolios)

I also crafted a Likert scale to get a general sense of importance of portfolios. We chose to keep it a 4-point scale to prevent users from picking the common neutral option. This wouldn’t skew our results because the Kano questions would reflect any user neutrality.

 

Calibration questions

I created calibration questions about “limit orders”, a feature we know users find critical to trading, to map how accurate their responses on portfolios were.

Phase 2 (Kano analysis and qualitative responses)

A kano analysis of the data revealed that a majority of users found portfolios a table-stakes feature and crucial to their trading.

Coding the qualitative responses from the survey into themes, we identified that users leverage portfolios mainly to apply different trading strategies, manage assets of/for different people, and organise their assets for tax purposes.

The numbers behind the people: the final kano analysis for the portfolios feature
The opinion that portfolios is crucial to trading was shared by both API and non-API users

These results were shocking and contradicted the team’s main assumptions

 

Extending scope of research

Given the resources that building portfolios on RAT would take, we need to dig into the WHY and HOW behind the survey numbers

Phase 3 (interviews)

I conducted eight 45 minute 1:1 semi-structured in-depth interviews with portfolio users (3 API users, 5 non-API users).

🚀  Learning: Coinbase Pro users tend to be more experienced traders. I was nervous about my interviews with them given my relative lack of technical knowledge. However, asking them to break down their thoughts and strategies not only helped me understand better, but also allowed me to identify the key jobs that portfolios served for them!

Phase 4 (data analysis, affinity map)

I built an affinity map of the interview insights, anchoring them to the research goals and potential product strategy options.

A high-level overview of my affinity diagram

INSIGHTS AND RECOMMENDATIONS

Insights

There were two main insights from the survey and subsequent interviews:

  1. The purposes of portfolios aren’t mutually exclusive
    • There seemed to be three main jobs the portfolios served for users, but they also had evident overlaps
    • Portfolios served this purpose for users agnostic of whether they were API-users or non-API users (although API users tended to rely more heavily on portfolios for tax management)
  2. From their portfolios, users seek the date of asset acquisition, type of cryptocurrency, and bird’s-eye view of current loss/gain standing
Purpose of portfolios: although some of these purposes of portfolios were known beforehand, my analysis highlighted the their interconnectedness and overlap

Recommendation

Based on the insights, I proposed that it was worth investing in building portfolios on RAT.

 

However, I acknowledged the extreme time and resource constraints the engineering and product teams were already under. Therefore, I recommended substitutes that could replace portfolios or be adapted to work as “band-aid” solutions: addressing user needs while accounting for business realities.

🚀  Learning: I knew that for my research to be actionable I had to go beyond just the “why” and emphasize on the “so what?”. Therefore, although the scope of the project only included understanding user sentiment and behavior with respect to portfolios, I built on my insights to recommend substitutes for the team to consider and explore. In the future, it would have helped to have this as a part of my research goals from the beginning of the project.

IMPACT

In the short term, my research shifted the conversation of portfolios on RAT. It got senior leadership from engineering, product design, and product management into a (virtual) room to brainstorm options and next steps.

 

In the long term, my research and recommendations informed product strategy. Even if the team decided to not go ahead with building portfolios, or a substitute, it would be an informed decision: they would be able to prepare for its consequences instead of being caught off-guard.

Slack messages from the Group Product Manager that were sent on the RAT group after I presented my research on the portfolios feature - he can't stop thinking about it!

REFLECTION

What I learnt:

  • Disagree and commit. As a researcher, my interactions with users made me have a strong opinion on what outcome would be best for them. However, this project highlighted how many factors go into making large-scale product decisions resulting in an outcome that might not be fully aligned with what’s best for the end user. I learned how to be objective: communicate my research effectively, advocate for the users where necessary (asking “How will this give our users what they need?”) and then aligning with the final decision the team arrives at.

What went well

  • Advocating for myself as a researcher. Because of the product consequences of my research, there was a time pressure. Occasionally, the product team wanted research to be conducted and analysed in a time frame that was unrealistic and would negatively impact the quality of the insights. I was able to advocate for the time and resources I needed to conduct rigorous research, while keeping in mind product timeline constraints.

Next time…

  • User segmentation. Given the focus on API users and developers, I would have liked to have been more intentional with my recruiting to ensure I was getting more of a representation from this segment.
Scroll to top