AI product management at Spotify: three layers, three ways of understanding a PM's job

The article, published on Spotify's engineering blog, starts by criticizing how the topic is usually discussed: nearly every piece of writing and podcast about product and AI refers to «the AI PM» as if it were a single job. Three Principal Product Managers from the same organization (Personalization) share the same tech stack and the same company priorities. They argue that the job changes a great deal depending on what is being built. Each describes a different layer: research, model and user experience.

All three use the same vocabulary («quality», «evals», «shipping»), but the meaning depends on the problem each one owns. The idea of impact changes too. In research, impact can mean the organization changes what it believes or where it invests. For the model PM, it means a model improvement leads to better recommendations across several experiences. For the experience PM, it means users want the product and come back. The authors stress that the fundamentals of product management still hold. What changes is the specialization that comes with what each person owns.

Research layer: product as technical leverage. Henok leads product in Tech Research and describes the role as «technical leverage»: applying scientific knowledge where it can change what Spotify builds, or how confidently it builds it. He highlights three ideas.

The first is that AI changes which problems need research. With better models and tools, product teams can now do work themselves that once required research. Research therefore focuses on shared methods, system design and emerging capabilities where the right approach is still unclear. He gives two examples: how to evaluate a model's open-ended behavior, and how to represent users and content so that several products can benefit. He warns that building an attractive prototype is easy, but getting from there to a product that is reliable for hundreds of millions of users can take a long way. A more complex model can look impressive in a demo without making any real difference on production tasks. That is why evaluation, latency, cost, data and integration are part of the research question.

The second is that a promising architecture is not a roadmap. Henok describes exploring an architecture that brings together capabilities currently spread across different systems. At first the question was whether it could work. As evidence built up, a harder question emerged: what the organization should do with it. His job is to turn the big idea into sensible decisions: start with a concrete problem, find a useful test, and agree on what evidence would justify the next step. It is not all or nothing. The methods and components developed along the way can already improve existing systems and guide the model and experience teams, even while the question of the full architecture remains open.

The third is the mix of long feedback loops and a fast-moving field. The impact of research can take months to show up, often in another team's roadmap or metrics, and meanwhile the technology and the priorities shift. That is why influence is central. Within the team, he provides product and strategy context without prescribing the answer. Outside it, he ties the work to decisions, brings the relevant teams in early and makes clear what the evidence does and does not show. If that only starts once the research is finished, the decision window may already have closed. The role also requires understanding the technology more deeply than other PM roles do. He relies on AI tools to work with papers and prototypes and stays close to one or two projects at a time.

Model layer: the capability as product, and another team as customer. Divita leads product for a foundation model, a language model built specifically for recommendations and able to reference real Spotify items. As with any platform, what she builds reaches users through other teams. What sets it apart is that behavior is shaped through data, outputs are probabilistic and quality means something different to each team that integrates it. Her customers are internal teams with their own roadmaps and metrics, and the same model has to serve different definitions of a good recommendation. Her task is to understand those needs and decide where a shared capability can serve them well. A request from one team may point to something useful for several experiences. The fundamentals become concrete. Product-market fit means knowing which problems teams can solve with the model. The value proposition means explaining what it offers for their specific experience. Business impact comes through what they ship with it.

Her first idea is that a model's behavior is not specified: it is shaped, and then you show that it matters. In a traditional role, you define what the system should do and work with engineers to build it. Here, behavior is learned through training rather than written in code. The two real levers are what the model is taught and how what it has learned is checked: data and evals. Much of her work is deciding what those evaluations should measure. A plausible recommendation is only the starting point. You have to assess how it fits the listener's context, how it balances familiarity and discovery and whether it stays relevant over time. Then you have to connect what is measured in the model with what listeners experience: offline evaluations explain the model's behavior, and online metrics show how it translates into experience. What is learned with partner teams helps prioritize the next training cycle.

Her second idea is to show what a shared model makes possible. The promise is that a single improvement benefits several experiences, but each team already has a system optimized for its own needs. The first step is to show that the shared model meets each experience's quality, latency and cost requirements, using criteria defined and agreed with partners. The second, longer-term step is to establish what teams gain by building on the same model: whether what is learned in one experience improves another, or whether a team can try something new by relying on an existing capability. She turns that promise into concrete, testable questions. She also tracks how the model evolves between training cycles and how changes carry over between experiences, to tell a promising result apart from a pattern worth planning around.

Her third idea is that a good model is not enough if nobody ships anything with it. Success means other teams launching products with the model. Much of the work looks little like model work. It means finding a team whose current system has plateaued on a metric it cares about and making the model a practical, compelling path forward. It also means agreeing on what the capability must deliver, defining the integration contract and testing the riskiest assumptions before asking the partner to commit. Her best result of the year came from a surface team that wanted an outcome they could deliver. It worked because they agreed on how to measure success before agreeing on the architecture.

Experience layer: the user is the customer and the model is just one ingredient. Humberto builds the product users interact with directly. It is the surface where a vague mood becomes a set of songs, and where the system has to do something useful even when the request is ambiguous. His customers are users, and that changes almost everything. The research and model layers are his underlying technology, and his job is to turn it into something enjoyable. Being closer to the user, he sees more clearly which model gaps and research questions matter most right now. As a result, depending on the other layers works more as a shared prioritization signal than as a bottleneck.

His first idea is to design for infinite input. In a traditional product, the user's action is bounded (pressing a button, choosing from a menu, typing a keyword), and the response is predictable and verifiable. In an AI experience, input is open-ended. Requests can be vague, underspecified or complex and may not map to a single catalog entity, so agents interpret what the user probably meant. That reopens constraints that seemed fixed. Latency that would be unacceptable in a keyword search can be tolerable in a conversation, because the user understands they are asking for something harder. But how to respond when the system is unsure, and how to recover from an error, have no settled answers, and these belong to the product, not the model.

His second idea is that evaluation is a product, not just a process. A/B tests and offline evaluation are still useful, but they fall short when not every scenario can be defined in advance. His team builds new instruments: datasets drawn from real usage patterns, feedback loops that reveal where the system misread a request, and LLM-based judges and custom evaluators for the product's specific interactions. With them, the team measures whether a change is a real improvement or just a difference. Evaluation is not a pre-launch phase but an ongoing product. A general evaluation valid across the foundation model's many uses is a starting point. The experience team has to close the gap between «the model is good in general» and «this interaction was good for this user».

His third idea is that conversation makes feedback natural. In a conversational product, users talk to the system, and when something goes wrong they say so in their own words and in real time, which earlier surfaces never allowed. A phrase like «that's not what I meant» carries information about the experience, the model and potentially the training data, all at once. The work is to build mechanisms that detect that signal at scale, interpret it and send it back along the collaboration chain so that every layer improves. Humberto sums up the role this way: it is the most direct line to the user and the most tangible ownership of the outcome. Yet it is built on systems he can influence without controlling them. It also serves users whose expectations are shaped in real time by every other AI product they use. The craft is achieving something reliable with ingredients that are probabilistic by nature.

Final reflection. The authors conclude that there is no single new skill set for the «AI PM». Product management is still the same craft: defining the problem well, knowing the customer deeply, measuring what matters and building conviction with evidence rather than enthusiasm. What the AI stack adds is a layer of specialization on top of those fundamentals. It comes in more variants than people usually assume and changes from one product to the next.

Editorial note: the piece reflects the internal experience of three product leaders and publishes no results figures or specific metrics. Its claims are qualitative and describe approaches and lessons learned, not quantitative evidence.