Introduction & Marcelo's journey
Ana: Hello everyone and welcome to another episode of the Cieden Podcast. Today I have a great guest – Marcelo Calbucci, who has worked in many big tech companies and numerous startups, and is also the author of the PRFAQ framework, which was released this year. So, congratulations on this great milestone.
Marcelo: Thank you for having me, Ana.
Ana: So, maybe let's start with your path and your journey. What was a turning point in your career that made you who you are right now?
Marcelo: Yeah, so I am originally from Brazil, where I got a computer science degree, and shortly after that, I joined Microsoft. Microsoft hired me in Brazil and brought me to the US. I spent seven years working there, and most of that time I was working on Bing, the search engine. Then I left Microsoft because I was thirsty for more innovation. I spent the next 18 years building startups, both in the US and in Europe. After that, I spent a couple of years at Amazon, then left and wrote a book about one of Amazon's innovation frameworks called the PRFAQ,
Core concepts of the PRFAQ framework
Ana: Maybe you can walk us through the core concepts that are the building blocks of the framework. And how does it help to align the team around the vision and strategy for the product?
Marcelo: Yeah, so PRFAQ stands for Press Release and Frequently Asked Questions. And although it has "press release" in the name, it's not a marketing tool; it's actually a product and strategy tool. It was invented by Amazon 20 years ago to make better decisions, to ensure that the product was addressing the needs of the customer, and to make sure that everyone collaborated on a better strategy.
The way it works is you have a document artifact that you create, a process for how to create and use it (how you review with people, how you seek feedback), and then you have a style of writing called "precise writing." These three things combined are what make the PRFAQ framework. Most people just talk about the document itself. If you go to Google and search for the PRFAQ framework, you're going to find lots of articles explaining this six-page doc that Amazon uses, but that is just one portion of it.
What this document is, is it starts with a press release that is written as if what you wanted to do is already done. It's a future press release. So you're writing a press release that is maybe a year from now, and you write, ‘Today we are launching this product that's going to solve this customer's problem, yada, yada, yada.’ Everything in the document is based on that. The idea is: you want to paint a picture of the future and go around to your team—your engineering team, design, executives, everyone—and ask, ‘Is this the future that we want? And do we have the right steps to get there?’ It's not a plan. It's not a roadmap of, ‘This is what we're going to do this quarter. This is what we're going to do next quarter.’ It's about the strategic decisions that we're going to make. Is it going to be mobile- or desktop-first? Are we going to launch in one country or all the countries? These are important strategic decisions that you need to think about early on, before you start executing or even building a roadmap.
The PRFAQ as a "straw man proposal" for debate
Ana: So, during the preparation or writing of this document, there's a lot of homework done to make it happen. It's not like you just sit down and imagine how this future will look. Even with the specific sections and methodology you highlight in the document, you're not just inventing things from scratch.
Marcelo: That's a great way to put it. Basically, you are drafting a document so people have a starting point. It's a straw man proposal that you put in front of folks, and then you start debating: Is this the best solution? Is this the right problem? Is this the right marketing and go-to-market strategy?
Using PRFAQs to test hypotheses
Ana: How can this tool help us evaluate opportunities, especially when there could be multiple? Does that mean we would create multiple PRFAQ documents to compare them?
Marcelo: It really depends on the stage of your business and your product. There are different types of PRFAQs. The earliest type of PRFAQ is for when you have identified a potential problem and a potential solution, but you don't know if the problem really exists. You might need to do some customer discovery around it. Is this problem worth solving? Is it big enough? Are customers already solving it? This might be a PRFAQ very early on, before you even say, "Yeah, I'm going to build this startup" or "I'm going to build this product." If you don't know the problem exists, you should pause. Go figure that out first.
Assuming that you have identified a problem—either an opportunity or a challenge—now what you have are facts about that problem and hypotheses about the solution. During the PRFAQ process, you're going to work on that hypothesis. You're going to come out of the PRFAQ with the best hypothesis you can, but it's still a hypothesis. You haven't built anything; you haven't proven it. But it is a hypothesis that the team feels is the right direction. So, you try to move in that direction. And by the way, the PRFAQ should be a week or two of work. You're not talking about months or quarters to do this. Then you start executing and you learn if that solution is feasible and if customers really value it or not. If they do, it's fantastic. If they don't, then you have to go back. You say, ‘Well, we tried and it didn't work. Let's pivot.’ And that's where the word "pivot" comes from. You can pivot both on a solution strategy and a problem strategy. Maybe while testing the solution, you discover the problem wasn't that interesting or big after all.
Reconciling the PRFAQ with agile methods
Ana: With a very volatile market, how long can we create a strategy for? Is it for a year, for three years, or just until the next pivot when we learn new things? I also felt that this framework resonates with the waterfall approach, where we align on the strategy and then build a rigid roadmap. How does it fit into agile?
Marcelo: The PRFAQ works really well for waterfall projects because they are long—they might take 18 months, two years, three years—so you do want to put effort into making the PRFAQ great. But I would also say it works well in Lean and Agile.
One of the problems I see is that people confuse what strategy is. They think strategy is a plan. They have a list of things that they're going to do, and that is not a strategy. That is a plan, and sometimes it's even called a strategic plan, but a strategic plan is not a strategy. Strategy is the choices you are making in terms of the solution and execution that are going to guide the team. The strategy is the guardrails that you're going to put on. In the PRFAQ, we also include the vision, which is your North Star. We're going to take this road towards that destination, but we're not going to specify what car we're going to drive, when we're going to stop, or what our milestones are.
In an agile environment, you're driving on the road and you have a flat tire—no problem. You change the tire and you continue. The guardrails tell you, "No, no, no, this is outside of the stretch of what we are doing." So you get back to the road. And this road is really the customer problem we're trying to address and how we envision a solution in terms of benefits for the customer, not in terms of features. It doesn't work if you start listing out the features, because the features are going to change, just like you said. Maybe you have a web interface and you want to move to an AI chatbot interface. As long as it is within the strategy, that's totally fine. The PRFAQ shouldn't change. You're still addressing the same problem within the same boundaries.
Why PRFAQ avoids visuals for "day zero" alignment
Ana: That actually answers my next question, because you suggest not sticking to visuals. That stood out for me as a person who works in a design agency that uses agile. We create prototypes because we believe people are visual. In order to visualize a solution, you need something tangible, like a prototype.
We had an experience where we worked on a project, and our CPO adopted the Play to Win framework. We created a huge presentation, then worked a few months on a vision prototype, putting in a lot of effort and thought. But it turned out that we hadn't dived deep enough into the strategic thought process around who we were targeting and what problems we wanted to solve. Retrospectively, I think having this approach of thinking strategically and diving into problems, rather than jumping into solutions, would have helped us a lot.
Marcelo: Yeah, that's right. Once you start talking about visuals, you're already in concrete execution mode. People are going to be critiquing and thinking about the flows and the UI and everything else. You want to do that, for sure, but before you do that, you want to make sure that everyone is on the same page about the North Star, the strategy, the problem you're solving, the persona—everything involved with that, like what jobs the customers are trying to do—before you jump into creating.
The PRFAQ should take a week or two before you start the project. I call this Day Zero. It's not even Day One; it's a framework for Day Zero. And often what happens, not necessarily in agencies but in organizations, is you write a PRFAQ and then you decide not to do the project. Because as you learn more, you realize, "You know what, we have other priorities," or "The opportunity is not what we thought," or "The customers told us that the biggest problem they have is not this one, it's that one." So you pause that PRFAQ and move on to another one.
Ana: How do you think that framework fits in with other popular frameworks for product management? Is it complementary, or can you skip some of them? How does it fit into the toolkit?
Marcelo: It's very complementary. In the book, I talk about how you use the PRFAQ with some of these other frameworks to generate ideas. The PRFAQ isn't going to make you generate ideas; it's not a framework for that. It is a framework to decide, to capture, and to know what you don't know. As you write your PRFAQ, you realize, "Wait a second, we don't know exactly this particular aspect; we need to go use some framework to discover that," including what the solution is. So you can use design thinking, brainstorming, and many other frameworks with the PRFAQ.
The "Four Horsemen" of product strategy failure
Ana: You have a concept of the "Four Horsemen." I really like it because it explains why product strategies fail in a conceptual way. Can we dive deeper into some of them? The first one is ignorance. Maybe you have some recommendations or approaches to fill in this ignorance part.
Marcelo: Yeah, so I've been in many, many meetings where big strategic decisions were made. And a year or two later, things fail. And sometimes they fail miserably. And then you walk back and ask, "What mistake did we make? What were our blind spots during that time?" And that's why I wrote that "Four Horsemen" blog post—to say, "Well, I've seen these patterns."
They're really about:
- Ignorance: Ignoring what you don't know.
- Arrogance: Assuming that you have the solution and you know for sure.
- Misalignment: People leave the meeting feeling like they understand what they're doing, but no one really understands. So each one goes in their own different direction.
- Short-termism: Thinking, "We're going to sacrifice certain things for now to get the benefit later." Usually, that doesn't work, particularly if you talk about customer value. "We're going to make it worse for our customers today, but it's going to be better in the long term." The customer is just going to leave.
I think the antidote for these things is really two things. One is empathy, particularly for the customer, but also empathy for your teammates. The second one is humility. Try to approach problems from a perspective of humility and curiosity. Don't say, "I know for sure." Say, "I assume," or, "This is a hypothesis." When you use that kind of language, everyone feels comfortable questioning things. You have to have some conviction in what you're doing, otherwise you're never going to get off the ground; you'll have analysis paralysis. But at the same time, if you don't have conviction while also being open-minded and curious about what's possible, you're going to fall victim to one of these Four Horsemen for sure.
AI's threat to empathy
Ana: Right now, many user researchers are being laid off due to AI, because you have instruments that can collect and process customer feedback, bugs, and so on. Transcript processing can also be done by AI, but it doesn't build empathy. Empathy is not only about having data about users but also analyzing and feeling it—basically, how you relate to people. Do you think that AI can threaten this core principle of building products with empathy and understanding real customer pain points and needs?
Marcelo: I do think so. One of the differences we make when we do UX research or any type of customer discovery is that we pick up on signals that are not necessarily what we're looking for, but they end up being very important. Many companies got started doing one product, and then halfway through, they learned about a bigger problem or a bigger opportunity. A lot of the "aha" moments we have during product discovery come from that weird moment when someone says something that seems irrelevant—and maybe AI would ignore it completely because it would say it's off-topic. But then it keeps you thinking. You go home, you start digesting that, and you wonder, "Wait a second, if that's true, that could be a big problem. Are there more people like that?" And then you take the product in a whole different direction.
I'm sure AI doesn't do that today. I'm not sure AI is going to do that in the next couple of years. Maybe later, if we train AI to observe those anomalies in customer discovery. But today, AI is going to give you the average. It's going to average out all the customer feedback, and you're going to build the most boring product.
AI's impact on user overwhelm
Ana: You also mentioned product discovery, and it resonates with the continuous discovery loop. It's really hard to build.
Marcelo: It is really hard, and I'll say something else. AI is making designers more productive, engineers more productive, and product managers more productive. What I think is going to happen is we're going to ship more features faster. It's going to be an even bigger challenge to measure if these features are having the right impact or not, because now you're doing a lot more in a shorter period of time, without giving customers the time to digest and process it. It's going to be a very interesting time in the coming years to see how customers react. Customers are already overwhelmed. All products are changing too fast, too much, and they're like, ‘Whoa, whoa, whoa.’ I think it’s going to get worse before we learn how to do this the right way.
Ana: Yeah, you raised a great point. Cursor has releases every two days. So every two days you have something new, and sometimes it changes the way you work. There's a learning curve and you're always in a state of flux where everything is changing. And customers don't like that.
Marcelo: I love a UI change that gets better, but when a product keeps changing every month, it makes me frustrated, because it takes time to get used to a new product and your patterns.
Ana: Yeah, definitely. That's a hot topic. I don't know how it will be solved. Maybe the changes will be aggregated and sent in batches. Hopefully, yes.
Tactical validation with AI prototyping
Ana: One more interesting topic: AI prototyping and validating ideas early. What do you think is the impact of this new opportunity, and should product managers and designers use it?
Marcelo: I think AI prototyping is great for at least two things. One is to validate the feasibility of certain solutions. Like, can we do it? If you can build a little bit of code that proves it, then you eliminate a lot of concerns. A lot of feasibility is about the user experience, but also about the API or data that is or isn't available. You want to make sure that you can do what the customers want. So it's great for feasibility validation.
The other one is usability—really, to think, "Can we build this in a way that customers can use it or not?" And I think AI prototypes like Vercel's v0 and Replit are great for articulating a feature in a way that isn't like a PRD, a document, or just a static mock. It makes it really hard for people to understand how it's going to work otherwise. These are very tactical approaches; they're not about strategy. They're about, "Hey, just before we decide to do this feature or not, should we spend a day or so seeing how this could play out?" And often you discover things that you wouldn't have discovered otherwise. It's so hard to design for all the use cases and corner cases, but once you have a bit of code, you catch these things really quickly.
Ana: With AI advancement, we can actually have top-notch personalization. It will create a headache for the implementation team, but it's possible. On the other side, how can the product adjust to your needs as your usage evolves?
Marcelo: I'm really excited about generative UI—the idea that the UI is going to change based on what you're trying to do. I've seen some demos recently that were really cool, which were a mix of chatbots and UI on the fly. I think we, as a society, don't know how to do that right yet. I think it's going to be a lot of problems in the coming years because you're going to see a lot of dynamic interfaces. Menus are going to appear and disappear. Boxes are going to appear and disappear. And it's going to be hard for people to understand or for us to develop the visual language to make this right. It takes time. It took 10 years for mobile apps to look like good mobile apps after they were just mini websites. So I think we're going to go through that phase as well. Maybe not 10 years, but it's going to take a few years for us to do this right.
Ana: Yeah, I have two concerns about that. As you mentioned, most AI solutions are chat-based. My concern is that they will all look the same—you start with a chat and then it provides you the output. So it'll be really hard to stand out. And the other thing is unpredictability and lack of control. Just recently, I used a tool, Granola, that takes meeting notes. It has the ability to capture your notes, and then AI fills them in. I was very frustrated because I thought it just rewrote my notes, and not in the way that I wanted. AI is a black box. No one knows how it works. So if we delegate everything to AI, how will people understand that they are actually in the driver's seat, not AI?
Marcelo: Yeah, the interfaces that we have today are very primitive. We're going to see a big development in that arena in the next couple of years, for sure.
A framework for delegating work to AI
Ana: Can you recommend for product managers and leaders where it is reasonable to leverage AI to make their work more efficient or speed it up?
Marcelo: Yeah, I have an opinion about it. I'm saying it's an opinion because it might work well for me and might not work for other people. But I think that everything we do can be divided into three different buckets.
The first bucket is what I call research, brainstorming, and planning. You are working with AI to get ideas, to think about how to approach something, or to outline the steps to do something.
The middle part is the doing or executing. Write the code, write the document, do the design, book the flight—whatever you're doing, that's in the middle.
And the last part is feedback, critique, and reviewing. Testing the code, testing the user experience, or giving feedback on a design.
You can use AI for any one of these three things or all three things. I think the problem happens when people ask AI to do the middle part, the "doing," for things that they should be using their own brain to do. And writing is one of those. Another is customer discovery. When you're writing, your brain thinks differently. If AI is doing the writing for you and providing you with the end result, and you say, "I'm just going to do the final editing," AI is going to provide you with something that is very convincing. But you didn't put the thought behind it. You can't articulate it well, you can't defend it, you can't question it. You don't know if it was the right content in terms of strategy or direction. But if you do the writing yourself, your brain goes in different directions, tests different things, and writes some sentences, and you go back and erase them. It helps you think.
So there are some types of writing, like a PRFAQ, that are strategic. You don't want AI writing it for you. You can use AI to help you, even to review it, but you want to do the writing yourself. A love letter is something you want to write yourself, so you communicate exactly what you're feeling and not what AI thinks you're feeling.
Then there's another type of writing where there is no value in the writing process itself. If you're synthesizing the content of a meeting, AI might be great for that. For the most part, it's a very mechanical job. Writing your resume is something that I don't think people need to think a lot about. They are capturing a bunch of information in a style that is good for recruiters and systems to evaluate. So AI can do a really good job of writing for that.
So what I say is: use AI for everything, just avoid the execution or "doing" part where you should be doing the thinking yourself.
Ana: This reminds me of a lot of PRDs or project descriptions that we get from founders that really look AI-generated. And I'm thinking, this is not cool.
Marcelo: Yeah, a PRD is an interesting document because I think there's a portion of it that is very strategic in nature, so you should be doing the writing yourself. And there is a portion that is more mechanical. You can use AI to do that for you, but you should work on the first part really well first before you go to AI for the mechanical parts.
Resources for strategic thinking
Ana: What would you recommend for someone starting on the journey of thinking strategically? What resources—books, articles, podcasts—would you recommend for them to get inspiration and to learn more about how to frame perfect strategies and align teams?
Marcelo: Yeah, so I think there are several books that I can recommend about strategy. I like Playing to Win and I like Good Strategy, Bad Strategy. Good Strategy, Bad Strategy is probably one of the best ones out there. In my book, The PRFAQ Framework, I talk a lot about strategy and innovation as well. So those are good ways to learn about it. The other thing is, if you can, sit in a room where strategic decisions are being made so you can learn by observing, particularly if they are good people making those strategic decisions.
Ana: Yeah. Thanks a lot for all the insights and practical advice that you shared. Very valuable. I will try to incorporate it. As you mentioned in the article, start not with a big project, but by creating a PRFAQ for something smaller and trying to iterate. So I promise I'll try it, maybe next week, to get our team aligned. We have trouble aligning and understanding who our customer is and who we're building this solution for. We're six months into the MVP and are still in this strange mode. So I promise I will try it out, and I think it'll help us align. Thank you very much for sharing and for this thoughtful adaptation of the framework and for spreading the word about it.
Marcelo: Thank you for having me.
Ana: Thank you.
If you'd like, you can buy Marcelo's book here.
