Brought to you by CustomerIQ, the AI platform to automate CRM data entry, surface opportunities, and provide actionable insights to your whole organization. Learn more here.

Superhuman, the email service known for its speed and efficiency, has always been at the cutting edge of innovation. Founded by Rahul Vohra in 2014, Superhuman has helped users tackle their inboxes with unprecedented speed, cutting down hours of email management each week. But as artificial intelligence emerged as a game changer in the tech landscape, Superhuman had to evolve, and fast.

It wasn’t just about keeping up with the competition—it was about maintaining their core promise: delivering the fastest email experience possible. That’s when Raul and his team made a bold decision to pivot their focus toward AI, partnering with OpenAI and fundamentally transforming their approach to email.

We had the opportunity to sit down with Rahul to hear the inside story behind Superhuman’s foray into AI and how it has redefined the company’s mission. Here’s what we learned:

• Why AI became inevitable for Superhuman
• The decision-making process behind their AI strategy
• How Superhuman implemented AI features that lead the market
• Superhuman’s “Theta Mode”
• Key learnings from their experience with AI

Please enjoy our conversation with Rahul Vohra, Founder and CEO of Superhuman.

Tell us a bit about how the idea of AI first came into the picture

I actually remember it vividly. I was on This Week in Startups with Jason Calacanis back in October of 2018. Jason asked me, “What does the future of email look like?” After thinking about it, I told him I saw email becoming incredibly collaborative. In the not-so-distant future, I imagined people sharing conversations, commenting, assigning tasks, and working together inside their email clients. And even back then, I knew AI would be a big part of that. I told him, “AI will summarize your emails, draft replies, schedule meetings, and maybe even send fully written emails on your behalf.” That was six years ago, and we always knew we were heading toward that day.

When did you know it was time to take AI seriously for Superhuman?

There was a pivotal moment in late 2022. It’s hard to overstate how important the launch of ChatGPT was. Everyone was talking about it, and by February 2023, it became clear that large language models (LLMs) like ChatGPT would change everything about how we work. The speed at which ChatGPT grew was staggering—it reached 100 million monthly active users in just two months, which was faster than any app in history. Suddenly, people were seeing that AI could write articles, edit text, research topics, summarize long documents, translate languages—you name it.

We started asking ourselves, “What if LLMs could help with email?” If they could summarize emails, write replies, or even handle the bulk of someone’s inbox, we knew we could save our customers a ton of time. And it wasn’t just us thinking about it—our customers were asking for these features too. In fact, AI features quickly became our top user request, far outpacing anything else people were asking for.

How did the team react to this new opportunity? What kind of challenges did it raise for you?

Honestly, the opportunity was there, and the demand was crystal clear. But it immediately raised a lot of questions. None of us had experience with AI. We didn’t have any AI engineers or machine learning experts on the team, and we had to ask ourselves, “Where do we even begin?” We had so many different options for what we could build, but we didn’t know what the best starting point would be.

Even if we had those answers, we still faced the question of prioritization. Every startup has more things they want to build than they have room for on the roadmap, and this was no different. We were suddenly faced with this massive opportunity, but how would we balance it against everything else we were working on?

So how did you go about prioritizing AI in the middle of everything else?

That was the hardest part. Historically, we prioritized features in two ways: How valuable will this be for users? and How valuable will this be for the business? Usually, that made prioritization pretty straightforward. But this time, we came up empty. We couldn’t measure how valuable AI features would be for our users because we didn’t know how well they’d work, or how they’d feel once people used them. And on the business side, we couldn’t measure the value either, because we had no idea what kind of revenue or growth they’d generate.

So, we took a different approach. Sometimes in startups, you have to take a philosophical stance. We knew, deep down, that LLMs would change everything. We couldn’t predict exactly how, but we believed that this technology would transform the way people work forever. And once we made that decision, the question became: Do we follow quickly or do we lead?

That’s an interesting choice—what made you decide to lead rather than follow?

It’s actually counterintuitive, but in many cases, it’s better to follow quickly rather than lead. Look at Apple—they weren’t the first to sell retina screens, and they weren’t the first to build Face ID. But they did both so well that we now call them “retina screens” and “Face ID.” Startups, too, often find that fast followers outperform first movers. There’s a reason pioneers are said to have arrows in their backs.

But in our case, we felt that we had to lead. We asked ourselves a simple question: What do our customers deserve? The answer was clear—our customers deserved the fastest email experience ever created, and they deserved a revolutionary AI email experience, too. We felt it was our duty to push the envelope. So, we made the decision to figure out how to build AI into Superhuman alongside everything else.

How does AI fit into Superhuman’s core value of speed? Was this something that naturally aligned with your product’s vision?

Absolutely. Superhuman has always been about speed—it’s the fastest email experience in the world. Our users save hours every week. So when we started thinking about AI, the question was: How do we apply that same positioning to AI? And we quickly realized that the core positioning still worked. Most other companies don’t share our product values. They don’t build products the way we do or hold the same things to be important.

Take something like Copilot in Microsoft Office or Gemini in Gmail. Yes, they can summarize emails, but the experience is slow. You click ‘summarize,’ and it’s not pre-computed. You have to wait while it processes, and by the time it’s done, you could have just read the email yourself! It’s kind of a useless feature because it doesn’t match the pace of how people actually work.

That’s where we set ourselves apart. We decided to push the boundaries of speed even further with AI. So we pre-compute as much as possible. For example, we have Auto Summarize, which provides a one-line summary above every email, and it’s pre-computed. Then there’s Instant Reply, which drafts replies to all your emails beforehand, so when you wake up, every email in your inbox already has a response waiting for you. And then we have Ask AI, which lets you semantically query your inbox, and we’ve already vector-embedded the entire inbox beforehand, making it much faster than traditional search.

These are things that companies like Google or Microsoft simply can’t do at their scale. When you have a billion users, it’s crazy to think about pre-computing these features for everyone. But we have a more focused user base, and since all of our users are paying for the product, we have the margin to push ourselves to that next level. When you compare Ask AI in Superhuman to similar search features in Gmail or Outlook, we’re two to three times faster. That’s a huge advantage.

It sounds like a natural fit, but implementing AI into a core product must have been a huge challenge. How did you approach that?

You’re absolutely right—it was a big lift. This wasn’t just another feature; this was about changing the entire foundation of the product. And the truth is, we didn’t have the talent on the team to pull it off at first. None of us were AI engineers, and I personally had never product-managed anything remotely close to this kind of technology. It wasn’t a question of if we should go all-in on AI, but how we would do it.

I realized that I needed to step back and think deeply about the future. That’s when I decided to get out of the office for a bit. Every year, I attend this small conference called Lobby Enterprise in Hawaii. It’s a gathering of about 100 to 200 founders and investors, and the conversations are always high-level and thought-provoking. That year, the talk was all about AI and large language models. Being out there by the ocean, surrounded by this peer group of people grappling with the same questions, gave me the clarity I needed.

I realized that the world was going to change fast, and instead of being intimidated, I saw it as a huge opportunity. By the time I got back to the office, I knew exactly what we needed to do—we had to go all-in on AI, and I had to lead that charge.

How did you make the decision to fully commit to AI, and what did that mean for you personally?

We took the leap because we had the privilege of being able to. By that point, we had already spent around eight years building the fastest email experience in the world. Our customers were getting through their inboxes twice as fast as before, replying to emails one to two days sooner, and saving over four hours every week. We had built Superhuman for every platform—Gmail, Outlook, iOS, Android, Mac, Windows, Web—it was a ton of work. But it meant we had a strong foundation in place.

We were also lucky in our timing. I had just hired our first president, Paul, who started to take on some of the executive responsibilities, freeing me up to focus on AI. We had built some executive availability and created a situation where we could afford to do something a little crazy—like diving headfirst into AI when no one on the team had any experience with it.

That’s when Paul and I developed this new mode of operation that we called Theta Mode. It was designed to contrast with our usual operating mode, which we call Alpha Mode. In Alpha Mode, project teams have high autonomy. They define their own goals and metrics and are responsible for their own outcomes. Theta Mode, on the other hand, is reserved for special situations. It’s when an executive, in this case, me, gets embedded directly into the project team as an individual contributor, providing daily guidance, feedback, and support.

Theta Mode is like the crazy card—you don’t pull it too often. But we knew that to make AI work, we had to pull it. So I embedded myself in the AI team, alongside a designer, a marketer, and several engineers. We operated independently of all the usual company processes. It was chaotic and sometimes painful, but it worked. In less than three months, we went from zero momentum on AI to launching Superhuman AI. We beat every other email client to market, and the reviews have been nothing short of amazing.

How did your customers react to Superhuman AI?

The response has been overwhelming. People are calling it a superpower. I’ve seen reviews saying things like, “Superhuman AI drafts my responses, I just tweak them and press send.” Or “Email speed has increased tenfold.” It’s really gratifying to see that kind of feedback because it shows that all the hard work and intensity of Theta Mode paid off. We managed to pull off something that even a year ago, none of us would’ve thought possible.

What did entering “theta mode” mean for the rest of the Superhuman team and its products? And how is your AI push going today?

Going into theta mode was a huge shift for us. It meant that I, as the CEO, became deeply embedded in the AI project team. I was there daily, helping to support the team, clear blockers, and provide feedback. Essentially, I took on the role of a de facto product manager. We used theta mode sparingly, only in special situations, because it requires such a concerted effort. It’s not business as usual—it’s a mode we only pull out when it’s truly warranted, and for us, AI was one of those moments.

In the past, there have been other projects that required me to step away from running the company. A good example is when I worked on product-market fit. That was a big one. I remember I took one to two weeks off from running the company to focus solely on developing that framework. I locked myself in a room and wrote for 12 hours a day until it was done. I test-presented it, wrote it up for First Round Review, and today, it’s the most shared entrepreneurship article of all time.

But AI was even bigger. It wasn’t something I could do in a couple of weeks. It was going to take months of intense focus, not just from me, but from the entire team. We had to build a small pod to tackle it. Fortunately, we had the scale, revenue, and staffing flexibility to make it work. If we had tried to do this two years earlier, it probably would have broken the company. We wouldn’t have been able to pull off theta mode for several months at that stage.

Looking back, how would you describe the phases of AI development at Superhuman?

We think of our AI evolution in three phases.

The first phase is on-demand AI. These are features that users have to remember to activate. For example, in Superhuman, we have a feature called Write with AI, where you hit Command + J, type a few notes, and we turn that into a fully written email in your voice and tone. On-demand AI features are relatively cheap to build and run. They don’t take up much space in the UI, and it’s usually easy to get sign-off on building them because they’re straightforward. So that’s where we started.

The second phase is always-on AI. These are AI features that run constantly in the background, like Auto Summarize or Instant Reply. Every time you receive an email, we reprocess the entire thread to generate a one-line summary, bullet points, or even a draft reply. This phase is much harder to execute because it requires significant infrastructure. Since launching these features, we’ve processed north of 10 billion emails. And our user base is scaling rapidly, which means the volume of emails we need to process is growing. It’s also challenging from a prompt engineering perspective. The prompt we use for both auto-summarization and instant reply is several pages long, and OpenAI told us that, at the time, it was the largest prompt in production at that scale.

Now, we’re moving into the third phase, which is agentic AI. These are AI agents that replace entire parts of your workflow. You can imagine agents that triage your inbox, schedule meetings, and even write and send fully drafted emails. These agents will be able to reason, problem-solve, and access other tools, systems, or APIs. It’s the future we’re working towards, and I believe it will free up people to be more creative, strategic, and impactful.

Do you have any examples of how Superhuman’s AI is already working in this agentic phase?

One great example is Ask AI, a feature we recently launched. With Ask AI, I can query my entire inbox semantically. For instance, after we launched Ask AI, I wanted to collect the best customer reactions to the launch and present them to the team. I did it the old-fashioned way: I ran five manual searches, read through hundreds of emails, copied and pasted the best quotes into a Google Slides presentation. It took about 20 minutes.

But with Ask AI, I can now just hit the question mark shortcut, ask it to find me the top 10 most positive customer responses, and within seconds, it synthesizes that information from hundreds of thousands of emails and gives me a bullet-point list. I copy it, paste it into my presentation, and I’m done. It’s a task that used to take 20 minutes, and now it takes 30 seconds. That’s what agentic AI looks like.

How do you see the future of AI evolving in terms of task replacement?

Sam Altman from OpenAI has talked about the length of tasks that AI will start to replace. He’s said publicly that he believes future models like GPT-5 will replace tasks that take three to five hours. And I think that’s coming. What I just described with Ask AI is an example of a half-hour task being replaced, but we’re constantly moving up that value chain. We’re asking ourselves: What’s the next one-hour task we can replace? The next two-hour task? That’s the direction we’re heading.

What advice would you give to other product teams starting their AI journeys?

I’d say two things: First, write down a clear vision statement that your whole team can get behind. Then, measure your progress against that vision. For Superhuman, we wrote down the vision: Superhuman is the most loved and advanced AI-powered email client. Customers, press, and partners consider Superhuman to be the flagship example of applying AI to email. Our customers adore our AI features and use them heavily, and Superhuman AI saves each user hundreds of hours every year.

We finessed that statement as a team, and now it’s our guiding vision for AI. We also made AI one of the company’s four strategic objectives. I think that’s really important—writing down words you believe in and sticking to them.

The second piece is metrics. Of course, we track things like ongoing cohorted retention—how sticky a feature is over time. For example, with Write with AI, we expected a certain level of usage, but it exceeded all expectations. Users are interacting with it 25 times per week on average, which blew us away.

Then there’s Auto Summarize—users are reading the one-line summary first, often not needing to read the full email because the summary is enough. We’ve been able to measure how much faster people are clearing their inbox with this feature.

And with Instant Reply, users are sending emails twice as fast. We measure the time from starting a draft to sending the email, and those using Instant Reply are cutting that time in half.

Finally, Ask AI—early users are already using it more than they use traditional search. That’s a sign that we’re on the right track. It’s all about having a vision and measuring against it.

A huge thank you to Rahul Vohra for sitting down with us and sharing his time and expertise. You can follow along with Rahul on Linkedin here.

The post Deep-Dive: Behind the Product: Superhuman appeared first on Product Collective | Organizers of INDUSTRY: The Product Conference.

Brought to you by CustomerIQ, the AI platform that seamlessly integrates with your CRM, help desk, and messaging apps to automate CRM data entry, extract valuable insights, and provide the whole GTM team with data they need to close deals faster. Learn more here.

Jason Fried and David Heinemeier Hansson, co-founders of 37signals found themselves at a crossroads. They had long been champions of simplicity and user empowerment, having created popular products like Basecamp and HEY. Yet, as the software industry increasingly embraced the subscription-based SaaS model, they felt a growing discontent with the status quo.

The duo reminisced about the early days of software, when users would pay once, install a program, and truly own it. This nostalgia sparked an idea: what if they could bring back the simplicity and ownership of those early days? What if they could create a new model where software wasn’t rented but owned outright by those who purchased it?

As they discussed this vision, they realized the potential impact it could have. The SaaS model, while beneficial for some, had become a burden for many businesses, locking them into endless cycles of payments for software they never truly owned. Jason and David saw an opportunity to lead the industry into a new era, one where customers had more control and autonomy.

Thus, the concept of ONCE was born. This new initiative would offer software products that users could buy once and own forever. They envisioned a model where the code was transparent, allowing users to host it themselves, free from the constraints of recurring fees and external control.

Excited by the possibilities, Jason and David began to lay the groundwork for ONCE. They knew it wouldn’t be easy, as it challenged the prevailing norms of the tech industry. However, they were driven by a desire to empower users and simplify the software experience.

As they prepared to launch their first product under the ONCE umbrella, they felt a renewed sense of purpose. They were not just creating software; they were crafting a new narrative for the industry, one that prioritized user ownership and simplicity. With ONCE, Jason and David were ready to lead the charge into a post-SaaS era, confident that their vision would resonate with those seeking a more straightforward and empowering software experience

I was so excited to learn that Mike Belsito, cofounder at Product Collective and our collaborators here at Behind the Product, sat down with Jason Fried to learn more about launching ONCE and its family of products.

Here’s what we learned:

• The founding story of the ONCE brand
• How they made the decision to start with Campfire as the first ONCE product
• The future of ONCE-style software purchasing
• Key learnings in pricing
• Hard-won lessons for product builders today

Please enjoy Mike’s conversation with Jason Fried, CEO at 37signals.

Tell us the “why” behind Once

One of the pivotal moments was when BMW announced they were switching heated seats to a subscription model. I remember talking with David about it, like, “What? Hold on a second. You spend around 60 grand on a car, and now you need to pay 15 bucks a month just to heat your seat? This is insane.” That moment made it clear that something was fundamentally broken in the way products were being offered.

Subscription as a business model is perfectly fine—we still have subscription products and believe in the idea—but when everything starts moving towards subscriptions, something’s off. It’s like when everyone’s buying the same stock, you know something’s up.

David and I had several conversations about this general shift. We were moving away from the cloud services we were previously paying for and began running our own solutions. Around this time, we achieved some technological breakthroughs that made it significantly easier to host software yourself. Before, it was a hassle requiring an IT person and a lot of configurations. But we figured out a way to streamline it—essentially, you’d just paste one command emailed to you into a terminal, and you’d be all set.

So, these observations about the world, technological frustrations, and a few internal breakthroughs led us to the idea of building something self-hosted. While we didn’t know at the time it would turn into what it is today, we knew we wanted to create something in this direction. That’s kind of the origin story of Once.

Why did you decide to bring Campfire back as a standalone product?

We had some friends who use Slack, and we were just curious about how much they were paying for it. When we heard it was something like six grand a month, we were shocked. They have a lot of employees, but still, we were taken aback by the amount. I started asking this on Twitter, “How much did you pay for it?” and I was just shocked to see how much money people are spending on something that is free otherwise.

The thing is, I know Slack does more than just chat, but fundamentally, chatting isn’t a high-technology product. WhatsApp is free, Signal is free, iMessage is free, Android Messenger is free. Facebook, these things are free. So we recognized a problem: why are people paying ridiculous amounts of money for what is fundamentally free elsewhere?

We started looking at chat as a commodity in businesses today. Typically, when there’s a commodity in an industry, prices come way down. Commodities are essentially identical products that are generic enough. We realized chat could be a wonderful place to start because it’s still too expensive despite being around for a long time. There are only a few chat products, and they’re all high-priced. We knew how to make something like this, and it was a concept tight enough to be a standalone product.

So, it was a confluence of various factors, but the main catalyst was realizing how much people were paying for business chat software, which made no sense to me at all. It was and still is outrageous.

Tell us more about your philosophy around software buying

The nice thing about owning software this way is you get to decide if you want it or not. There are no dependencies or contingencies on us. You can modify it to your own use—whatever you want to do, it’s yours after you buy it. Of course, there are some license restrictions, like you can’t resell it or base other products on it, but you can use it for your personal or commercial needs however you see fit.

A lot of people in the industry now grew up with software as a service (SaaS) and can’t imagine any other way to buy software. They’re used to paying a subscription fee forever, with all their data tied up in a company that could cut them off if they don’t pay their bill or goes out of business. I believe there should be an alternative to that model. We haven’t seen a different approach for over a decade, maybe even longer, so we thought it was a good idea to bring this concept back. While software you buy, install, and run yourself still exists for desktop computers, it’s less common at the business level, where everything has shifted towards services. We’re not inventing anything new; we’re just reintroducing this ownership model into the current market.

How do you see this playing out?

It’s definitely a barrier for some, I mean we’re early. We’re very early on this. This is sort of our general pattern. We’re typically very early to things and sometimes that works out well and sometimes it doesn’t. It worked out well for SaaS because we got a really early head start on Basecamp. We were early on remote work, and we were early on some other stuff, but we were also probably too early on the original version of Campfire way back in 2006. At that time, group chat just wasn’t a thing. I remember having to shove it down people’s throats. They didn’t understand why you would ever want to chat with your company. And, of course, Slack came around seven or eight years later, and then everyone wanted to do it. So we were way early on that, even though they had a great product too.

I feel like if Slack had launched when we launched Campfire, it wouldn’t have taken off either. The timing just wasn’t right. So we’re early here as well on installable software, and time will tell how it all plays out. We’ve sold thousands of copies so far of Campfire, and we’re very happy with that as an initial start. It has been a really wonderful testing ground for us to try some new ideas and build some new technology. It’s also a foundation for future products that we’re going to be building.

Right now, it wouldn’t pay the bills, it wouldn’t keep the lights on, and it wouldn’t pay 70 people’s salaries or anything like that yet. But it wasn’t intended to do that just a few months in. It was more about seeing if there was any interest in this whatsoever, and clearly, there is. So we know there’s something there. Whether or not it’s viable to run the whole company on a model like that is yet to be seen. As of now, it’s not. But again, it wasn’t meant to be. It’s funny—this is the way it used to be, and it’s funny how you could be early again to the way it used to be. I think we’ll see how that plays out.

What were some of the learnings along the way to launching this first ONCE product?

We didn’t know that we could ultimately make it as easy as we made it to install the products. It’s incredibly easy and almost ridiculously easy, but we didn’t know we could get there. We were hoping we could and it turned out we eventually could, but we had to work through some ideas to make that work. We are using a different database system than we’re used to historically, and there were a lot of things we hadn’t worked with before. We didn’t know if we could support installation on all sorts of different environments or if it was just going to require a very specific specification. Turns out you can install it on a whole bunch of things including a Raspberry Pi. You can go buy a Raspberry Pi and put Campfire on there and support something like 250 concurrent chatters, which is pretty wild.

We also had some early discussions about what we wanted to make. Do we want to break new ground in this chat product style or do we just want to nail the basics? We decided to start with the basics initially. There were other ideas about dealing with communication software that we didn’t incorporate into this product, and we weren’t sure how far we wanted to go.

One thing we were sure about initially, but didn’t know how to achieve, was the goal to make universal software. We wanted to create software with no words in it whatsoever, so it would be immediately usable in every market around the world without needing translation. The icons and UI had to be so intuitive that anyone would understand it, much like chat interfaces which are universally recognizable. In the end, we had to include a handful of words, which was frustrating, but we came up with a novel way to do on-demand translations for those who needed it.

Another big unknown was the updates. How were we going to update the software? We eventually came up with a solution to auto-update every night at 2:00 AM unless turned off by the user, ensuring the software was always up-to-date without manual downloads. This turned out to be a significant part of the product’s appeal.

We were also uncertain about the price, whether people would buy it, or if they would pirate it. The code included is open, so while you pay for the product, the code is freely accessible. We didn’t know how people would react to that. But in the end, most of the things you worry about never happen anyway. It’s a good reminder that you have to find a comfort level with the unknowns, launch, and see what happens.

What surprised you about the launch of Campfire under the Once family?

How innovative and resourceful the Small Bets community turned out to be. They switched from Discord to Campfire and essentially forked it completely, building on top of it in a very significant way. They’ve added numerous features that we don’t have and done a bunch of things with it that we hadn’t even thought of. Some of their additions are really great ideas, so good that we might want to incorporate them back into our product. Others are very specific to their needs.

What truly caught me off guard was not just the ingenuity but the efficiency of their approach. For just $300, they managed to save themselves six months of development time. So, in essence, we sold them six months of development for $300. They began with a foundational chat product and built upon it, which was just wonderful to see.

It’s incredibly smart and clever. I wasn’t expecting anyone to take this route. There have been some other inquiries too, from people wanting to replicate what Small Bets did. My response has been encouraging but clear: they have to be willing to put in the work, just like Small Bets did. Their approach was a really smart move and quite a surprise to me. I’d love to see more people do that because it also gave me ideas about the potential for building commercial products that are fully functional yet designed to be modified.

It was like selling a kit car or a prefab shed, where you get a solid base and then you can modify it as you wish. This concept of selling a headstart is intriguing. We’re offering something robust, fully functional, but also modifiable, essentially giving people a bundle of constructed raw materials. You get a great product out of the box, but if you want to add windows or a ceiling fan to your ‘shed,’ you can. You didn’t have to build the shed; it came ready, and now you can personalize it. Seeing what Daniel and his team did at Small Bets made me ponder if there’s a viable business model here. It might just be, at the very least, it’s a very interesting idea.

Will this be the de facto model for 37signals as you continue to launch products?

We’re working on two more products right now and we’re thinking about what models make sense. It could be one, the other, or both. It’s possible that maybe we can make something that would work both ways. I don’t really know. There’s a lot of complexity involved with that because there are many things you take for granted with a service, like even simple things such as notification systems and email. If you want to blast an email notification to somebody, the service model is simple, but in the ONCE model, you also have to install a mail server, and that’s a more complicated level of installation. So you’ve got to configure that. There are some complications there, but we’re going into these products with open eyes. We’re just going to build the best things we can and as we get deeper into them, we’ll start to decide which way we’re going to go or if we’ll go both ways and we’ll see what happens. Maybe there’s a third model. I don’t know.

I mean, there just might be other things that come about. I’ve always been curious about a once model service where you pay once for a service and then it’s yours. Or once you’ve paid us enough, like let’s say you paid us $5,000 over the course of using Basecamp or whatever, if you hit the $5,000 mark, it’s yours. We’ll never charge you again. Now, we don’t do this currently, but it’s something I’ve thought about. Maybe we could do something like that where once you’ve paid enough, we’re just happy to have you forever at that point. Again, we don’t do this, but there are other possibilities here that we could explore over time.

I’ve always been interested in business models. Back when we were a design firm, we did this thing called 37 Express, which was a one-page redesign. Instead of redesigning someone’s entire site, it was just one page in one week for $3,500. Fixed price, fixed scope, fixed timeframe, very short. You want another page? Cool. You want us to just look at your search results page? Cool, we’ll do that. I want to stay curious and continue exploring these kinds of ideas.

Are you getting any blowback from other founders who are selling the traditional SaaS model?

Well, I mean, there are different business models. We sell Basecamp with a limiter on the price. You can’t pay us more than $299 a month, period. You could have 9,000 employees using Basecamp, and we’re only going to charge you $299 a month max. So, you can still have a wonderful business and still cap your prices at some point. We used to be $99 a month for unlimited users. Now, we have a per-user plan and a max plan. So, there are other ways to do this.

But, I mean, charging thousands of dollars a month, or even high hundreds, or low thousands—my point of view is that software should be affordable. Out of all things, software should be among the most affordable goods in the world, like it’s software. There’s no raw material costs. I mean, there’s time to build the thing, right? But there’s almost no marginal cost after you’ve built it.

Of course, you have to support it, which is where services become expensive, but I just find it to be outrageous. I don’t think software should be expensive. I have heard from some founders, but I said, “Don’t worry, you don’t have anything to worry about right now. It’s early days, and we’ll see what happens.” And hey, if it turns out to be a viable business model, like a really viable business model, you have to sell at this price point—a $299 price point, which is what Campfire’s price is—you have to sell a lot of them to make it viable to really run a business. But if it does, if it is, you can convert your product to that or offer a flavor of an installable version.

This is hopefully shining a light on what’s possible. It’s not just for us. Maybe this is a viable model for others too. We’ll be the guinea pigs. We’ll try it out and we’ll see what happens.

What advice do you have for folks working on pricing?

Well, first and foremost, the pricing has got to be viable. This is the thing I think not enough people consider. They just pick a number and think they’re good to go. For example, a price of six bucks a month isn’t going to work if you have 35 employees and only sell 1200 seats. You’ve got to find something that accommodates your cost structure because the primary goal is to stay in business. I don’t care how good your product is—if you’re out of business, you’re gone.

I actually tweeted something about this morning. People always talk about product reliability, but your product is only as reliable as your company. If your company isn’t there, the product is gone, especially if it’s a service. So, you really need to focus on making a reliable company first, before worrying about product reliability. That means finding a viable pricing strategy, baking in some profit, and considering your costs, your growth aspirations, and other related factors.

There are also some psychological price points to consider, but they still need to make sense for your business. Again, there’s often too much focus on the customer. You’ve got to think about your own company first. Can we make this pricing work for us? If we’re not here, then the product isn’t here either. This is something not discussed enough, especially for venture-funded companies that just get a bunch of money in the bank, pick numbers, and focus on generating revenue. But you can go broke generating revenue if you don’t make it financially sustainable.

So, I’d start with an economic mindset: What are our costs? How do we keep them as low as possible? This gives us the most flexibility and options for pricing.

Tell us about Writebook. Why is it free?

We initially didn’t set out to give Writebook away for free. Actually, initially, we didn’t even have a price, but we thought it might be quite a bit higher—maybe something like $799—for companies to run internal books, handbooks, run books, and internal resources. As we got deeper into it, we realized we loved this for our own personal books. We even had an employee whose father had always wanted to write a book but found it too intimidating to get online. This small book wouldn’t be picked up by a publisher, and there really wasn’t a good option for him.

This made me think that we should create something for everybody to be able to publish a book online. It should be dead simple, really clear, and really easy to use. Something you might love to use. So we thought, why not just give this away? We didn’t think individuals would pay much for this sort of thing, though businesses might pay a little here and there. Initially, we had a commercial version with a price and a personal version for personal use. We ultimately decided to give the whole thing away as a love letter back to the independent web.

In our opinion, the web is the greatest computing platform ever made. It’s free and open, and nobody owns it. We’d love to see more and more people publish things. Many people publish lots of things—blog posts, social posts, whatever—but publishing a book is hard because there’s no real native format for a book online. Sure, you can massage a CMS to make it work, but we felt we had something to offer here. Let’s just give it away; we don’t need the revenue from this. Plus, let’s expose people to the process of downloading and installing software, showing them that they can do it.

A lot of people wouldn’t try something if it cost $300 because they would doubt their ability to do it. But if it’s free, they just need to follow the directions. It takes less than 10 minutes and there are like eight steps. It will give people confidence that they can do this sort of thing, which is generally a good thing for people to have.

Any final thoughts for the product builders out there?

I hope there’s a spirit that I’m trying to share, which is there are other ways in all the things that you do. It’s very easy to fall in line and be like, well, we have to make a SaaS thing. We got to do it this way. They do it that way and we got to make our interface really dark. What’s cool right now and all the things, yeah, you can do that, but there are other ways to do things, and there’s something to me very exciting about trying to blaze a new path or an old path.

It’s kind of like when I went hiking recently. We were going down a well-maintained path and then there was clearly a path that shot off from that maintained path that wasn’t as maintained but used to be. You could see that there used to be a path there and it’s kind of cool to go back down that old path and see what was there.

That’s kind of what we’re doing here and I think that there’s a lot you can hopefully take away from that spirit. Be curious. Explore. Of course, you got to make it work, but also, I think there’s something to this idea of what are your luxuries and with those, can you share the largesse?

Basically, can you share from your luxury position? It is a real joy to do that and to see people light up and be so excited to get something that’s good for free. Most things that are free are kind of cheap and bad—not all things, but most things tend to be. We really didn’t cut any corners. We tried to make something great and give it away for free with no strings attached. It’s a rare experience to have something like that, and to be able to give something like that is also rare and it feels really good. So I think there’s something to that too.

A huge thank you to Jason Fried for sitting down with us and sharing his time and expertise. You can follow along with Jason on Linkedin here.

The post Deep-Dive: Behind the Product: Once appeared first on Product Collective | Organizers of INDUSTRY: The Product Conference.

Brought to you by CustomerIQ, the AI platform that seamlessly integrates with your CRM, help desk, and messaging apps to automate tedious administrative processes, extract valuable insights, and provide the whole GTM team with data they need to close deals faster.  Learn more here.

In 2012, MIT graduates Spenser Skates and Curtis Liu founded a startup called Sonalight, an app for voice-activated text messaging. While developing this app, they also created a piece of analytics software to understand user behavior.

Ironically, that analytics software started generating significant interest from other companies facing similar needs. Despite Sonalight’s initial traction (about 200,000 downloads), Skates and Liu realized it would be a grind to achieve significant impact. 

They decided to pivot to focus on their analytics software. And Amplitude was born.

Amplitude quickly gained traction with its digital analytics tools helping businesses optimize user experiences.

The company grew through strategic acquisitions, including ClearBrain in 2020 and Iteratively in 2021. In June 2021, Amplitude secured a $150 million investment, valuing the company at $4 billion. On September 28, 2021, Amplitude went public on the Nasdaq, achieving a valuation of $7.1 billion on its first trading day.

Today, Amplitude is a leading player in digital analytics, serving over 14,000 companies globally, including 26 of the Fortune 100.

I was so excited to learn that Mike Belsito, cofounder at Product Collective and our collaborators here at Behind the Product, sat down with Francois Ajenstat, the Chief Product Officer (CPO) of Amplitude.

Here’s what we learned:

• Advice for implementing AI in your product
• What AI means for the industry
• Lessons learned from shipping AI features
• Francois’ “Don’t give a f****” philosophy
• How Francois thinks about experimentation
• What Francois thinks about frameworks (spoiler: he’s not a fan)
• What the next few years will look like

Please enjoy Mike’s conversation with Francois, CPO of Amplitude.

What advice do you have for building with AI?

This space is moving so fast, it’s moving at a clip I don’t think we’ve ever seen before. If you’re not continuously learning, experimenting, and trying things out, someone else will overtake you very quickly. You have to get the reps in; you have to get the learning in because that’s how you get the better product.

A big part of this is overcoming fear. Just get it out there—experiment, make mistakes, learn from them. Have an open mindset to the possibilities. There are literally thousands of companies that no longer exist because they weren’t open to new technologies. For instance, look at how Snowflake and Databricks have become hot companies in the data space. They were born in the cloud. Meanwhile, older companies that optimized on bare metal saw these new technologies and dismissed them, thinking they could never catch up. But the cloud got to a good enough state for the vast majority of customers, and those older companies are now irrelevant.

Whenever I hear someone say “it will never be” or “it is bad for these reasons,” I see opportunity. That skepticism can create blind spots. So, don’t have the blinders on. Open up the aperture, have a learning mindset, and lean into transformation.

What’s surprised you about AI and product building in the last few years?

The thing that surprised all of us was the speed of ChatGPT in terms of how it captured people’s imaginations and how it changed literally every company’s roadmaps and perspective of their priorities. I mean, it’s incredible how fast that happened. I don’t think we’ve ever, ever seen anything that fast. Even the mobile revolution happened really fast, but not that quickly. In the grand scheme of things, this is unlike anything else.

What’s also been interesting to see is the number of models and the speed at which these models get developed while at the same time how you’re kind of almost having a race to the bottom. It’s now not only just about the largest models, but about which model is actually smaller and cheaper while still driving the right level of accuracy. So there’s actually more choice than there’s ever been, which can be a bit overwhelming at times.

Another surprising aspect is how last year seemed to be all about AI hype. Every company had an AI press release—it was AI this, AI that. But how much of it has actually been used in practice? How many companies are truly transformed as a result of AI? I think every company is better with AI, but the real potential hasn’t fully become a reality yet—we’re still in that learning phase.

It’s also kind of fun to liken it to the launch of the iPhone and mobile technologies. Back then, you couldn’t imagine not having a mobile app, and everyone rushed to build one. Today, you have web apps and other ways of interfacing with products, but it took a few years for new companies born in mobile to really make an impact, like Uber. Uber wouldn’t exist without mobile. So I think it’s going to be really fun to see in the next two years what new companies will emerge that are made possible because this AI technology is available.

The iPhone was big. How big is this AI movement?

I mean, think of the transitions we’ve gone through: the cloud transition, mobile, even social. Everything’s changed. Has it completely transformed how we work every day? Not really, but it’s transformed maybe how we communicate, how we build relationships, how we’re all connected, and maybe how we consume news. I think this is, again, transformational and you’re starting to see it in your daily lives. Think of the emergence of Copilots. You write an email and you have auto-complete, you have suggestions. I use ChatGPT every day to write better. So, I think summarization is one of those core use cases that we see, but the launch of agents where you can have more automation, I think will be truly transformative. Some of the AI you won’t actually see; it’ll be happening behind the scenes, but it gets smarter and smarter over time. And the way you interact may evolve. There’s still a human in the loop, but some tasks get, again, simplified and automated with the use of AI.

Tell us about how you’ve integrated AI into Amplitude’s product

I’ll tell you the good and the bad of the story. Good. Actually, no, I’ll start with the bad because that’s always more fun. It gets more spicy. We launched about a year ago, we launched a capability that we call Ask Amplitude, and the whole idea was instead of interacting with Amplitude and picking and choosing the fields that you want to analyze, what if you could just speak it or type it in and say, here’s the question that I have, and Amplitude should figure out how to convert that into the appropriate chart or the appropriate answer. I put it in the bad category because while we thought it would dramatically open up the possibilities and bring more people in, it didn’t actually do that. What it did instead was two things.

First, it actually lowered the entry point into the product, so it was kind of like the bootstrapping process that accelerated, but then people went in and they did all the other things. So there was some of that low-value drudgery that I replaced because now you bootstrap faster. Second, it highlighted where there were taxonomy problems. For example, users couldn’t actually express their questions clearly due to data deficiencies in the product. Bad data equals bad answers, and if your data is not structured in the right way, it doesn’t matter how you ask the question, you’ll never get a good answer.

On the good side of things, we started investing in what we call the data assistant. Now, as data came in, we could start applying rules—some of it’s machine learning, some of it’s more predictive rather than generative—where we could tell you, here’s the data that you have and here are some suggestions to improve it. It even provided a data quality score. For instance, if your data quality score is at 52, it’s obvious that’s not good. It showed exactly what you needed to do to improve it. As the data quality scores went up, customer satisfaction went up, usage went up, and people became more successful with the Ask Amplitude functionality. By tackling the data assistant problem first, which was highlighted by users’ interactions with Ask Amplitude, we were able to add value and improve the overall experience.

So the integration of AI into our product was very much a learning experience. It was about moving fast to learn and then iterating based on what we discovered.

What advice do you have for those thinking about integrating AI in their products?

The first thing is to focus on the customer problem you’re trying to solve. I think we sometimes get so enamored by the technology that we forget the core problems people have that you could solve using this technology. For example, Slack recently introduced some new AI features to address the problem of being overwhelmed by too many messages in various channels. People find it exhausting to keep up. Slack AI can now summarize or get you up to speed on what happened in a channel. So, you start with the problem and then figure out how the technology can come in.

In a different space, like customer service or incident management, you have all these conversations happening, and you need to do a Root Cause Analysis (RCA). AI is great at summarizing all these things and writing the RCA document that you need. It can save 80-90% of the effort. In product analytics, it’s similar—you’ve got tons of data and feedback coming in. AI can help summarize and guide the user to where they need to go, instead of having a human look under every rock to find the problem, which is slow and laborious. AI can automatically find those valuable insights and guide you there.

While AI may not have the right answer every time because you still need human augmentation, you always start with the problem. If analyzing data is too hard or laborious, especially in a complex organization, you need to find ways to get to insights faster. Can AI help with that? Yes. We started experimenting with AI in this area by asking how we can simplify processes, augment humans, and automate tasks. But you’ve got to start with the problem—fall in love with the problem first, and then figure out how technology can help accelerate solving it.

Tell us about your “don’t give a f****” philosophy

I sometimes call it “don’t give a care less.” It’s all about not worrying about other people and just doing the right thing. Really, the core of this mantra is that people often self-center themselves. They worry about how they’ll be perceived or that they’ll say something dumb in a meeting, and as a result, they limit their possibilities. I discovered early in my career that when you stop caring about how you are perceived and focus more on the outcomes, you get dramatically better results.

The “don’t give a f***” philosophy really means stop caring about yourself, stop caring about how you’ll be perceived, and instead care deeply about the outcomes and things that are outside of you. You’ll find it liberating, and it creates a kind of shield because it’s not about you; it’s about the outcome. This approach leads to dramatically better results.

This whole mantra came to me very early in my career, around when I was 22 or 23. As a new product manager at Microsoft, I was asked to present on the state of the market to the most senior people in the company. They had assembled the top 80 people, and I was supposed to present on business intelligence. I arrived late and didn’t know who was in the room. One of the leaders made a comment I disagreed with. I respectfully said, “No, that’s actually not true. This is the reality. Here’s why,” and I presented the facts. People were stunned. They wondered how I could talk that way to a senior leader.

But I spoke the truth, and what’s the worst that could happen? They’d fire me? Great. But speaking the truth started propelling me in my career. It opened new doors because I wasn’t afraid.

When I started at Tableau, my first six months were uncertain. I kept asking for permission all the time. Then I realized I was self-censoring and decided to just go do the right thing and solve problems. Suddenly, people saw me differently. I was pulled into more meetings, got a founder’s award in my first year, and kept getting promoted.

So, if you stop caring about yourself and think more about the outcomes you want to drive, you’ll get a lot more success. You’ll feel liberated, and you’ll achieve better results.

How do you think about experimentation in product management?

Depending on the kind of company you are and the scale or size of the company, you always want to have room for innovation and experimentation. You might have a core part of the product that you’re focusing on, and I don’t want to call it the “slow lane” because that implies something negative. Instead, I like to think of it as the rigorous lane, where you’re a lot more thoughtful about what you do.

In contrast, you need an innovation lane where you can explore, learn, and test out a bunch of hypotheses. That’s where you can start integrating new models, especially with AI, as those models are constantly improving. If you don’t have a mechanism to move fast, you’ll never be able to adapt to changes in the space. Whether it’s infrastructure behind the scenes or user experiences and interaction modalities, there are numerous opportunities. Staying static is a risk in itself and can be a path to obsolescence. You have to continuously learn, evolve, and experiment to uncover real possibilities for your business.

Risk in experimentation is tied to the tolerance or fear of failure. How willing are you to fail? In modern companies, experimentation often involves AB testing and feature or product experimentation. The goal is not just to find what works but to have enough experiments that fail so you learn along the way. Risk-taking is very much tied to managing this fear of failure and adopting a beginner’s mind, a willingness to learn, and a growth mindset.

Every one of us who’s built something has experienced these “oh moments”—those instances when something completely unexpected happens, like a traffic spike or a server crash. While they can feel uncomfortable in the moment, they are excellent learning opportunities. If you try to avoid risk and think through all possible scenarios, you’ll be paralyzed with inaction because it’s impossible to think of everything. You have to be willing to take some risks, even if it means stepping into discomfort, as that’s where growth happens. Of course, there are areas like security where risks can’t be tolerated and need to be handled with utmost priority, but in other areas, a bit of leaping in and dealing with discomfort can lead to significant growth and learning. I’ve had many “oh moments” in my career, and while they’re uncomfortable at the time, they’ve always led to valuable lessons and improvements.

Give us an example of when things went wrong

One particularly challenging incident occurred when we launched Tableau Public, which we envisioned as the YouTube for data. Essentially, it allowed anyone to upload any data or visualization and share it publicly on the internet. Things were going smoothly; the platform was experiencing good traffic and steady growth. Then WikiLeaks happened. Someone took all the cables from WikiLeaks and published it as a data visualization on Tableau Public. Overnight, our traffic spiked by a hundred times, and it seemed like the whole world was pointing at Tableau Public.

Our hard drives were spinning relentlessly, CPU usage maxed out, and we simply couldn’t keep up with the demand. This was before the days of cloud computing, so we were still operating on bare metal servers. On top of the technical strain, we started receiving calls from senators telling us we couldn’t publish leaked data, along with furious emails from customers questioning our judgment. We found ourselves in a two-week storm of trying to keep the servers running while dealing with officials and irate customers.

Ultimately, this chaos forced us to scrutinize our system’s behavior under extreme load and catalyzed several improvements. We developed fail-safes, revamped our terms of service to better define fair use, and refined our response plans for mitigating similar incidents in the future. Interestingly enough, in the grand scheme of our product’s growth, what seemed like an overwhelming spike in traffic then is just a typical day’s traffic now. That incident significantly raised awareness of our platform, proving to be both a marketing moment and a critical turning point for our resilience and scalability.

How do you feel about product management frameworks?

I have some strong feelings. People often fall in love with their frameworks rather than falling in love with the impact they want to create. Frameworks are meant to help you think through a problem and approach it in different ways. However, an over-reliance on the framework can become an issue when you start celebrating the framework itself instead of focusing on the desired impact. Steve Jobs talked about this as the disease of the process people, those who are so enamored with the process that they forget about creativity, the customer, and the reason behind what they’re building.

I constantly battle to ensure there’s more focus on the impact and then work backwards from that point. The key is to see how the framework can help you achieve that impact. One framework I love, which is more of a tool really, is the PR FAQ framework that Amazon uses as part of their Working Backwards process. It’s a great mechanism for articulating and thinking through problems. It doesn’t tell you exactly what to do, but it helps you think critically. I love those kinds of mechanisms—they’re super helpful. But again, even if they’re not perfect, what matters is going through and thinking about the customer problem you want to solve.

What do you think the next few years will look like?

I wouldn’t be surprised if three years from now, most of the world looks fairly similar to what we have today. However, we’ve transitioned from AI merely being a topic of conversations to AI becoming an ingredient in every single one of our products. I believe that differentiation in the future will come through the proper use of AI. If you have two tools, one with AI and one without, the AI tool will win every single time. So, as you’re building, you have to think about how you’re integrating that technology to solve customer problems.

It’s going to be exciting to see new companies emerge that disrupt the dinosaurs of the past and come up with a whole new approach to this space, whatever space that may be. You have to have a willingness to disrupt yourself, to innovate, and to move things forward. There’s a classic innovator’s dilemma: do you want to be the one disrupting, or do you want to be the one being disrupted? I’d rather be the disruptor, the winner, the leader. That’s where I want to be.

No matter what industry you’re in, it’s clear that this technology will have an impact. So, I really push people to adopt a learning mindset, a growth mindset, and lean into the opportunities in front of us instead of shying away and being scared of them. Lean in and focus on solving problems. Those are core aspects of every builder out there. No matter what you’re building, you have to get uncomfortable to grow and to innovate. If you already knew where you were going and took the safe path, well, you’re not the innovator—you’re the follower.

I think what I would tell folks is that success is not guaranteed. The only thing that is guaranteed is your ability to learn, grow, and make an impact. That’s in all of us. The more you’re open to growing and learning, the more you’ll be open to how technologies, whether it’s AI or whatever comes next, can enable you to amplify your impact. Whether you’re a founder or a product manager, always be learning, always be growing, and always be focusing on how you can have the most impact. Don’t ever settle.

A huge thank you to Francois Ajenstat for sitting down with us and sharing his time and expertise. You can follow along with Francois on Linkedin here.

The post Deep-Dive: Behind the Product: Amplitude appeared first on Product Collective | Organizers of INDUSTRY: The Product Conference.

Brought to you by CustomerIQ, the AI platform to turn calls, emails, and tickets into structured insights and answers for product and sales teams. Align every team with the needs of your customer and deliver an exceptional experience with one powerful AI platform. Learn more here.

Today’s guest is Glen Coates, VP of Product at Shopify Core. Glen leads the development of Shopify’s core commerce platform, including the storefront, checkout, backoffice, marketing, and analytics tools. Glen joined Shopify in May 2019 when the company acquired Handshake, and he has been in the Vice President role since October 2020

As of this writing, over 1.7 million businesses in approximately 175 countries use Shopify to power their business. Over the years, Shopify has transformed from a small startup into a major player in the e-commerce industry, continually innovating and expanding its services to meet the evolving needs of online merchants worldwide.

Shopify was founded in 2006 by Tobias Lütke, Daniel Weinand, and Scott Lake. However, its origins trace back to 2004 when Lütke and his friends attempted to open an online snowboarding equipment store called Snowdevil. Dissatisfied with existing e-commerce solutions, Lütke, a programmer, decided to build his own e-commerce platform.

The Shopify platform was officially launched in 2006, evolving from the software created for Snowdevil. Since then, Shopify has experienced significant growth and introduced several key features:

1. In 2009, Shopify launched its API and App Store, allowing developers to create applications for the platform.
2. In 2010, Shopify released its first mobile app for iOS, enabling merchants to manage their stores on mobile devices.
3. In 2013, Shopify introduced Shopify Payments, eliminating the need for third-party payment gateways.
4. In 2015, Shopify went public, listing on both the New York Stock Exchange and Toronto Stock Exchange.
5. In 2017, Shopify expanded into brick-and-mortar retail with its iPad-based point-of-sale system.

As of 2024, Shopify has grown to become one of the largest e-commerce platforms globally. Some key statistics that demonstrate its scale:

• Over 1.7 million businesses in approximately 175 countries use Shopify.
• Shopify has over 8,300 employees as of December 31, 2023.
• The Shopify App Store features over 13,000 apps.
• Shopify’s market value was estimated to be around $100 billion in 2024.

I was so excited to learn that Mike Belsito, cofounder at Product Collective and our collaborators here at Behind the Product, sat down with Glen to dive into how Shopify builds product.

Here’s what we learned:

• What it’s like to ship product at Shopify
• This year’s theme of “unified”
• Shopify’s new AI Assistant
• What it’s like implementing AI into existing products
• The unique “fresh” perspective for Shopify product teams
• Advice for PMs starting to work in AI

Please enjoy Mike’s conversation with Glen Coates, VP of Product at Shopify.

Tell us a bit about your background and what brought you to Shopify

I’ve been at Shopify just about five years now, which is a pretty cool milestone. I joined Shopify through an acquisition in 2019. Before that, I ran a startup in New York called Handshake, which was a commerce platform like Shopify, but it was exclusively focused on wholesale. We were acquired to help make Shopify better at wholesale, which I’m proud to say that we have actually done now. And before that, I was a video games programmer and a computer scientist and all that kind of boring, nerdy stuff.

What impact does Tobi (CEO) have on product?

Tobi occasionally shows up with what he wouldn’t describe as mandates, but I would definitely describe them as mandates. He’ll come in with these great ideas that he thinks we should do, and although it can be a bit of a scramble, it tends to be great in the end. For example, in 2022, he noticed that even he, along with a few of us, was finding it hard to stay on top of what was actually changing in Shopify. People would ask us about the big things we shipped over the last year, and honestly, we’d look around and think, “I know it was a lot of stuff, but I can’t quite remember.”

This led us to realize that we weren’t doing a great job of telling the story of the product—how it’s changing and getting better for our merchants. With millions of merchants across different industries and countries using Shopify in various ways, it’s clear that not every release is relevant to every merchant. So Tobi’s idea pushed us to think about how we could package up all the updates and make it easier for our merchants to see what new features had come out in a way that is relevant to their specific needs, whether they’re in retail or wholesale.

It’s so easy in product to get trapped in a cycle of “build the thing, ship the thing, next.” Sometimes you just need to stop and celebrate the wins a little bit. One way we’ve done this is through Editions, which has become a bit of a celebration for us. It’s a way to show how the product is getting better in really cool ways and also a way for us to flex our web storytelling muscles. Shopify is a web-first company, and we pour a lot of love and care into these Editions pages, using new animation, 3D tech, and other innovations to make each page a love letter to the web.

What’s this year’s theme?

This year’s theme is “unified.” We chose this theme because our software has grown to encompass a wide range of functions: we operate online and in-person through point of sale, cater to B2B and direct-to-consumer (D2C) markets, and provide solutions for both domestic and global operations. We also support various platforms, including desktop and mobile. This diverse functionality is fantastic for merchants who can expand their business capabilities through a simple click in Shopify, eliminating the need for multiple software purchases and complex IT integrations.

However, the value of being a one-stop shop for all these operations only truly shines if the features work seamlessly together. For instance, if a merchant uses point of sale and subscriptions, they should be able to handle subscriptions directly through the point of sale. Similarly, if they’re dealing with B2B and use a specific payment method, those should be compatible.

A prime example of this integration is our headline feature for this edition, “Markets.” We’ve evolved the existing Markets feature, which used to simply manage regional settings—like those for France or Germany—to now encompass a broader go-to-market strategy. This includes wholesale, retail, and other market segments. With the new command center in the Shopify admin, merchants can view and control all their different settings in one place, simplifying both the setup and management processes. This central hub helps tame the complexity of their business, allowing them to concentrate on what truly matters: running their business efficiently and effectively.

Ultimately, our goal with this unified approach is to make it simple for someone who just wants to sell products, like candles, to focus on their craft without getting bogged down by the complexities of software management.

Tell us more about Sidekick

We’ve actually rolled out Sidekick, which is our AI assistant, to a few thousand merchants so far, and we’re progressively extending it to more. The really cool thing about rolling Sidekick out is that we had all these hypotheses about what people would want to do with it—like creating discount codes, changing theme colors, and so on.

What’s been fascinating is observing the actual conversations people have with the AI assistant, and some of it isn’t even software-related. For instance, people ask questions like, “What could I do to grow my business?” or “How do I find more customers?” We’ve encountered some unexpected scenarios too. For example, there was an instance where payments had failed to process, so orders were received, but the payments weren’t captured. Merchants were seeing the orders were in, the payments were authorized but not finalized, and they actually asked Sidekick for guidance on whether to ship the orders or not.

It’s almost like having a co-founder with you, where you can ask, “Hey man, what do we do here?” This has provided an interesting lens into entrepreneurship, revealing that many merchants are doing it alone and may not have a co-founder to bounce ideas off. So much of Shopify is about delivering functional software, but Sidekick introduces a very human element by offering an open-ended feature, and it was hard to predict how people would actually use it.

How is AI shaping the Shopify roadmap?

Right now, best practices for building with LLMs and training models are evolving at breakneck speed. You can quickly create compelling prototypes that seem quite smart and sophisticated. However, transitioning from a 70% quality prototype to a full production-ready model at 90-95% quality is surprisingly tough. It’s not like the traditional software development curve we’re used to; it’s brutal and challenging in unique ways.

The pace at which AI models and techniques are advancing necessitates a different mindset. It reminds me of when video games like Doom or Quake first came out—they were designed knowing that the current hardware couldn’t run them well. Yet, developers anticipated that future graphics cards and computers would catch up, making the games run smoothly eventually. We’re in a similar phase with AI. We need to build with an awareness that today’s tools might not fully support our vision, but soon, as technology progresses, everything will come together more efficiently. It’s a little uncomfortable, like skating to where the puck is going, but we’re confident that improvements are on the horizon that will enhance what we’re building now.

What’s unique about product management at Shopify?

We’re currently in the middle of a major release, which is a substantial upgrade to our theme platform. This update includes new style support, and while it might seem simple, it’s a significant improvement for users who want to customize their theme and storefront aesthetics. This project has been quite the journey; we thought we had nailed the technical aspects a few times, only to realize through feedback and challenges from Toby that we could do better.

Each iteration involved going back to the drawing board. This is something that happens often where PMs and engineers get so used to the complexities of a problem that they become numb to it. You stop seeing it from a fresh perspective – how confusing it might actually be for someone encountering it for the first time.

These “go back and make it better” sessions with stakeholders like Toby were crucial. They often highlighted that what seemed simple to us wasn’t actually simple for fresh eyes. I’m incredibly proud of the team for taking that feedback seriously, revisiting their work multiple times, and pushing to create something we can all be proud of.

I can imagine they’re crunching away as we speak to get it done. When you’re part of a company that’s performing well in the market, like Shopify, it’s beneficial to remember the hustle from startup days or from times when a company was in a tight spot. It’s about pushing ourselves and meeting deadlines. Even though missing this particular edition wouldn’t be the end of the world as there’s always another edition coming up, it’s important to push and get things done. I really commend the team for their dedication and drive on this release.

What advice do you have for those starting to implement AI in their product?

Recognize that LLMs (Large Language Models) are essentially text in, text out machines—they excel at generating text based on the input they receive, but they can also “hallucinate,” meaning they sometimes produce outputs that aren’t accurate or grounded in the data.

The easiest and most practical place to start is with applications where there’s a lot of text content that users find annoying or challenging to navigate. For instance, you can use an LLM to make your help documentation more accessible and context-sensitive. Every business has an enormous amount of help docs that are tedious for users to sift through to find answers. LLMs are great for this—load all that information into a vector database to help your users get quick answers. This approach will give you your first experience with some fundamental concepts like vector search, prompt engineering, and evaluations.

By starting with these simpler implementations, you can learn important basics and develop a foundation in using AI. It’s like taking baby steps—get your sea legs in these low-risk scenarios before moving on to applications where mistakes could have serious consequences. For example, don’t let the LLM perform actions like clicking the save button on something that could potentially cost you money if it goes wrong.

What should PM’s think about in their journey into AI?

In the journey into AI, product managers should think about how AI can enhance content generation and media creation, like the way we’ve implemented AI for image editing, helping users create professional-looking products. We’ve already seen millions of these images edited and live on our platform, which is pretty exciting.

Another critical aspect to consider is how AI can bridge the gap between novice users and advanced users. This has always been a challenging balance to strike. For example, at Shopify, we have a segmentation editor that’s incredibly powerful but also quite complex since it involves writing code expressions. With AI, users could simply write their needs in plain English, like “my customers in New York who’ve spent more than $5,000,” and the AI translates that into the appropriate code.

This smoothens the user adoption curve and helps progressively disclose complexity more effectively. Another thing for PMs to consider is how the integration of AI operates at the system level. For platforms like Shopify that serve as hubs for various business applications, it’s essential to determine at which layer of the stack AI should be integrated to ensure it connects different apps and features seamlessly.

Moreover, AI can facilitate interactions among different tools in the platform by selecting the right tools for specific jobs or even combining multiple tools to complete complex tasks. As we continue to develop and invest in our AI initiatives, especially with projects like Sidekick, there’s tremendous potential to explore these integrations further.

Any parting advice for product teams?

If there’s one piece of parting advice for product teams, it’s to really double down on the value of having a tightly integrated product. It’s easy for product managers to focus on shipping new features and overlook the importance of ensuring that these new additions work seamlessly with existing functionalities. This might seem like the unsexy part of the job, but it is incredibly important to users.

Additionally, this is an intriguing time for software teams to experiment with and learn from new technologies like AI and large language models (LLMs). No one has all the answers yet; we’re all figuring things out continuously. Sometimes we hit roadblocks and get frustrated, thinking maybe we’re behind, but then we talk to partners at OpenAI or Google and discover we’re actually right on the cutting edge. So, be a part of figuring it out. This is an excellent opportunity to understand the nuances and weirdnesses of these new tools and learn how to incorporate them effectively.

A huge thank you to Glen Coates for sitting down with us and sharing his time and expertise. You can follow along with Glen on Linkedin here.

The post Deep-Dive: Behind the Product: Shopify appeared first on Product Collective | Organizers of INDUSTRY: The Product Conference.

Brought to you by CustomerIQ, the AI customer intelligence platform to help teams aggregate, search, and synthesize their unstructured customer data. CustomerIQ unlocks insights from channels like CRM notes, surveys, support tickets, and call transcripts, to help drive retention, product improvements, and revenue. Align every team with the needs of your customer and deliver an exceptional experience with one powerful AI platform. Learn more here.

DoWhatWorks was founded by Andres Glusman and Will Howard, both of whom led product and engineering teams at Meetup for nearly a decade. They launched DoWhatWorks in 2021 after spending two years building an engine that can detect and analyze thousands of product and marketing tests run by over 1,500 companies online each day.

The idea behind DoWhatWorks stemmed from Glusman and Howard’s experience at Meetup, where they ran numerous experiments and realized the value in learning from what works for others instead of constantly reinventing the wheel. They sought to create a platform that could aggregate and share successful experiments and strategies across companies.

In the early days, Glusman and Howard went through a painful process of sharing their product and pitch across various sectors and customer types to refine their messaging and identify their target audience. They eventually found traction with clients like top streaming services, Fortune 500 SaaS companies, and meal subscription companies, who provided valuable feedback to improve the platform.

The founders named the company “DoWhatWorks” because they believed businesses don’t need to reinvent every aspect but can learn from experts and apply proven strategies to their situation, saving time and money.

As of 2023, DoWhatWorks has amassed a vast collection of experiments across areas like homepage design, pricing, marketing messaging, and sign-up flows.

This week Mike Belsito, cofounder at Product Collective and our collaborators here at Behind the Product, sat down with Andres to learn about the origin and vision for DoWhatWorks. 

Here’s what we learned:

• How Andres validated DoWhatWorks before building
• The moment it became more than an idea
• How they’re building the Grammarly of UX
• How one feature almost killed their product
• Feeling product-market fit
• What Andres looks for in new hires
• The future of DoWhatWorks

Please enjoy Mike Belsito’s conversation with Andres Glusman, Co-Founder and CEO of DoWhatWorks.

What inspired you to build DoWhatWorks?

As entrepreneurs, we realized we could be the solution to our own pain and solve it. The spark for this idea goes back to my days at Meetup.com, where I first fell in love with the process of running experiments and seeing the significantly better results they lead to. However, this experimentation always came with a side of failure, which can be grueling.

The idea for DoWhatWorks was a sudden flash of inspiration that struck while I was working on a different project. My co-founder, Will, was on board from the get-go, and we created a rough prototype within days. When we started showing this prototype to contacts in the product world, they found it captivating and were even willing to pay for it. This validation was crucial; it motivated us to build an actual product instead of pursuing traditional jobs.

We quickly put together a basic online dashboard and delivered it to our initial users. From there, things took off in a big way. We turned down other opportunities and poured all our energy into building DoWhatWorks. Since that point, it’s just been about scaling our product and serving our growing user base.

How long was it from that founding moment until your first stripe payment?

It was probably about two months, maybe three months. During that time, we were playing with a lot of different ideas and hadn’t fully committed to just one yet. We could have done it faster, but we spent a lot of time in early explorations and trying to prove out various concepts, especially when it came to gathering or detecting the experiments. We were deep in the weeds, really trying to make something valuable before building the technology around it. Will and I aren’t afraid to roll up our sleeves and do whatever it takes to create the experience first and then figure out the technology. Those first few months involved a lot of manually moving things around just to see what we were working with.

What was that moment for you where you realized this was going to be more than a cool idea and you were onto a big business?

I had just exited Meetup and had some money in my pocket. My wife, knowing my entrepreneurial spirit, told me, “Okay, you’ve got one year. I know you want to go build something new. You either get traction by the end of that year or you go get a real job.”

Right around then, we were starting to get the pieces to come together, and for whatever reason, recruiters started calling me out of the blue. I was getting these ridiculously cool job offers that were offering me boatloads of cash. But then, when we got those first three stripe payments—those three little credit card charges on a recurring basis—it felt like a sign. Although it wasn’t enough to pay us yet, it gave me the confidence that I could turn down those high-salary offers and truly commit to building this. That was the moment I knew we had something big on our hands.

What is the overall mission? What’s the big vision?

From my experience in the lean startup movement, I’ve developed the belief that launching a new product or company is comparable to visiting an amusement park, where you can only access one ride at a time. By completing one ride, you earn a ticket to the next one. This is analogous to the milestones we set for ourselves. Initially, we had to confirm whether our technology functions as intended. Luckily, it did. Then, we determined its appeal through a small audience of friends, which also proved successful. Subsequently, we established that people were willing to pay for it and that we could reliably satisfy our paying customers, and continue to do so. Including whether we could scale our operation which we’ve proven by partnering with six of the top streaming brands, globally recognized banks, leading B2B SaaS companies, educational applications and so on.

Our database now includes the analysis of 20,000 experiments featuring a myriad of elements like hero images, button colors, and specific copies among others, which has allowed us to identify patterns that show us what works best, in terms of specific industries, pages, and geographical regions.

This brings us to our big vision. We’re working towards becoming something like an auto-correct for your website. We’ve used the results from our thousands of experiments to gain an understanding of what works and what doesn’t. This enables us to evaluate any webpage and detect any weaknesses, strengths, missing elements, necessary changes, and support these observations with specific experiments. Our central goal is to accelerate success with minimal resources. Everyone wants to succeed, so we’re dedicated to simplifying that process as much as possible.

Currently, we have the only database that definitively indicates what does and does not work in this realm. This data can help us train AI systems, simplifying the process of optimizing experiences online. This puts us in a unique position, as AI’s effectiveness is primarily determined by the quality of data it is trained on, and we have a rich, exclusive dataset at our disposal.

How long was it between your first payments to today?

Four years ago, we made our first transactions. And I remember it quite well – that feeling when I bought the Do What Works website, it’s also been four years this month.

How has customer feedback influenced the product since then?

The most significant change in our product has come from having a deep understanding of what our customers need and where we come from. As a team, we’ve always ensured that our customers are intimately involved in our process. This intimacy isn’t just because they are friends, but because we genuinely enjoy engaging with them to understand their needs and goals.

We’ve noticed that as we grew, we became victims of our own success. Initially, our primary focus was on proving that we could effectively identify and analyze experiments at scale. However, the downside to this is that it became increasingly challenging to find the most relevant aspects for our customers.

We invested substantial time into research and development to figure out a way to help our customers pinpoint exactly what they want. As a direct result of our interaction with them, we built alerts into our operations and improved our methods of understanding, filtering, and analyzing.

The ultimate game-changer for us came when we recognized a critical customer need. What they wanted wasn’t just data; they wanted wins. Seth Godin’s concept that “customers don’t buy a drill, they buy a hole” greatly influenced our perspective in realizing that our customers don’t just want data – they seek wins more often, faster, and with fewer resources.

About a year and a half ago, we leaned into this realization. We took our data and learned how to extract patterns and map them to our customers’ pages. We began doing this manually and soon offered it as a service, layered on top of our existing offerings.

In the last year, we found that these patterns can consistently generate multimillion-dollar wins for our customers. The ultimate question is, can you create value for your customers? We positively answered this question last year. The challenge we’re now facing is to scale this method beautifully. We’re looking to increase our analytical capacity tenfold and then multiply it by another ten. It may sound like a Herculean task, but it’s the path we’re enthusiastically treading.

When did you feel product-market fit?

That’s a great question. We’ve taken an inside-out approach, starting from the central question and working our way outwards. I think we’ve successfully demonstrated that we can identify tests and implement them at scale, and that our product makes a meaningful difference for our customers. They enjoy using it and continue paying for it, which implies high customer retention rates. We have loyal customers who have been with us for over three years now and their number keeps increasing every year. This indicates a high level of satisfaction.

These are all strong signals of product market fit. Our next challenge is figuring out how to elevate this success to new heights. I believe we’re getting quite close to that, and that product market fit is the last piece to fall into place.

However, I feel it’s important to mention that product-market fit isn’t a fixed concept. It’s a moving target and things can fluctuate rapidly. You might achieve it at one moment, but the market continues to evolve swiftly and you need to constantly find it anew. It’s like that scene in the movie Top Gun – they lock on the aircraft in front of them and fire, but it’s in their sights for just a fraction of a second. They have to keep locking onto targets repeatedly. I feel that this analogy perfectly encapsulates my own experience. Of course, the closest I’ve ever come to a fighter jet is in a movie theater, but I think it helps visualize the concept.

Tell us about a few challenges you experienced along the way

Oh, absolutely. As an entrepreneur, I grappled with challenges on a daily basis, depending on the hour of the day. We are very problem-focused in the way we think of our roadmap. We always ponder, “What’s the next thing we need to prove? What are the next two or three really substantial things we need to prove?”

There were moments, however, when the resolution of today’s issue essentially generated the next one we had to tackle immediately. I recall a time when we figured out how to dramatically scale up the ability to detect and report on experiments. We noticed our customers becoming less satisfied, because finding what they wanted became increasingly difficult. This, in turn, led to an increase in customer churn, which was disheartening.

We found ourselves doing exactly what we thought we needed to do, but in solving the problem, we created a new, more consequential one. We had to invest half a year into devising a smart search system and an alert notification system. This turned out to be one of our major breakthroughs to boost our customer engagement. Ever since we launched these features, we’ve seen a rise in system activity, constantly moving up and to the right.

How has scale impacted DoWhatWorks?

There are myriad ways to scale, but often limited resources. The key is to quickly identify the strategies most likely to work. For example, if running ads doesn’t seem to yield the desired results, do we keep investing or shift our focus elsewhere? Our approach is experimental; we aim to identify a signal, a sign that there’s something worth pursuing.

The conversation around scaling up also involves hypothesis evaluation and a lot of gut feelings. Let’s take our website, for instance, we had not updated it for three years, and we felt that by refreshing it to more accurately represent our current work, it could make a significant difference. And honestly, it’s starting to do a much better job since we relaunched it a couple of weeks ago.

However, these are only a part of the many questions we’re grappling with. Other concerns include identifying the smart distribution and partnership strategy, deciding if we should invest in ads and the like. From an entrepreneurial point of view, part of the learning curve has involved identifying the right talents to help. Until you have a clear growth strategy, it’s crucial to work with brilliant people, well versed in different strategies.

A strategy that’s worked for us involves seeking the smartest person writing about whatever we’re dealing with, reaching out to engage them as a consultant, and learning from their playbook. While the goal has always been to create our own playbook eventually, if we can learn from someone else’s experience quicker, it’s an opportunity we’ll grab immediately to avoid learning everyone else’s lessons the hard way. After all, our very name, DoWhatWorks, is indicative of our ethos; we take what works and make it our own.

What do you look for in new hires?

We seek out individuals who are extremely curious – those who are yearning to find answers, to figure things out. In other words, people with an entrepreneurial spirit who exhibit a “can do” attitude.

But above everything, and this is non-negotiable as we grow, we want “good humans”. We insist on individuals who display high integrity, kindness, and a genuine care for those around them. A candidate who doesn’t show these traits will not be a fit for our team.

As for skillsets, think of it like knives. You have your Swiss army knives, multifaceted and adaptable, and then you have your single use tools, highly specialized in one area. A metaphor I think fits better comes from the military: Marines and the Army. Marines are these nimble, versatile groups – they’re the ones taking the beach. The Army represents the occupying force that follows, focusing on logistics and maintaining operations. You need both types of talents on a company team, the Marines who forge ahead, and the Army to create order out of the chaos and maintain the solution. At present, we stand at an intersection with a lot of “Marines”, and we’re starting to hire “Army” personnel as well.

How did your experience at Meetup influence your work at DoWhatWorks?

Meetup had a wonderful culture that we’ve tried to borrow from at DoWhatWorks. They were extremely transparent with their employees, everyone had access to most of the financials and had a clear understanding of the vision. Meetup was a place where people looked out for each other and a lot of thought was put into finding talent from different pools to ensure diversity. There was definitely room for improvement on certain fronts, but on a cultural level, they did a lot right.

When it came to strategizing, one lesson I took from Meetup was the importance of having a revenue model from the very beginning. Meetup launched with a model where restaurants would pay to host meetups, proving that there could be an initial revenue stream. However, the service then became free for a period of time and switching back to a fee-based service was extremely difficult. I absolutely didn’t want to go through that again. That’s why with DoWhatWorks, we aimed to be revenue positive from day one.

Furthermore, during my time at Meetup, we upheld a principle of spending money as if it’s your own. We operated in a very capital-efficient way, raising only around $20 million despite our substantial exit. We’ve carried this lesson over to DoWhatWorks and aim to operate in the same prudent manner.

Looking ahead, what are you most excited about?

I’m truly excited about this vision that I’ve labored over for years – this notion that wouldn’t it be stellar if we could take this critical mass of experiment data and mold it into something truly significant and impactful? That’s what we’re starting to prove, which is heartening.

I’ve plowed four long and challenging years into this journey, with the aspiration of becoming the Grammarly for UX, a system that quickly provides insights and direction to people to prevent missteps and enhance their ability to identify and execute what works more quickly, efficiently, and frequently. At the crux of it, that’s what I’m really thrilled about.

We’re so close to achieving it that I can taste the culmination of our efforts. I can visualize the entire picture coming together. Of course, it would need work to bring that vision to a tangible, seamless user experience reality. But the pace at which the world is moving, coupled with the rise of AI that has serendipitously fallen onto our plate, fuels my exuberance. We possess this treasure trove of data, and now have the perfect opportunity to utilize it and train AI to perform truly miraculous tasks. That’s what revs me up as we look ahead.

What parting words or advice do you have for the product builders and entrepreneurs reading this?

Run towards the buzz saw. Imagine a James Bond movie. At some point, James Bond is strapped to a table with a buzz saw heading his way. The fear of inching towards the buzz saw is maddening.

I’m suggesting that you need to figure out what is the buzz saw in your business. What’s the one thing right now that’s threatening to crash your entrepreneurial dreams? Is it your technology not working? Is it the worry that no one’s going to use it or pay for it? Whatever your buzz saw is, don’t shy away – run towards it as fast as you can.

If you can overcome this buzz saw, congratulations! Now, find the next buzz saw. Continue identifying and overcoming these ‘buzz saws’, as they represent hurdles in your business. If you can figure out how to solve these problems quickly, you’re going to be in a really good shape. Or if you can’t overcome it – well, good news, you didn’t waste years trying to build something that eventually didn’t work out. So it’s a series of questions, a series of buzz saws, but always remember to identify the most critical one first and run towards it.

A huge thank you to Andres Glusman for sitting down with us and sharing his time and expertise. You can follow along with Andres on Linkedin here.

The post Deep-Dive: Behind the Product — DoWhatWorks appeared first on Product Collective | Organizers of INDUSTRY: The Product Conference.

Brought to you by CustomerIQ, the AI platform to help teams aggregate, search, and synthesize customer feedback. CustomerIQ aligns teams with insights from channels like CRM notes, surveys, support tickets, and call transcripts, to help drive retention, product improvements, and revenue. Align every team with the needs of your customer and deliver an exceptional experience with one powerful AI platform. Learn more here.

Before writing a line of code for CustomerIQ we interviewed over 25 product leaders to learn how they worked and what their biggest challenges were. We were working to validate some of our assumptions around customer feedback management. 

We learned two main things during those conversations. The first, of course, was that teams struggle to aggregate and synthesize their customer feedback.

But separately, on the topic of tools these teams used and loved, Miro came up time and again. Teams we interviewed used Miro for everything from brainstorming to managing product roadmaps and seemed to love every aspect of it.

Naturally, I was so excited to sit down and chat with Anna Boyarkina, Head of Product Excellence at Miro, to learn more about how Miro builds products people love.

Today Miro supports over 60M users across 200K organizations including Nike, IKEA, Deloitte, WPP, and Cisco. But Anna has been leading the product management function in its various shapes and sizes since Miro was merely a prototype.

Over her 13 year tenure with Miro the company has grown from a fledgling startup to a $17.5B enterprise (as of January 2022).

Here’s what we learned:

• How Anna discovered Miro’s first use cases
• Miro’s “Painted picture” planning strategy
• Their AMPED team structure
• How Miro runs experiments
• The challenges with synthesizing customer feedback
• The importance of trust if you want to move fast
• What Anna looks for in new hires

Please enjoy our conversation with Anna Boyarkina, head of product excellence at Miro.

Tell us about your first role at Miro

I’ve been at Miro for almost 13 years, basically since the beginning. I initially joined on a part-time basis as the company was in its exploration phase and my skills are not engineering, but rather somewhere in the mix of marketing, product, UX. Eventually, I transitioned to full-time as we worked to figure out the business aspect. 

I started as a PR and community manager, which didn’t quite capture what I was doing, but involved a lot of reaching out to customers and trying to understand how they were using our product, which at the time was called Real-Time Board. I had to figure out who would buy it and why, so I spent a lot of time talking to customers, answering support tickets, and just generally seeking to understand their experience and needs. 

As the company started to scale up, it became clear that my skills were best suited to focusing on product development. This wasn’t a role I consciously pursued – it was more of a natural progression. I think it caught me by surprise when I realized that I was essentially doing product management.

How did you identify the best use cases for Miro?

When we first built Miro, it wasn’t initially clear what the specific focus would be. We toyed around with the idea of it being a B2C product, possibly even integrating some social network elements, for example, allowing people to subscribe to view other people’s boards. However, through engaging with our users and observing their interaction with our product, we realized that most individuals were actually using Miro for work rather than personal use. This marked our first significant insight and milestone. 

Following that, we also noted that Miro functioned more effectively as a team-based tool, rather than individual use. This influenced our perspective on our business model and product functionality. It became apparent that building a B2C product differed vastly from building and evolving it into an enterprise-level solution.

Who were your first users?

Our first audience were primarily those focused on elements similar to design thinking, especially UX researchers conducting workshops. At that time, design and customer centricity were quite the trends. So naturally, these were the people who found major use in our product, using it for things like customer journeys, facilitating workshops and so forth.

Product managers and engineering teams were also a significant user base. Instead of using complex software, they found it more convenient and efficient to use our product for visual tracking on the board – you know, kind of like good old sticky notes.

The education sector did have a notable presence, but it wasn’t the core focus of our business. But all things considered, I’d say our early users were quite a diverse group!

Were there any big inflection points over the years that led us to today?

There were three significant inflection points over the years that helped shape the Miro we know today. The first was when we switched our platform from Flash to HTML in 2015. Frankly, it was a big leap. The thing is, nobody was using Flash, and it was almost impossible to pass security protocols in enterprises with it. Plus, we had to manage multiple versions of our product as we already had hundreds of thousands of users.

The second turning point happened when we dove into the enterprise field around 2016. At that juncture, we weren’t sure if we were equipped or even had the potential to enter the enterprise market. We were unsure if our product was exclusively targeted towards small teams for miscellaneous tasks. However, our experimentation provided us with enough evidence that our product could be utilized in enterprises. So that informed our strategy for the next few years.

Lastly, the COVID-19 pandemic presented an unforeseen but impactful shift in our market. As remote work and collaboration became the norm, our product suddenly became indispensable for teams as they tried to define their shared workspace in a virtual setting. Our product essentially became their shared space.

How far out do you plan and how has that changed over the years?

When we started, planning was very basic. It was just a spreadsheet with weeks as columns, so everyone involved knew what the goals were for each week. It was every week releasing experiments or functionality and we would have company all hands meetings with 12 to 15 of us. We’d discuss what we were doing and what every person’s focus was. 

But then, a few years later, when we felt the company became more sustainable, we began to use OKRs. This became popular in the market, around 2013, so we thought, why not? At first, OKRs were on a personal level, done quarterly, but then we switched to company level and removed the personal ones.

A couple of years after that, we saw a need to think about a longer-term vision for the company. We were greatly inspired by Atlassian and admired their company culture, growth, and focus. That’s why we started planning three years ahead. We called this our “Painted Picture”, a snapshot of what we envision the company to look like in the future, covering different aspects like company culture and business direction. This then influenced our strategy and biannual OKRs.

What percentage of ideas come from the top-down vs bottom-up?

I wouldn’t definitively quantify it as 50% top-down or 50% bottom-up. What I can tell you is that our strategy largely comes from the top down, but solutions are significantly informed from the bottom up.

We somewhat operate under the concept of a double-loop model. When we put forward a direction, we invite our teams to validate it and also provide their own valuable input. It’s not so much that we follow this process rigidly, but rather that there’s always a continuous feedback loop in operation.

When we’re formulating our strategy, for example, we hold share-out sessions where feedback can be freely given. It is this input that helps inform a lot of our strategic thinking. An important point to keep in mind is that whatever strategic decisions we take, they should be appealing not just to the end-users of our product, but also the admins who are effectively the product’s custodians within the organization. 

But it’s not just about setting a direction and feeding off of internal feedback. We then take the proposed direction to our customers to try to understand the practical use cases and prioritize them accordingly. We recently launched a new product which had already seen a lot of demand and received excellent feedback during its private beta testing phase. This, I believe, is testament to the incredible work of our team and that definitely makes me proud.

How are the product teams structured?

Yeah, so in our setup, we have several product streams. These streams focus on different personas or specific use cases. One key aspect is that every product stream is united by a common mission and vision. You know, I strongly believe that teams are much more engaged when they’re working within the same domain and have a shared mission. It gives a real sense of purpose.

However, we try to ensure a product team isn’t too big. Coordinating 200 people can be a real challenge. So, as a rule of thumb, we try to keep it under 100, but admittedly, sometimes that’s just not possible.

We’re also really big on cross-functional work. Within each stream, there are three EPD leaders who work on a day-to-day basis. In addition, they have cross-functional partners in analytics and product marketing. We’ve tagged it with this term AMPED, an acronym that stands for all these functions.

Every decision is driven by these people because they’re the ones in charge, and it’s essential they’re all in sync. We experimented with different setups, with separate leaders who weren’t communicating. You’d often end up with design handing off something to engineering without context, then it’d rebound to product marketing. We learned it is more effective when including all those involved in collaboration from beginning to end. 

Everyone understands the challenges of each function, the customer problem we’re tackling, and how we’re going to solve it. Honestly, promoting this interdepartmental collaboration was a game-changer for how we operate.

What’s the ratio of engineers to PMs?

On average we have around eight engineers for every one product manager (PM). But, it’s important to underline that teams can vary quite a bit. We usually operate with one designer per two to three teams. 

And then there’s the matter of analytics and marketing – these are shared resources among the teams, and it’s an evolving situation. We’ve recently adjusted the operating model of the teams, even changing the focus of some roles. 

In general terms, it shakes out to about one analyst for about three to five teams. Product marketing roles work in a similar way. They usually zero in on a specific domain, and inside that domain, you might have one or two people working across anywhere from four to ten teams.

How are you managing behavioral tracking and how has that evolved?

From the get-go, we’ve been all about measuring everything. And that data needs to be cross referenced with customer feedback, another huge element we’ve always considered. So essentially, we scrutinize every single click, every step – you name it, we’re on top of it. 

We’re also big on experimentation; we’ve integrated that into our approach over time.  Now, we’ve got this beautiful fusion of using customer feedback to validate the signals we spot in the data, and vice versa. So, in a nutshell, that’s how we’re managing behavioral tracking. It’s been an evolution, but I like to think we’re always improving, always morphing into something better.

What’s an example of experimentation at Miro?

One interesting example of experimentation at Miro is our website experimentation. This involves changing the copy, altering how it works, how it looks and even experimenting with our brand’s visual identity. We recently did restyling with a new visual identity. This was a stage where we had to test things beforehand to ensure we didn’t see a dip in our metrics. You see, when we did rebranding or restyling in the past, there was usually an initial period where people didn’t respond very well, so it was important to keep an eye on that. 

The other facet of our experimentation is within the product itself. It could involve introducing new functionality and then gauging user response. I consider this an experiment as well, because we’re testing things to understand if it’s garnering a positive response and if the overall use case is improving. Essentially, we’re trying to ascertain if people are spending more time in the product than before and whether it’s enabling them to work faster, among other things. 

We’ve also dabbled with pricing experiments, testing different things within that area. I’m always curious to see the outcomes and learn from them.

People most often talk about successful experiments, but how do you handle failed experiments?

When it comes to handling failed experiments or features in product operations, first you need to comprehend if the experiment or feature has market potential. Then, we usually test it with a subset of customers before releasing it on a full-scale. We label this stage as ‘private beta.’ Doing so, the customers have a high-touch experience with the new feature and understand clearly that what they’re interacting with is experimental. So, even if an experiment fails or we decide that we’re probably not going to work on it further, the message and expectations were clear right from the outset.

How do you manage qualitative customer feedback and what are the challenges?

This part is definitely a challenge. I still remember the good ol’ days when we could manage feedback manually. We would manually cluster all the feedback that was given to us in support tickets and social media on a spreadsheet. The beauty of that approach was being able to directly trace the release of a feature back to the exact users who asked for it. 

However, when this spreadsheet reached more than 1000 rows, it became virtually unmanageable. That was probably a decade ago. We began experimenting with different solutions, such as Productboard amongst others, to capture the information. 

The main challenge was that these solutions still required a significant amount of manual tagging. Understanding the information and making sense of it, especially if you weren’t the person who input it, can be difficult. 

We then decided to create a customer feedback system in a BI platform, Looker. That helped but we are still exploring more efficient solutions to manage it. We get over 100,000 pieces of feedback monthly, making manual processing just impossible. 

At the end of the day, the goal is to extract valuable insights from the feedback – to understand customers’ needs and wants. And that’s challenging in an ocean of thousands of comments. So, we are looking into more efficient solutions that can facilitate this process.

How do you go from feedback to product?

Initially, we think about the strategy, looking closely at insights such as trends and patterns. We have research and insights teams that support us with analysis and interpretation so everyone can access and validate the assumptions. 

We heavily prioritize our customers and want to interact with them, delving into their insights which then shape our hypotheses. Once we have this feedback, we try to validate if it’s something that makes sense. This is done by describing your idea, usually in a product review process. In this process, our product managers, designers, and various other managers come together to propose something in what we call a kick-off. 

The aim is to figure out if the problem is worth solving, and if there is evidence to support it, such as customer feedback. If the idea is supported, the next steps include designing a solution and a testing phase where we talk to customers to validate the product. Sometimes, if monetization is a significant aspect, people are asked to pay beforehand. 

Lastly, the product goes to the user, and we aim to close the feedback loop thereafter. I don’t think we’re doing awfully well in this last area currently, but we’re aiming to refine the process and get better at communicating to those who are requesting specific features.

How does Miro use Miro?

We use it for many things, though of course, we operate within its appropriate use-cases and avoid using it for tasks it’s not designed for. For example, we use Slack for communication, even though Miro has a chat function.

In terms of specifics, we utilize Miro extensively for ideation, for strategy thinking employing different mental models, creating sticky notes and for workshops, both virtual and in person. In fact, presentations are an area where we extensively utilize Miro and chose not to use any other platform. Solution designs, wireframes, user flows – they all have a place in our Miro workflow and it aids in providing a comprehensive visual platform.

We have also implemented Miro at a micro-level, allowing teams to track tasks, particularly benefiting from the Jira bi-directional sync Miro offers. This helps teams collaborate on work in Miro, synchronize it in Jira, and ensure everyone is up to date.

How do you manage the communication of new features with users?

Our process for communicating new features is pretty sophisticated because we have a lot of teams involved and it’s crucial to keep everyone in the loop. Marketing, sales, customer success, and support teams all need to be aware of what’s going on. We also make sure our customers know, especially if they are sensitive to change management, they should be informed of upcoming changes, not afterwards. 

Most, if not all, the features are released to Miro employees initially because we want to test things first. This allows us to assess all aspects of the product, from bugs and user experience to the specific use of the product. Once the beta phase is over and it’s been thoroughly tested, we usually have a private and public beta stage before it goes into general availability. 

We don’t launch anything without completing an extensive pre-release checklist to ensure it’s accessible, compliant, localized. This is really important because we have a global array of customers with different needs. 

Our customer-facing teams are then notified about the new features. They typically have access to the customer-facing roadmap, so they understand when new things are coming. We also notify our enterprise customers via a special portal, so admins can stay updated with changes. 

It’s a strategic move we have made to prevent any unexpected surprises. It’s crucial to keep everyone informed because we have people who manage change management and if they are not aware, they can’t do that efficiently.

If there were one lesson you send everyone who wants to learn about how you think and operate, what would you want to include?

The organization moves at the speed of trust. 

Over the course of my experience at Miro I’ve worn a lot of hats. And with every new project, I’ve learned that without trust, all your energy is spent wondering who’s aligned with what, what the agendas are, and trying to understand the goals of differently incentivized people. 

So invest in trust, especially when setting up a new team or a new project. Create a safe space where we’re all pursuing the same goals. This ties into the idea of vulnerability. As a leader, it’s absolutely crucial for me to model this trait because without it, teammates don’t feel safe. I believe vulnerability is an essential leadership trait, irrespective of your title – it’s more about behavior.

Lastly, the importance of continuously learning. Look at what’s happening in the market right now. I’ve heard people say – it feels like we’re in the early days of the Internet with the advent of AI. You can’t avoid this. You have to upgrade your knowledge consistently, focusing not just on big things, but smaller ones too. It allows you to expand different avenues of thinking about customer problems and gain new insights. Keep talking to people, keep reasoning about things that might not be in your usual area of business focus. Embracing new evolutions is just a must.

What do you look for in new product hires?

I always look for curiosity, empathy, attention to detail and a strong alignment with our company’s values. 

Curiosity, because a product manager should always be driven by an intrinsic motivation to learn. Empathy, because to be customer-focused, which I consider essential, one needs to empathize with the consumers’ needs. 

We also need people with attention to detail because our product is very visual, and everything we do is seen by the customer. This focus on detail extends to the whole craft of product management. How you do anything is how you do everything. 

The final key point is alignment with our company’s values. This is not just a box to tick during recruiting or performance reviews but something that’s evaluated continuously. 

Our hiring process via recruitment and hiring manager interviews often culminates in a test case which aligns with what the candidate will potentially be focusing on. Usually, it has a visual component, like asking candidates to create wireframes of solutions, just to gauge their product craft and attention to detail. 

A favorite question of mine is asking candidates to name products they admire in certain areas and exploring why with follow-up queries. 

In the end, recruiting is a careful balance of evidence-based questioning against our four values and selected behaviors. The stories people tell us in the interview process can reveal much about a candidate’s fit with our company.

What are fun traditions or rituals you have on your team?

One fun tradition is what we call “Friday wins.” It’s a weekly gathering where the builders in our team present a quick, five-minute demonstration of what they’ve delivered to our customers through the past week. It’s like show and tell, only less salesy. It provides the team with a sense of realism and accomplishment, seeing the fruits of their labor in action. 

Another favorite event is our biannual hackathon. In our last one held in December, we had over 50 teams participating. These hackathons take us out of our day-to-day work and give rise to innovation and creativity. Being able to accomplish something unique creates an incredible sense of motivation.

We are also reviving the tradition of “fails night.” After all, it’s not all about successes – we want to normalize the concept that it’s okay to fail. This event helps us learn from each other’s missteps.

Lastly, one of the more serious, yet vital traditions is our product reviews. Though they could sometimes be stressful due to the review process, they’re key to our product-focused company. These reviews serve as a significant learning platform as colleagues get to learn from each other. What we’re currently changing is the size of the meetings, since decision-making becomes challenging with too many people involved.

A huge thank you to Anna Boyarkina for sitting down with us and sharing her time and expertise. You can follow along with Anna on Linkedin here.

The post Deep-Dive: Behind the Product: Miro appeared first on Product Collective | Organizers of INDUSTRY: The Product Conference.

Brought to you by CustomerIQ, the AI platform to help teams aggregate, search, and synthesize customer feedback. CustomerIQ aligns teams with insights from channels like CRM notes, surveys, support tickets, and call transcripts, to help drive retention, product improvements, and revenue. Align every team with the needs of your customer and deliver an exceptional experience with one powerful AI platform. Learn more here.

I started my first company out of a small coworking space in downtown Raleigh, NC called HQ Raleigh. This was before the WeWork craze. Even slightly before it was cool to be a founder. The building was full of small startups, all with different pitches and plans to build the future. We were in the far back corner, tucked away from the main common area. On the opposite side of the building, through the common area, up the steps, in a small office was a startup with a short, latin name, building software for a persona whose job responsibilities alluded most of us young entrepreneurs.

Pendo always seemed to know what they were doing. But at the time, none of us could explain what it was. Product management was still a relatively new concept. We weren’t sure if Pendo was selling software to engineers or to marketers. Of course, it was neither. We weren’t sure if it was an analytics tool or a product guide. Of course, it was both. Pendo set out to build a platform for product managers. Out the gate they built what was essentially two products stitched together. This confused most early investors but made sense to anyone who had managed product.

And it worked.

Suddenly Pendo found their stride. Soon after focusing on the mid-market they found product-market fit. In 2015 they announced their $11 million Series A round led by Battery Ventures. Their little office got crowded – full of smart, hard-working people. Then they moved across the street. Then they moved a block over to their own building. Then they moved to a new, bigger building. Then they put their name on that building. And now Pendo’s signature pink brand highlights a growing Raleigh skyline. 

The quintessential startup story.

Pendo grew to over 800 employees between 2014 and 2024, with a valuation of $2.6B as of 2021.

I was so excited to learn that Mike Belsito, cofounder at Product Collective and our collaborators here at Behind the Product, sat down with Todd to hear the origin story. 

Here’s what we learned:

• Pendo’s origin story
• Building the early team
• How Pendo got their first customers
• How having young families built a culture of resilience
• Feeling product-market fit
• Building a culture at scale
• The future of Pendo

Please enjoy Mike Belsito’s conversation with Todd Olson, CEO at Pendo.

What sparked the idea for Pendo?

The idea came from my past experiences in my career. Prior to starting Pendo, I was the head of product at a company called Rally Software. Rally Software was one of these probably early SaaS solutions or early cloud-based solutions focusing on agile project management. We had a homegrown system for product analytics that required developers to instrument things. We had a data warehouse, we had Tableau on top of it. And I constantly wanted more answers out of this system and every time I asked for more answers from the system, it was met with, “Well that’ll take more engineering time,” or “We need to instrument this,” or, “Oh, we need to bring in this data to it.” So it felt like an expensive endeavor. The last thing you want to do as a head of product is put your engineers on some internal system.  

So that was one challenge I was experiencing. The other big challenge I was experiencing was, I’ll basically describe it as how do you enable customers in a modern world? What I saw was as companies were adopting more and more agile development methodologies, they were shipping features more swiftly. It meant that we evolved as an industry from this place where we used to do big releases and we used to do big releases because we used to ship physical media. When you ship a CD, you’re not going to do it very often. I remember when I was cleaning out my garage recently, I found stacks of Microsoft CDs I used to get when they would do a release. In college, you’d get these very inexpensive sets of Microsoft software and then you’re shipping once a year, twice a year, whatever, it was going to be a big event.

But when you’re shipping a handful of features a week, you need to evolve the way you think about enablement. One of our customers had rolled out a big feature release and one of these customers called and yelled at me as the head of product, saying, “Hey, you didn’t enable my team.” Well, it’s like, look, we did a webinar, you didn’t come to the webinar. They’re like, “We’re not going to come to all your webinars. That’s madness.” So we started experimenting at Rally with in-product messages, and the idea was highlighting new features and trying to draw attention to areas of the product that people weren’t using. But the challenge was that we had no idea whether people were seeing the messages, whether they were working or not. If I call something new and you logged in this week and one of your peers logs in six months from now, is that still new?

It’s a great question. It’s new to them maybe. So I kind of saw this as another challenge and part of my vision for Pendo was: how do we create a platform that handles all of this? One, it captures the data you need to capture in order to have good insights into how people are using your product, because that just informs a lot of product management questions. Everything from “Is this feature adding value? Should we invest more here?” to even things like pricing and packaging combined with “How do we message users and drive engagement?” The ultimate vision was we want more people to use more of our software, very simply because the idea of more people using more parts of our software, that’s good for them, that’s good for us, it’ll drive growth. So that’s kind of the ultimate backdrop of how we started.

How did you build the founding team?

I mean I think with any startup, I am a fan of a founding or co-founding team. I mean, there are some stories of people doing it by themselves, but it’s always nice when you have someone drive accountability amongst yourselves. I think it is super valuable. I had worked with Eric Boduch right out of college at Carnegie Mellon. We started our first company together. But we sort of went our separate ways for years, not necessarily by design, but that’s kind of what happens. I took a job, he took a job, you diverge. And we’d always said if we had an opportunity, if our lives somehow aligned, we would love to work together again. So the good news, the world was aligning.

So that was one. Rahul (Jain) was another one who I had worked together with, he said he would love to do something if I was starting something. And the last piece of it was our CTO, Erik Troan, who a bunch of people had said we should meet. I was just always so busy and he was always so busy we had never met. But in the months leading up to Pendo, I had some time and I was talking to someone who said, “You should really meet this person for once.” So I met with him. I think he was a key part of that original founding team because you think about our backend and how much data we process and just handling the engineering side of the business. He was our CTO, just a key force. So we kind of formed this founding team and then we went out to talk and raise capital.

I think we knew from day one that this was a capital raise type of company. I think there are some companies that are very suited for bootstrapping and some probably more suited to go out and raise capital. And I think based on our backgrounds, based on the opportunity, and I think what I would describe as the nearness of the opportunity, time to market speed really mattered here. Whereas some things that are very early stage probably don’t matter as much. You can probably take a few years to incubate an idea. But given the way the world was, we felt like we had to do it now. We tried to raise seed funding for a while. We met with a lot of seed firms. I spent a lot of time in the Bay Area. I think given that we are a North Carolina based or headquartered company, I think we struggled to find funding in the Bay Area.  

I think I can use the word “struggle.” We didn’t get it. So I guess my definition of that would be struggling and I tried hard and even with track records, even with good backgrounds as entrepreneurs, it was hard. Back then, that would be 2013, finally we got commitments from a DC based venture capital seed fund and some New York based funds. We put together roughly a million, maybe a million and a quarter dollars to get started. That sort of was the catalyst to get things going. Of course, while we were raising capital, Eric Tron and I were also building the initial mechanisms for what Pendo is today. So he was building the core of the backend. I really worked a lot on the front end and everything from the data capture and everything else. So I think he and I were iterating in the background, sort of just building things out. So we had a product to actually show and start potentially onboarding customers onto.

How long did it take to raise that round of funding?

The process basically took the entire fall of 2013. So we started in earnest around early September through December 31st. We actually closed the deal on Christmas Eve, so on December 24th we were trying to get signatures and wires from VCs on Christmas Eve, which was not the easiest thing to do. But we got it all wrapped up and got the money in the bank. And then we had also, once we had commitments and term sheets, I think starting in early December we started recruiting some of that founding team. So we brought in our first product manager, Shannon Bauman, formerly of Google and some other places, as a good early stage PM, and then two engineers that we had worked with because they were very known quantities, great engineers, scrappy, loved being at a startup. So that was kind of the early days. And then starting January 1st, 2014, we were in a coworking space going and basically building all day long in this room, honestly a room not much bigger than the room I’m in now, but all of us were in there and those were fun days.

What were you looking for in your early team? How did you all go to market?

In the early days, we were hiring people that, generally speaking, we knew. We wanted very low risk hires in the early days. Although like Shannon for example, I had never worked with him before. So that was one where we took maybe a bit of a risk. But I think he had shown a proclivity towards startups after Google that really attracted us to him. And generally speaking, you want a startup oriented individual in those early days – someone who is comfortable working with a lot of uncertainty, arguably a lot of risk. And interestingly, everyone we hired actually had families. These were people that had families, they had experience, they were mature, but they were also willing to take calculated risks. And honestly, it all worked out. So in hindsight, they looked like smart hires, but at the time it was super risky.

You want someone that is also willing to speak up and do what’s necessary and not have an attitude of “this isn’t my job” or whatever. There were no set jobs. We were all sort of doing everything. And I personally wrote code for the first eight to ten months in the company’s history. I haven’t written as much since then. Eric, our CTO, still writes code today, but those were wild days. We were sitting back-to-back in a room in front of our computers, iterating on problems. We would turn around and say, “Well, how are we going to solve this problem?” We would put together something and it was like, “Oh, we don’t like that. It doesn’t look good.” And there was a lot of conflict, I think healthy conflict, but if we didn’t like certain things, there was this classic model of forming, storming, norming, and performing. Oh my gosh, we were storming in those early days.

It was a brand new team working together, which is very natural, but it’s different than when it’s like six people in a tight room together and you’re storming. There’s no real place to go and hide and be in your own thoughts. So yeah, it was really fun days. And I think one of the hallmarks of the team is we are big on shipping, we’re still big on shipping. It’s one of my classic mantras. If you know me as a product person, I’m very aggressive on shipping software, so it is kind of who I am. And we got our first version of the product out in April of that year, so only basically three full months after going full at it, which was really fast. Of course, it was very light initially. But we started onboarding beta customers in April, which again was very early.  

And then my job sort of split between coding and soliciting, finding and onboarding beta customers. It was super easy to commit them initially. It was like, “Hey, we just want you to try this thing and tell us if you get value out of it.” 

And we knew early on that just installing Pendo was an ask – you had to take this JavaScript snippet, you had to get a developer to put it into your product. That was an ask. So we didn’t want to ask for money because we figured that ask was large enough out of the gate, given that we weren’t a known quantity. 

Then once we started getting beta customers, we could start watching and interviewing and sort of obsessing over those beta customers – how were they using it? What were they using? What did they dislike? Was it growing? How sticky was it? So at that point, it really shifted to what were we learning from these folks and how could we get them hooked on the software, which is kind of what you want. You want to get people hooked and that will start to get you some measures of product market fit.

One thing you said earlier stood out to me – those early employees on your team, many of them had families. Do you feel like that played any sort of role in your early success?

I absolutely think having a family played a role. I think that having a family was kind of a core part of our identity in the early days. We have a bunch of core values. One of the core values is respect lives outside of work. And that’s the third articulation of this concept. The first one was something like “all work and no play is no fun at all,” or something to that effect. And I think we knew that we had to have a culture that balanced both work and life, but there was still a focus on work. We weren’t just sitting around playing ping pong. And I know that some people say that startup life is about working 20 hours straight or what have you. And honestly, I did that in my first startup when I had no family and no kids.  

I basically lived at the office and that was honestly probably fine for those days, but it wasn’t fine for Pendo. A lot of us had to pick up our kids from school or from after-school activities because I paid myself so little money at Pendo, things like using my wife’s insurance and so on. So she was working. That meant I had to be more of a parent and be responsible, and I wanted to see my kids as well. So we would take time out for dinner and then I would get back on the computer and work until midnight very often. I started building, I think, an early cadence that I live by now, a little differently since I have more kids, et cetera. But generally speaking, I reserve time for dinner with the family almost every night if I don’t have another work dinner. And then I do sometimes work after dinner. And I think that cadence really helped. 

We also had a more mature startup environment, and it is funny because we were in a co-working space surrounded by a lot of startups that weren’t mature. And that’s okay – I mean I think they were just earlier in their career, but I think we had a focus that was grounded in the fact that all of us had something more to gain or to lose.

When did you feel product-market fit?

I don’t know if there’s a moment I actually felt product-market fit, but I would say I really started to recognize it around late 2014 or maybe 2015. It probably took a while to get there. I distinctly remember reading Brad Feld’s blog post about product-market fit every single week because I loved how he described the phenomenon of people coming at us, wanting our product. That blog post really resonated with what I hoped to feel, and over time, it did start to happen at Pendo.

Then, there was this dinner with a VC in 2021, which is pretty recent, and by then everyone would’ve agreed that Pendo had achieved product-market fit. At that dinner, I described how our young reps, who were around 24, 25 years old, were closing deals in a single call with inbound prospects. Our board member affirmed it saying, “Todd, that’s just really good product-market fit!” It was like a lightbulb moment – a reminder of what we had accomplished.

If we go back to the early signs back in 2014, we started to convert at a healthier clip, which was another sign. Then during our Series A, we pitched to someone from Battery Ventures’ portfolio who challenged us to build a feature for ingesting qualitative feedback. They promised a large deal if we could build it, and we did, rapidly, which was a huge indication that we had something valuable.

However, another defining pivot was in the strategy regarding our target market. Initially, we tried a PLG approach focused on small startups around us in a coworking space, which honestly, wasn’t working out great. Then, Shannon, an early product manager, pushed us to target the upper mid-market and larger enterprises. This shift, paired with a mindset change to embrace active selling, began to ease the process and directly impacted our success. Moreover, in 2016, I spontaneously decided to increase the price of our lowest package by tenfold, which also proved to be a pivotal point for Pendo.

Eventually, as the company scaled, we had to beef up our go-to-market strategies and training. We encountered new challenges like ensuring non-product heads could sell our product effectively. By continually addressing these issues, we strengthened our market position and refined our product-market fit. And as scary as it was at times, like when we decided to hike our prices without a solid data-backed business case, following our gut paid off significantly, shaping Pendo’s growth trajectory and solidifying our place in the market.

What have been the biggest challenges maintaining that culture as you’ve grown?

Yeah, culture’s a huge part of the company, and it has been since the early days. From January 4th, 2014, when we first gathered to define our core values—those values have slightly evolved in wording, but the essence remains: a focus on work-life balance, recognizing that we all have lives outside work which we value deeply. Over the years, we’ve maintained a strong action orientation, where bias to act and a disdain for unnecessary bureaucracy drive us to be fast and efficient. Let’s just get things done, man, let’s go build something.

As we’ve grown, adapting our values while keeping their core intact has been a key challenge but also a strength. We’ve implemented systems like the poker chip recognition system that began around 2016 or 2017, which scaled into a digital format over the years. This system allows us to continuously recognize individuals who exemplify our values, such as collaboration, helpfulness, and a shared commitment to winning together.

However, aligning everyone with these values as we scale has its complexities. For instance, the value we formerly articulated as ‘brutal honesty’ had to be reconceived not because the core value changed but because the wording led to misunderstandings. We’ve learned it’s not just about having values but living them authentically and ensuring they are clearly communicated.

Maintaining our culture isn’t about sticking rigidly to an aspirational script but rather being honest about who we are and making sure new hires understand this. It’s about embodying these values in our daily operations and using them as a filter to ensure alignment within the team. It’s a continuous process of reflection, feedback, and adjustment. Ultimately, keeping the culture alive as we grow is about commitment to these values, openness to change, and a relentless focus on recognizing and reinforcing these behaviors every day. It’s a journey, man, and we are constantly learning and adapting.

How do you view Pendo now versus how you viewed it 10 years ago?

I think there’s still a lot of similarities in our attitude and how we think and our thought process to 10 years ago. I would like to think I’m pretty similar, I wouldn’t say the same person, but a very similar person to how I operate. I think most people would describe me as being very hands-on. I’m still hands-on today. I’m maybe with different things, but I think that the company operates in that way. I think the biggest difference is yeah, I think certainly with our size and scale we have probably a little more responsibility even I feel like a lot of companies now depend on our products for real mission critical applications. We employ 800ish people globally.

We have responsibilities to the market and to our customers and to our large team now. And so I’m certainly honored by that, but also I take it very, very seriously. We need to make sure that we do things the right way. We need to continue to behave in the right way. And look, it was easy when we’re five people making sure we behave in the right way. It’s much harder when you 800 making sure 800 people all behave in the right way. So I think, and that could be everything from how we serve customers, making sure we still serve ’em with the same sort of excitement and focus that we did when we’re really, really young. And do we always get that right now? Probably not. We probably make a few mistakes here, but let’s figure out how we can continue doing a better job. Because look, scale is challenging.

When you scale, something is going to break and you never know what that thing is until it actually breaks. You can guess. But until it actually happens, you won’t see it. So I think a lot about our responsibility in the market and our leadership position now, but we’re still the same people. And I think you see how we operate and a lot of folks in the product community know us because we’ve not abandoned the product community. If anything, we’ve leaned into it and we continue to lean into it. 

And I didn’t tell this story, but our earliest marketing was going to product camps on weekends. And I’d personally go, I would have to fight for a speaking position like everyone else. I would pitch when I went up to talk and I’d bring a piece of paper for people to sign up on.

We didn’t have any sort of lead system at the time, and we would capture as many names and emails as we could. And that was it. And there was one weekend trip where I flew to Austin and my wife and I worked the table. She wasn’t working at the company. I just needed someone next to me to make sure that we could capture names. And then we flew to LA and drove up the coast. So we did three and a period of two weekends. And I think that level of grassroots, like practitioner focused, is still very core to how we operate and who we care about. I think if anything, if you think about Pendo in 2024, we’re trying to now get audiences with executives. We have probably finally earned the right to have those, but most of the core Pendo users and passionate advocates are practitioners. They’re product people, they’re people in it day to day. And I love that. And we never want to lose that because if we lose that, that would be sort of antithetical to who we are. And I think that’s super important.

What are you most excited about now in 2024?

I’d have to give you three things because, as a product person, there’s just so much happening that really jazzes me up. First off, I am super excited about our platform’s developments. We’ve launched new modules and features, making our product the best it’s ever been. I’m really, really happy with how it has grown.

Secondly, I’m thrilled about the AI aspect. Our platform has processed 14 trillion events, and leveraging this data to train models is opening up fascinating generative applications. It’s something that gives us a roadmap for years to come, which as a product person, knowing there’s so much to build and so little time, is incredibly exciting.

The third aspect that’s really energizing for me is the transformation we’re driving in very traditional businesses. I’m talking about helping these companies become more product-led—truly rethinking how they prioritize, manage, and measure their applications, and the skills their product teams need. Consulting with companies like Pitney Bowes and exploring what the digital future holds for them—it’s just very exciting.

Lastly, I’m also focusing more on our international expansion. Having predominantly been a North American company, I’ve set a personal goal to engage more internationally. I’ve already increased my travels and participation in this sphere and am keen to see how the challenges faced by businesses overseas differ from those in North America. 

So, those are my big three things I’m currently excited about. It’s going to be a busy but thrilling time ahead!

If there’s one big takeaway that you hope a reader could get out of hearing the story of Pendo, what would you want that to be?

Focusing on the core fundamentals is absolutely critical. Look, when we started, everyone sort of doubted our market and product; they saw our solution as two disparate things cobbled together. It’s hard, creating something from nothing. But really, if you hone in on what the pain is, and how acute that pain is, you can forge a path to success. So, just focus on those fundamentals, ensure you’re solving a real, hard pain, and I truly believe you will be successful. That’s the crux of why we’re here today—because we managed to solve a hard pain, and solve it well. That’s it, really.

A huge thank you to Todd Olson for sitting down with us and sharing his time and expertise. You can follow along with Todd on Linkedin here.

The post Deep-Dive: Behind the Product: Pendo appeared first on Product Collective | Organizers of INDUSTRY: The Product Conference.

Brought to you by CustomerIQ, the AI platform to help teams aggregate, search, and synthesize customer feedback. CustomerIQ aligns teams with insights from channels like CRM notes, surveys, support tickets, and call transcripts, to help drive retention, product improvements, and revenue. Align every team with the needs of your customer and deliver an exceptional experience with one powerful AI platform. Learn more here.

For  years, Forrester has been a trusted source for business and technology leaders worldwide to drive growth and innovation.

The company’s insights are derived from extensive annual surveys involving over 500,000 consumers and business leaders worldwide, underpinned by rigorous and objective methodologies.

Forrester generates around $500M/yr offering a range of services including research, consulting, and events, with headquarters in Cambridge, MA, and London, along with North American offices in McLean, VA, and New York, NY.

Today we’re sitting down with Jeff Lash, Forrester’s VP of Global Product Management, to learn more about how Forrester builds product.

Here’s what we covered:

1. Structuring teams by customer need
2. Using sales to test products in-market
3. Tracking initiatives vs OKRs
4. Coordinating with CSMs for customer feedback
5. The unique relationship with sales in enterprise PM
6. Appreciating the complexity in product marketing
7. The importance of subject-expertise in niche markets
8. Oreos and New Orleans’-style funerals (if that’s not a teaser we don’t know what is)

Please enjoy our conversation with Jeff Lash.

Walk us down memory lane, how did you get to today?

My journey began when I went to school for marketing – this was just before the first dot-com boom, so while I was in school studying “traditional” marketing, I was really intrigued by the Internet economy technology. Believe it or not, I began my career when HTML was a prized skill and those who set up the Apache server also handled graphic design. I worked in web design and development for a while, though I realized I was not a great graphic designer or programmer.

But, I realized I was good at understanding user needs and making websites easy to use, and it was then that I found my passion for user experience and information architecture, and that was my focus for the next six or seven years. Working alongside product managers, I started to see how they shared my focus on user needs and creating valuable products. I realized that if I wanted to make the products better, I would likely have more impact in a product role.

So, I took up a role in product management in the company I was with at the time. In a month or two, it just clicked. It was a combination of the marketing concepts I studied in school, the tech background I had from growing up around computers, and my UX skills. Since then, my focus has been on product management – as a product manager, and as a leader of product management teams.

In 2012, I joined SiriusDecisions to work as an analyst and advisor, helping B2B companies improve their product management practices. After SiriusDecisions was acquired by Forrester, I transitioned into a role where I’m running the products for Forrester, and now, I’m following the very advice I gave my clients. It’s been quite a ride!

How big is the product org and how is it structured?

The product organization I manage is rather unique and exciting with about 30 individuals divided into various teams, with the structure being quite diverse. We have traditional product managers, but we also have people who are responsible for the operations and fulfillment of our products, and we also have people people in client-facing roles – for example supporting learners when they are taking part in our Certification, or working with clients of our Forrester Analyst Relations Council

In terms of those occupying product management-type roles, we’re looking at roughly 10 individuals. And we’re responsible for much more than just the Forrester digital platform; our services include access to analysts who are experts in their fields.

Let’s say you’re a chief information security officer, and you’re a client. You get to use our digital platform to find research on security best practices and trends. You’ve got access to collected data from our numerous surveys about the happenings in the security and risk market. We also have webinars, peer discussions, and more, all accessible through our digital platform.

Furthermore, you can get direct advice and guidance by speaking with experts from our team. If you’re attempting to implement something like ‘Zero Trust’ at your company, we connect you with our in-house experts to guide your journey. This approach is part of our ongoing relationship with clients.

So, essentially, while we offer a digital product and platform, there is also a significant human component, making us more than a traditional SaaS or information services product. Particularly, our human touch defines us – we firmly believe in being on your side and by your side.

How is the team structured?

A traditional approach to structuring a product management team would be to say, for example, I have four products, so one person will manage the first and a different person will manage the second, and so on. But that’s not the way we operate, since most of what we’re delivering is shared. For instance, we have a digital platform that is used by nearly all of our clients, and it supports over a dozen different products. It wouldn’t make sense for each product to have a different product manager who each is trying to add new features to the digital platform – users of all of our different products use a lot of the same digital product capabilities.

So instead, we decided to structure our team based on customer needs which, from what I’ve seen, is a pretty unique approach but has proved highly effective. We outlined and identified specific needs that our clients have. For example, they approach us because they want access to our research or they want to connect with our experts. Additionally, we also take into account unique needs for certain market segments we cater to.

Then I aligned the product leaders who report to me around each of those needs. Each leader is accountable for a need, and each need in turn aligns to some specific products and services. Although there’s some overlapping, this system also provides a clearer picture of who’s responsible for what. It’s been great because it clearly defines the roles based on the customer needs, not based on our product architecture or our technology stack.

So rather than having product management focused on owning a particular technology platform or SKU, they are focused on a customer need. This also allows team members the freedom to determine how well we’re meeting those needs and what we can improve. Rather than being confined within the boundaries of a specific product, it encourages them to think about what new products, projects, or anything else that might be necessary to meet those needs.

What is the ratio of engineers to PMs?

It’s interesting, I don’t really think about that too much because so much of what we do is not just technology development, it’s people-powered services too – so other resources from other functions are key when we think about capacity or bandwidth. For engineers, we’re probably somewhere around 7-8 engineers per product person on average. And then, on top of that, we have what we call our ‘digital experience’ group, or DX. In this group, we have a mix of UX designers and user researchers. For every product manager, we probably have approximately have 0.5-0.75 of a DX member.

How far out do you plan?

Over the past couple of years, there have been times when we were planning major initiatives like a new product launch, and we were well over a year in advance. During that period, we knew exactly where we were going and weren’t doing much else besides that. After the launch, our approach was a bit more flexible. We certainly had a plan, but we also wanted to leave space to adapt based on what we were hearing from the market.

But we’re also trying not to plan too far out because you know, we want to remain flexible. For example at this time last year, there were a number of things that weren’t on the roadmap that we ended up delivering – a new capability leveraging GenAI, for example.

So, I think there’s a balance. You don’t want to be in a position where you have no idea what you’re doing next week, but you also don’t want a roadmap that’s set in stone for the next 18 months. I think we’ve been doing a good jobof facilitating this balance and also conducting more in-market experiments.

This is something I’ve introduced where we pilot an idea quickly, let the sales team try selling it, and depending on the results, we either formalize it or learn from it and pivot. Sometimes it works great, and other times we say , ‘Thank goodness we didn’t invest too much time in that!’ So, the planning timeframe is kind of a moving target, one we’re always adjusting based on many factors.

What percentage of ideas come from top down and what percentage come bottom-up?

I would say that the majority of ideas are bottom-up. Take GenAI for example. It came about because of a directive from our CEO, but the bulk of the ideas about how to leverage GenAI came from the product manager and designers and engineers who were assigned to the project, as well as input from others across Forrester My team and I aren’t just tucked away in a back room hatching ideas out of nowhere. A lot of our process is driven by feedback we receive, sometimes from our Customer Success Managers or sales team, but more often from direct customer research or quantitative data we’re gathering. Basically, we’re continuously weaving in what we hear from the market into our work. So, most of the innovative ideas are bubbling up based on what we’re learning from customer experiences and trends in the market.

Do you use OKRs?

We’re not currently using OKRs in the formal sense, though we use a similar approach.

At the start of each year, we set out initiatives that are based on the company’s strategy. As a product team, we then break the initiatives down. Some initiatives, like HR or employee experience, are ones that we’re a part of but do not lead. Other initiatives are ones we’ll play a heavy role in.

From there, we determine the projects we want to undertake. In most cases, there are metrics around these initiatives, so we’re not entirely devoid of the OKR structure- we’ve just modified it a bit. Each year we tweak based on what we learn works and what doesn’t.

We follow a monthly routine of producing a set of key performance indicators (KPIs) that tie in with our big initiatives. For example, if our initiative is to improve retention, we put together various retention projects and track related metrics – not just how we’re doing on retention itself, but also leading indicators that can help us understand if retention is likely to increase in the future

For instance, in the case of any SaaS product, user logins and usage are key factors of retention. If no logging in or usage is happening now, it likely foreshadows a risk in renewals later. But beyond that, it’s not just if people are using it, it’s how they’re using it – there are certain types of activity and engagement that are better than others, and it’s important to understand potential correlations between what we call “activity metrics” and the higher level “impact metrics” like retention and revenue.

How do you track product usage?

Over the past two years, we have implemented a few commercial off-the-shelf tools specifically for product analytics. Earlier in my career, I felt like a lot of the products were designed more for e-commerce or public use rather than to help understand usage and analytics within a logged-in, paid-for product. In particular, we have a digital analytics platform that we use for most of our general usage tracking and analysis, and we also have a tool we use for product engagement and digital adoption.

Now, the unfortunate truth is, a lot of it does involve manual work In part that’s because as I’ve mentioned, a lot of the value of the client experience is not based on what happens through our digital platform, and we have different systems that track those interactions. But the redeeming factor here is that we do collect a lot of good records and can connect some of the dots. For instance, we can track that a certain client communicated with an analyst on a Monday, logged into our platform on Tuesday and downloaded some research and then registered for a webinar, which they then attended the following week. A lot of tracking, as it turns out, is down to good old elbow grease like spreadsheets and CRM reports. We supplement these manual efforts with some of our off-the-shelf tools to glean an understanding of product usage.

How do you manage customer feedback?

Like most companies this is definitely a challenge. When I started there was really no process at all, just emails and one-off comments. So we’ve implemented a formal idea collection process where salespeople, client success managers, and really anyone within Forrester can pass along customer feedback or a feature request. We also have the option for clients directly to provide feedback within our digital platform, and we do a lot of proactive outreach too – regular client surveys, or asking for feedback after a session with an analyst. That’s where a lot of feedback comes through initially.

Like many companies, we still struggle with handling feedback from various places. But we’ve found that rather than trying to collect every piece of feedback, we focus more on themes we’re hearing or areas we’re looking into. A prime example would be when we were trying to migrate clients from our heritage products to our new product portfolio. We knew there were some cases where migration was easy and some where it was challenging.

We focused our feedback collection on understanding why these challenges were happening. So I wasn’t worrying, “How do I gather and track and respond to every piece of feedback from any client.” Instead, my focus was on understanding what we were hearing in these challenging scenarios. This involved talking to those clients and account teams, and looking at relevant data.

Years ago I read something I still remember: one school of thought is to collect every piece of data and go through it, and the other is, “If it’s important, we’ll hear about it again.” My team talks to customers all the time. We have another team doing ‘voice of customer’ research, and researchers in the DX team. So while we do have a lot of mechanisms to keep our ears to the ground and not miss anything, , we’re generally more focused on the things that are bubbling up the most and dealing with those.

Do you have a regular practice for synthesizing feedback from teams?

Yes, absolutely, we do have a regular practice for synthesizing feedback from customer-facing teams. For example, within my organization is a team responsible for our Certification product and team team is constantly interacting with people who are undergoing the certification process, as well as with sales and account teams managing client relationships. So, they’re really in touch with what’s happening on the front lines and getting a lot of customer feedback that way, and there’s a pretty regular practice of synthesizing that feedback and acting on it.

We also have a dedicated voice-of-customer team within our marketing organization. This team carries out traditional voice of customer research and we work with them to also lead specific focused studies if needed.

Another unique aspect of our business is that we have analysts who are in constant conversations with clients, and they serve as an additional source of valuable feedback. While we still conduct our own interview and primary research, these conversations allow us to gauge what’s happening in the market on a daily basis.

How do you manage feature/project rollouts with other departments??

We really leverage our close working relationships with our portfolio marketing team and our sales enablement team. Both these departments are set up based on Forrester research about what good product marketing and sales enablement look like.

Interestingly, for us in product management, the relationship is not just with the engineering and design team– even though those connections are crucial– but also significantly with marketing and sales. I would say my team probably works more closely with marketing and sales than with engineering. This is important because as much as we can come up with the best idea in the world, the fact is we rely heavily on a direct sales force– given that we don’t use an e-commerce model or don’t necessarily have a product-led growth model.

Given the nature of our subscription-based product, many of our clients are in multi-year contracts. We’re not waiting for the contract renewal to show value, we’re really focused on adding value and raising awareness of enhancements and improvements to make sure people can immediately benefit from it.

We use a system called Launch Tiering – this is based on some of our own Forrester research. Every new product or change or enhancement is classified into one of five tiers, with Tier One being a significant announcement that might get discussed on our earnings call, all the way down to Tier Five only garnering a mention in a weekly sales newsletter. Each tier has pre-defined activities, deliverables, and recommended resources.

The importance of this system can’t be overstated. It’s almost like an air traffic control for our roadmap, making sure all the upcoming projects have the necessary mechanisms in place for a successful launch. This way, we avoid situations where we create an excellent new product or capability, and then it doesn’t meet goals we outlined for it because the launch was mishandled.

It’s like if a tree falls in the forest: if we spend a lot of time and energy to launch something, but no one even notices, does it even happen?. Thankfully, with our present system, that happens a lot less than it used to.

What do you think is unique to B2B product management vs B2C?

In my perspective, B2B product management, particularly when sold to SMBs through e-commerce, can resemble B2C. However, when it comes to enterprise sales and larger dollar amounts, the process vastly differs from consumer-grade scenarios. The nature of sales plays a significant role. Even if product management creates an incredible product, if the sales team isn’t on board due to how they’re incentivized and compensated, there will be resistance.

I’ve seen this in previous roles – sales team members refusing to push a product because it was not as profitable to them as others were. Another notable difference I see is the speed of changes. As much as we try to speed up our operations, our customers cannot adapt to rapid changes. In one of my previous roles I had a customer base that couldn’t handle the idea of monthly product updates; that was just much too frequently for them to process and adapt to them, and even quarterly was quick for them.

However, things are changing slightly, and parallels can indeed be drawn between B2B and B2C environments. But the complexity and the ecosystem of involved parties often make B2B deployment a more complex process.

At Forrester, for example, when we onboard a new client, it’s not just onboarding them with access to our digital platform; we can’t – and don’t – just send an automated email with their login details. We have a kickoff meeting where we understand the client’s objectives and initiatives for the year These initiatives and outcomes then get captured within our product, and we provide recommendations on the activities that we can work with them on to help them achieve their goals. There was certainly design and technology work that needed to be done to enable this, but the bigger lift was that there were people involved with this – our Client Success Managers, who coordinate the relationship with the client, and the analysts from our Research team, who are providing those recommendations.So here, the complexity here was not in the technology but rather in internal process changes This is why I think that while there’s a lot that’s similar between B2B and B2C, there’s a lot related to how we develop, launch, and operate products that will be different in more ways than people might think.

Tell me about the role sales plays in enterprise B2B product management?

There’s a few key elements in the role that sales plays in B2B product management, and a lot has been said by others already about this – sales asking for a feature they need to close one big customer, while product management is trying to create products for lots of customers. But one important aspect of working with sales in B2B products isn’t talked about as much, and that’s getting the mindshare of sales.

In my current position, the products my team andI manage are the main ones sold by our direct sales team. In previous roles, I’ve sometimes managed products that were just one of 10 or 20 different products that could be sold by our sales team, creating a real mindshare game.

I mean, I didn’t want them to ignore the other products, but I had targets to hit too, right? Therefore, getting people excited about my product and facilitating its sale was a substantial part of my role. I used to think of it like this – if a salesperson had an extra hour, I wanted them to choose to focus on my product.

To do that , I had to make selling my product as straightforward and efficient as possible since any barriers or complexities could mean that the salesperson just decides to spend time trying to sell a different product. I spent a lot of time working not just to improve the product but also to help sales understand it, help make it easier to explain and sell.

It’s a different game than simply releasing a product and waiting to see who signs up for a free trial.This is where sales become paramount in B2B product management. It becomes about how to make it easier for sales teams to sell the product effectively when they do secure that meeting with a potential client.

What’s something unique or central about your approach?

One thing I firmly believe now that I didn’t 10 years ago is the depth of complexity involved in product management. In more straightforward environments, you might think it’s as simple as doing some customer research, creating a product, launching it and expecting success. Yes, those are crucial components, but there’s so much more that happens along the way.

I was an analyst at SiriusDecisions and then Forrester for about eight years, and I had the chance to collaborate with and learn from with some of the brightest minds in B2B marketing and sales. That experience gave me a vast education in things like working with ecosystem partners, best practices in demand generation, marketing operations, and sales enablement. So, I’ve developed a considerable appreciation for all these nuances – there’s a lot of focus in product management on working with engineering and design, which is important, but alignment with marketing and sales is just as important (if not more) and that doesn’t get nearly as much attention, but it’s something I’ve been really focused on.

Moreover, my approach to product leadership has also evolved. I’m continually reminded of the importance of focusing on the big picture and reminding my team of that – ununderstanding our client’s needs, coming up with potential solutions to their problems, and focusing on the outcomes that we want to achieve

Also, even though we operate in a B2B environment, we are doing a lot of live, in-market experiments, which I think a lot of people think are only possible and applicable in B2C environments.For instance, we recently did an in-market test, where a version of our product was only available for sale by a specific group of salespeople, and we decided whether to roll that product out to the broader market based on the success from that pilot and comparing their results to others.

Experimentation is often talked about in terms of digital A/B testing of different page designs or call to action copy, but we’re genuinely testing real product differences in the market under complex circumstances. So instead of arguing about which idea is better, more often, we’re testing ideas with real clients and prospects. I’m proud that we’ve managed to incorporate these sorts of methodologies into our operation. The key is learning to question ideas and test them quickly, and accepting there’s a risk of being wrong – but we’d much rather learn that before we get too far rather than invest a lot of time and money in an unproven idea.

What do you look for in product hires and what does your interview process look like?

When it comes to product hires, I would say our approach is somewhat unique. A lot of my team actually came from other parts of Forrester. While we have hired people with previous product management experience from other companies, that’s not the background of the majority of my team.

That’s not to say we sought out blank slates – everyone we’ve hired is very experienced, just not not all of them came with deep backgrounds in product management. There of course has been a learning curve for some in terms of basic product management concepts, but it has the benefit of the team having expertise in the personas we serve and their needs, and also not having preconceived notions about what product management should be.

When we do hire, we look at how a candidate can complement the existing team. For example, when we were hiring for our Certification product, we realized we had a gap on the team in terms of depth of expertise around specific adult education and pedagogy . So rather than someone who had deep experience on product management across different industries or technologies, we hired someone who had some product management experience but more importantly understood adult learning and online learning platforms. So more than having tons of product experience, having the right current skill we need or the ability to learn specific new ones is key.

In terms of our interview process, we use this same philosophy. We leverage the research Forrester has created on product management – and lucky enough, a lot of that research includes things that I wrote! But we’re not limiting ourselves exclusively to people who are familiar with or used our product management frameworks. I’m always looking at how candidates can complement our existing team, whether it’s through their unique experience or their ability to learn skills, and also their overall approach and philosophy regarding product management and working across roles.

When hiring for product roles, I’m looking for people who understand the collaborative nature of our processes. As I mentioned earlier, our structure isn’t one where where one product manager handles one product. It’s more about managing major aspects within the whole portfolio, so people need to be comfortable with that.

I’m also considering the unique nature of our business. We’re not a traditional SaaS business. We’re kind of a blend of services and technology. So having someone with a good understanding of our business and our customers is huge. Actually, a number of people on my team have been at Forrester for a while – a few for over 15 years each. So they bring in a lot of understanding of our business and more importantly the personas we serve. Some of them even offer insights from the research side of things, having been analysts and done a lot of work collaboratively with clients over the years

But I’m not just looking for institutional knowledge. I also appreciate having team members who are new to the business. It’s about striking a balance. Like, the nice thing is, they’re not tied to the notion of, “we’ve always done it this way.” Most of them have held different roles before they came to product management. So even though they know the company, this is often a new area for them. It’s been nice being able to leverage their experiences in a way that doesn’t feel stagnant, that allows us to break away from the “we’ve always done it this way” mentality.

What are fun or unique traditions or rituals on your product team?

One unique tradition in our team, especially being a remote team, is our weekly meeting where we have an opening question to break the ice. As a manager, my natural inclination is to be efficient and just get right to business, but I realized that it’s also essential to build connections – especially when we’re distributed. Even though many of the people have worked together for years, when you’re not physically together it’s harder to learn more about each other.

These opening questions are usually random and fun; for example, it was National Oreo Day and I asked people about their favorite way to eat an Oreo. It sounds like a silly question, but there were very passionate opinions about the “right” way – twist off one cookie? Dunk it in milk? Eat it in one bite? This tradition that we developed rather spontaneously has uncovered a whole bunch of really interesting things about our team members and it’s a great way to get to know people on a more personal level, beyond just business.

Another meeting tradition we have is what we call Bright Spots – this isn’t specific to my team, it’s done across a lot of areas at Forrester. iIt’s just a reminder to highlight some good things that happened – positive customer feedback, a big milestone on a project, good financial results, even bright spots from our personal lives. It’s a good reminder that even when we’re having challenges, there are always bright spots to make note of.

Huge thank you to Jeff Lash for sitting down with us and sharing his time and expertise. You can follow along with Jeff on his Linkedin here.

The post Deep-Dive: Behind the Product: Forrester appeared first on Product Collective | Organizers of INDUSTRY: The Product Conference.

Brought to you by CustomerIQ, the AI platform to turn customer feedback into value. CustomerIQ is the AI platform to automatically discover and quantify themes across customer feedback channels like meeting notes, surveys, tickets, and transcripts. Align every team, prioritize work with conviction, and build a customer-obsessed culture with one tool. Learn more here.

Tripadvisor is renowned for revolutionizing the way we explore and experience travel. Founded in 2000 by Stephen Kaufer, Langley Steinert, Nick Shanny, and Thomas Palka, spans across 40 countries and is available in 20 languages.

Tripadvisor.com boasts an impressive repository of approximately 1 billion reviews covering around 8 million establishments, embodying the essence of user-generated content in the travel sector. 

Acquired by IAC/InterActiveCorp in 2004 and spun off from Expedia, Inc. in 2011, Tripadvisor underwent a transformative rebranding and has expanded into many verticals including hotels, things to do, vacation rentals, and restaurant booking. 

Today we’re sitting down with the former CPO of Tripadvisor’s European restaurant unit, Fabrice des Mazery, to learn more about how he manages and shapes Product.

More of what we learn in this article:

1. Fabrice’ dual-track roadmap planning
2. Setting product destinations vs OKRs
3. Realizing the effectiveness of KPI trees
4. How sales/CS teams pitch improvements to product
5. What’s in Tripadvisor’s tool stack
6. What to look for in product hires
7. Are frameworks overrated?
8. Fabrice’ favorite restaurant

To learn more about Tripadvisor, visit their website here. You can follow Fabrice from his Linkedin here.

How far out do you plan?

When we first started, there was no clear planning cycle. Our product and engineering seemed like a black box. There were no clear outcomes for our teams to pursue. They focused on various KPIs but there was no link to the consumer experience or a clear strategy. 

Over time, we’ve managed to turn that around. Now, we consider a timeframe of 12 to 18 months with qualitative and quantifiable strategic goals we call destinations.

We follow a dual track roadmap: we plan what we’re going to deliver in the next 6-week cycle with a clear ROI ambition, as well as what we’re planning to discover. Anything beyond that line we view as options. 

Our stakeholders understand and appreciate this format because they co-invest with us. They recognise the future can’t be predicted, so they follow a now/next/consider structure due to the inherent uncertainty of the product system.

What percentage of ideas come from top down and what percentage come bottom-up?

We embrace a more transversal logic. Our teams and stakeholders carry continuous dialogs, collaborating to identify opportunities designed to bring us one step closer to our business goals. 

Now, you might ask, where does leadership fit into this? Well, our primary role at the leadership level is to provide the teams with sufficient clarity and competence. We’re here to ensure they’re well-positioned to hold the reins of their own roadmaps and ROI. In essence, it’s about instilling autonomy, not dictating ideas. So the high-level strategic destinations come from the top, but what it means on the field and how we get there come from our product trios.

Do you use OKRs?

Yes, we do utilize OKRs, but not in the traditional quarterly sense as it tends not to make sense for product teams. We observed that the internal competition that quarterly OKRs foster doesn’t work everywhere. 

For me, it’s like trying to mix the universally cooperative Swedish way of doing things (think Spotify “model”) with the competition-based OKR model. It’s a scrambled blend of two entirely different flavors, and, frankly, didn’t work for us. We found that mixing such cultural approaches caused more chaos than cohesion.

I believe that OKRs should be based on a core strategy of reaching a common destination. That strategy should say, “we have destinations, and we need to find a way to support each other in reaching them.”

However, I’ve seen too many companies, including ours at some point, transitioning their OKRs into fragmented goals. Marketing has their OKRs, and individual countries have their OKRs, but the concept of working together to achieve something gets lost. This leads people to claim, “it’s not on my OKRs, so I won’t do that.” 

So we shifted our approach. We now follow yearly OKRs that are product-led and strategically linked. Our product teams and tribes decline these OKRs with the help of stakeholders and take responsibility for reaching the outcomes.

I’d like to stress that it’s not the OKRs themselves that are the issue, but rather how they’re implemented and understood. OKRs can be effective if used considerately, fostering cooperation, focusing on qualitative customer outcomes and quantitative product & business outcomes. It requires an understanding that everyone who can contribute should be warmly welcomed. 

OKRs are not meant to create borders or division within the company. That is why l feel they should not be universally applied. For an organization to thrive and for OKRs to not “stupidly” function, they should just be objectives strategically linked to specific desired outcomes. It’s about maintaining focus on what truly matters.

How is your product org structured?

Our product organization used to be structured into three primary domains: supply, demand, and payment, with additional teams handling platform, business operations and salesforce tasks. However, we shifted this arrangement to form three tribes focused on the Diner Experience, Restaurant Experience, and Internal User Experience. 

Now, to clarify what I mean, when I refer to “teams,” I’m talking about how we rethought our approach to the earlier problem, where teams became overly focused on individual components. For instance, the “Search” team was only about search, missing out on the broader understanding of what people were really trying to find. 

We realized that this led to a lack of holistic thinking. The teams were operating in silos, focusing only on their specific tasks and not really considering the larger journey. So, we decided to remodel the team structure around the actual user journey. 

The first step in this process was a role play situation. We wanted our teams to understand and internalize the user experience. Essentially, we challenged them to imagine guiding a friend visiting Paris for the first time through the process of finding a restaurant.

From these exercises, we renamed teams after the behavior we’d like to model. This approach made it much easier for teams to understand and empathize with the user’s journey. 

So, under our new structure – take the Diner Experience tribe as an example – teams are organized around each step of the journey. They no longer “own” parts of the product but are “guardians” who work together to improve the overall user experience. 

Those teams contribute to metrics related to the strategy. They act as a “team of teams”, focusing on improving use cases linked to the strategy, and our strategies guide our organizational structure and not the other way round. 

So, when I say structure, think of it less as a rigid framework and more as a fluid, adaptable journey everyone’s participating in.

What’s the ratio of PM to engineer to design?

The structure of our teams is based on a trio concept, which to me is non-negotiable. It’s an approach based on the military, specifically the concept of trippers, who are always in groups of three for surveillance and such. The idea is that with three individuals, if there are disagreements, a majority can always be found.  

Applying this to our product operations, we operate in trios with product managers, engineers, and designers. While the ratio within these trios is essentially 1:1:1, we also extend the count of engineers, usually ranging from four to six. In the past, we used to associate more engineers, which, upon retrospection, seems to have been a mistake due to legacy organizational constraints.

Further, we have supporting roles in the organization including specialists, marketers, researchers, writers, and content designers. Additionally, management considers itself to be in a support role as well, ensuring that everything is running smoothly. Rather than positioning any hierarchy as a vertical line with me at the top, I view the organization as a whole working in a supportive, triadic model for the best possible outcomes.

What would you say is unique or central to your approach, organizationally or professionally?

How we’ve operationalized the autonomy and responsibility of our product teams. This particularly applies to our product trios. It’s this initiative that enables stakeholders to reach a clear return on investment or ROI. It brings a sense of responsibility while promoting creativity and strategic thinking within our teams.

How has behavioral/product tracking evolved?

Well, initially, to be frank, there was not a lot of tracking. Most teams complained about their lack of autonomy in the matter, but I have to be honest, it was sometimes just an excuse not to track. 

Then we introduced product metric trees, a real game-changer, that capture the success equations of our teams, tribes and strategy. This was done with a clear ambition so all evolutions were centered around customer outcomes, product outcomes, and business outcomes. Without this focus, it was pretty much impossible to even consider the potential ROI of the work it takes to implement tracking. Now it’s clear.

How do you capture qualitative feedback?

On the B2C side, we collaborate with marketing teams to comprehend their objectives and attempt to merge them with our operations on the consumer side of the application. Yet, it’s tricky as their Marketing OKRs, such as Click Through Rate for example, don’t always align with product goals, like improving the overall user experience. Hence, one has to be cautious and strategic about what’s put into effect.

Capturing qualitative feedback in our B2B context involves a multifaceted approach. We usually start by working closely with people who have direct contact with the clients, our proxies. This includes sales representatives, trainers, and customer care. We estimate that they have about 80% of the information that we need, at an 80% quality mark. To fill in the remaining gaps, we supplement our data with analysis and interviews. Our job is essentially to cross-reference and organize the information gathered from the field, a task typically managed by team leaders.

This feedback collection process is ongoing and can take place at any time, and not just within specific deadlines. The ability to efficiently adapt to new challenges is an essential part of our operations, as it allows findings to be collated and acted on in real-time.

In an effort to cut through the “product gibberish” (I call it produxplaining) I encourage my team to focus on the benefits we aim to offer our customers rather than focusing solely on problems or needs. Thus, they can pitch at any time. The pitches, usually a concise document, get shared on Slack for collaborative input. Then, they delve deeper to capture additional insights, sometimes collaborating with our counterparts in other areas such as France and Spain. 

Marriage between the feedback from customer care and trainers, and our own initiatives completes our approach. We can direct these teams to focus on certain problem areas if needed, or even have them propose a call with us to provide more detailed insights. The beauty here is that they are eager to help. Trainers, despite handling only a small fraction of our customers, still manage to provide us with insightful feedback. And, to ensure that there’s no backlog, everybody has a part to play. 

In terms of success rate, let’s just say we’ve been pretty lucky. The system works well for everyone and the reciprocal benefits are undeniably a big reward. So, capturing qualitative feedback involves a copious blend of strategies, all aimed at delivering benefits to the customers and ultimately, the company.

How does qualitative feedback impact your prioritization? What’s the journey from feedback to delivery?

Our journey from feedback to delivery doesn’t actually start with feedback but with strategy and strategic destinations. What we typically do is look for investment opportunities that align with our focus, in terms of business and usage segments as well as use cases and outcomes. Feedback, in this context, is instrumental in helping us identify patterns, thereby uncovering new opportunities.

Once we’ve singled out an opportunity, we earmark a cycle of about 6 weeks to progress from a place of mid or low confidence, to finalizing the design for a first solution iteration that can be implemented in the succeeding cycle. This phase involves digging deeper to understand user needs better, grasp the market and its competition better, and analyze what our current solution might be missing.

From this, we refine a problem for further solution discovery, in acceptance to anything that brings increment of confidence, even if this involves pushing code in front of a user. By the end of this discovery cycle, we not only have a refined problem but also a refined design to kickstart the solution, all of this neatly tied together with a clear ambition set by our appetite.

The actual delivery cycle is dedicated entirely towards iterating on the developed solution and ensuring that both Go-to-market and Go-to-customer aspects are dealt with appropriately.

What’s in your product team’s tool stack?

Notion, Figma, ContentSquare, Tableau… but also a lot of Slack, Google Meet (beyond pure remote, my teams are spread in 6 offices in Europe) and a bit of Reforge!

How do you manage feature/project rollouts with other departments?

It involves various departments which includes those involved in the investments and in the go to market, as their input is absolutely crucial. Our product managers, with the aid of product marketers, play a significant role in making sure the go to market and go to customer strategies are refined and well-understood internally.

What do you look for in product hires and what does your interview process look like?

I look for people who understand the law of the double impact. To be more specific, I want someone who understands that product organizations are not aimed at ensuring ROI for the business AND for the users. Candidates who only focus on user-centricity or who obsess over metrics and growth without taking into consideration the real-life experiences of our users don’t fit in with us. 

My interview process is really about exploring the candidate’s experience. I’m eager to hear how candidates talk about success and failure. 

One question I love to ask is, “What was the worst product decision that you made in your life?” 

Those who can’t think of one probably were either not responsible or they haven’t reflected on the failure. That’s not what I’m looking for.

To understand what they consider ‘success’ is crucial too. Did they deliver a great user experience? Do they mention the financial aspect even with design? For example, I’m not just looking for UX designers but for product designers. These are individuals who appreciate design in a way that they understand the constraints of the product.

The same goes for product managers. If they only talk about profit and usage, but overlook the importance of user experience, we’re going to have a problem.

My end goal is to find people who understand that their responsibility includes both aspects – the ROI for businesses and end-users. I’m more interested in their approach to problem-solving, their perception of success and what they consider a success personally, rather than their theoretical knowledge. Because honestly, theory can be learned, but the way you look at problems and success is inherent. And for someone with the right inherent abilities, I’m willing to compensate for what’s needed to keep them here.

What did you believe 10 years ago that you no longer believe today?

Oh yes, a decade ago, I hung my hat pretty heavily on the frameworks. I threw my weight behind the biggest defenders, you know? I used to be adamant that there should always be 80% unitary test coverage when testers checked product integrity, and cling to the traditional two-week sprint cycles.

Now? I consider the way each team organizes between functions is not my concern. What interests me is the investment process; I want to understand the decisions you’re making, and why. It’s all about accountability. I want to know why you’re investing here, or why you chose this point for delivery. That’s it. That’s what it boils down to for me – because I’m going to give you a top line projection and from there, it’s your responsibility to manage it. 

So now, I approach things based on necessity, the maturity of the team, and the circumstances of the company. In the end, what really matters to me is that the teams are genuinely autonomous and responsible for the outcomes. Not just in word, but in deed. That’s what I’ve come to learn over these ten years.

What are fun or unique traditions or rituals on your product team?

We love our food! We’re spread out across three countries with strong regional food specialties, with a mix of nationalities: chinese, ukrainian, argentinian… It brings together many different food cultures. But, no matter what the cultural differences might be, we find common ground in going to restaurants. We embrace the “Eat your own food” philosophy, and I mean it quite literally! I find that the expression has never been so true and hilarious at the same time. This tradition certainly adds a fun flavor to our product operations team!

Alright, I have to ask: What’s your favorite restaurant?

My favorite restaurant has to be Le Clarence in Paris. It’s a two-star restaurant that I initially visited with my product director.It’s on the expensive end, really, really expensive. On my visit, I went all in, including wine-food pairing & truffle option, and that ended up costing 600 euros per person. I know it sounds exorbitant, but it was absolutely amazing. Truly a once in a lifetime type of culinary delight.

A huge thank you to Fabrice des Mazery for sitting down and speaking with us. You can find him on Linkedin here.

The post Deep-Dive: Behind the Product: TripAdvisor appeared first on Product Collective | Organizers of INDUSTRY: The Product Conference.

Brought to you by CustomerIQ, the AI platform to turn customer feedback into value. CustomerIQ discovers and quantifies themes across customer feedback channels like customer calls, surveys, tickets, team messages, and more. Align every team, prioritize work with conviction, and accelerate development with one tool. Learn more here.

JCPenney is an iconic American retail company and key anchor store in shopping malls across the United States. JCPenney traces its origins back to April 14, 1902, when founder James Cash Penney and his partners opened the Golden Rule dry goods store in Kemmerer, Wyoming. Over the next two years, they expanded to other Wyoming frontier towns. In 1907, Penney bought out his original partners and began a journey that would shape the retail landscape for decades to come.

Over the years, the company expanded its operations, was one of the first to introduced credit cards and catalogs, and underwent international expansion and diversification. But from 2012 to 2020 the company faced challenges, including a troubled pricing strategy and the impact of the COVID-19 pandemic, which led to a bankruptcy filing in May 2020.

The company has since been working on a turnaround plan, including a $1B initiative to revitalize its business, focusing on digital capabilities, in-store upgrades, merchandising, and supply chain improvements.

A huge part of that strategy is led by our guest today, Vanathy Lakshmi, VP of Product Management and Experiences. Vanathy has over a decade of experience leading ecommerce product teams at iconic brands like Pier 1, Walmart, Sally Beauty, Albertsons, and now JCPenney.

In our conversation we learned:

1. The unique challenge of shaping digital products in retail
2. The four-in-the-box strategy
3. The importance of a great VP of Product
4. Vanathy’s decision-making template
5. Using stage-gates in product discovery
6. Using a tiered approach to feature rollouts
7. What Vanathy looks for in product hires
8. The importance of technical skills vs UX skills in product management

Please enjoy our conversation with Vanathy Lakshmi.

JCPenney is an iconic retail brand that’s making a big transition to ecommerce. What unique challenges does that transition present?

Most people don’t realize that retail hasn’t really seen a whole lot of innovation in the last few decades. This makes the problems very complex. 

Moving to ecommerce means that the focus shifts more to driving traffic rather than strictly making sales, as many people tend to visit a website to research a product they’re interested in before they actually make a purchase. So essentially, the digital channels serve more as traffic drivers than sales drivers. 

One thing that I believe we need a better grasp on is that ecommerce is not a one-size-fits-all operation. Everyone has their own shopping preferences. For example, I might prefer to read an email, click on a link, then shop. And after being followed around by a Facebook ad, I might finally decide to buy something. However, my kids would rather see an ad on YouTube, click on a link, and just buy it. 

It’s crucial that we keep these preferences in mind as we plan our ecommerce strategy. At the end of the day, we’re not needlessly trying to attribute customer behavior to the nth degree. What we’re trying to do is understand where we should optimize. What areas should get more attention? Less attention? Which channels should we be focusing on? 

And another thing worth mentioning is that many of our tasks are end-to-end solutions. What I mean is, when we make a change to a product, it affects how it’s sold on the website which in turn influences how it’s sold in stores. So you’re always seeing this continual, end-to-end roadmap developing.

How big is the product team?

My team is made up of close to 45 people and I like to split it down the middle between product and UX. Now, UX isn’t just design for us – we’re also talking about conducting AB tests, diving into research, contemplating the full life cycle of our work in UX. And the team we have, well, they are just phenomenal. Always ready to handle all aspects of these responsibilities.

How do you structure the product team?

In structuring our product teams, we utilize what we call the “four-in-the-box” strategy. For every product we build, regardless of the organizational structure or the product manager, we consider this arrangement. Essentially, we have one product leader paired with one UX or user experience person, one engineering squad, and then there’s an analytics alignment. Now, these roles may often fluctuate depending on the scope of the project area – it may not always be one-to-one. 

The way the ‘four-in-the-box’ is determined is based on how they all come together, taking into account their capacity and how they are aligned to an outcome. For instance, when we target an outcome like automating segmentation, we form a product crew or product team consisting of the ‘four-in-the-box’ setup. 

Now, you’ll see a varying degree of maturity within each of these areas as the strategy evolves. But basically, if I’m told to place a product team on a specific outcome, we strategize on what the four-in-the-box setup might look like. This is done keeping in mind the scope of the outcome. If it’s complex, we might require a leader and two sub-leaders as there might be sub-outcomes within the larger outcome. Further strategizing on what the rest of the team looks like helps shape the team structure and dynamics.

How far out do you plan?

That’s a good question. I’ve never been a big fan of building a large backlog. I believe that, in our industry, if we are truly practicing agility, having anything beyond six months planned out with an nth degree of detail is just a waste of time. That being said, I do consider a healthy two-quarter roadmap to be beneficial. On top of that, I sincerely think it’s critical for teams to have a clear plot or plan of where we aim to be in a year or two. This allows us to think ahead and envision the future. However, we tend to plan more narrowly within a quarter, which I always see as an opportunity for continuous improvement.

What percentage of ideas come from top down vs bottom up?

From my perspective as a middle leader, I regularly see the struggle to balance these ideas. Often, when leaders above me suggest something, it’s taken as a ‘go-do’, or a directive. 

But I really think the go/no-go sits in that middle layer more than anything else. Many times, when an executive says something, we take it at face value and execute it. But, is that where our roadmap stops? No, this is where we need to dig deeper to understand the purpose behind what they are trying to say, their expectations, and execute their suggested business strategy. Then product leaders must work with the second level business leaders to build purpose-driven outcomes. 

Given this perspective, it’s very hard to put an exact percentage on how many ideas come from top down or bottom up. The crucial part is that there’s a balance and that we understand the ultimate goals behind every directive and use them to inform our roadmap.

Are we perfect at this yet? No, not quite. I think middle leaders, including myself, need more education to master these tasks effectively. That’s the area where we need to come together.

Do you use OKRs?

Yes, we do use a form of OKRs, but we refer to them as outcomes or outcome-driven KPIs. The application of these differs across our organization, with some areas being more mature than others. A challenge I often see is that these tend to be very single-dimensional, focusing on aspects like sales or conversion. This sometimes hampers the team’s ability to progress freely. 

What I believe we need to gravitate towards as an industry, is a system where we consider that while these KPIs all ultimately contribute to our broader objectives, they are lagging indicators. The real question we need to be asking is, what are the leading indicators we should be tracking against? 

Having said that, this becomes much complex in larger, transformational companies due to the existence of different and sometimes even conflicting entities. For instance, if I introduce a new lever, the impact of my actions can ripple through other divisions. However, it’s this complexity in navigating a bigger organization that makes the job of a product leader interesting.

What would you say is unique or central to your approach, organizationally or professionally?

Changing how we perceive ownership of the roadmap. I used to think I owned it and would push to achieve specific results. But over the last two years, I’ve learned that it’s my team who needs to own the roadmap. 

My role, in fact, has evolved into helping my team access different tool sets to assist in their problem-solving efforts and constructing roadmaps. This shift in approach was born out of considerable research and it introduces a unique dynamic that’s absent in many of today’s existing frameworks. The truth is, a common framework may or may not be effective, because every organization is unique. The success lies in adapting a framework into practice and bringing it to life.

For example, within my team, I introduced a template to facilitate decision-making and empower the team members. They were grappling with a sense of disempowerment, feeling as if they should make decisions because they didn’t feel a sense of ownership over the product. But I advised them that their role isn’t necessarily to make the final decision.

Their role is to identify and present the best solutions to their business partners, complete with data, and have partners agree to the best one. They are facilitators, verifying if a solution is effective and gathering sufficient evidence to back up their recommendations. This shift in perception helps to avoid a misunderstanding of their roles and empowers them to solve problems rather than getting caught up in decision-making.

To sum it up, it doesn’t matter so much who’s the decision maker because at the end of the day, what’s crucial is: did you solve the problem?

Tell us more about the template to articulate different solutions

Absolutely! It’s pretty simple, it’s a straightforward guide with two or three options. Each option has its own pros and cons and outlines how various role responsibilities might alter as a result of the work. And then our team puts forward our recommendation. 

We use the system across board—up and down the levels—to gain consensus. But mind you, we don’t employ this for every little decision. We use it when we are talking about game-changing decisions that could dynamically alter the landscape.

Through this method, we’ve achieved some brilliant results, which has been awesome to witness. The flexibility of the system has allowed the teams to expand on the template and tailor it to their needs—for instance, outlining what we plan to do for the next quarter or different delivery packages. Seeing how the product managers have adapted this tool to meet their needs has been incredibly exciting.

What is the role of product operations on your team?

As a product leader, I find it near-impossible to expect a product manager or anyone on my team to excel both as a detailed, process-oriented individual and as an innovative expert with a deep understanding of the business. You see, to me, there’s a clear divide there. I’ve seen folks who are strong in process, able to put everything into structure, but not necessarily innovate. Neither skill set is superior; it’s like comparing left brain with right brain. So, I determined that I needed an ops role, specifically a product ops team. 

The role of product operations sits like a fulcrum in the team, acting as the lens through which we see the product managers operate and the process of product creation. Interestingly, product ops people don’t own decisions, but they do own recommendations. They provide the tools that the leaders can use to create productive conversations and progress within their teams. 

Now, the product ops individuals may not dictate processes, but they suggest and propose. It’s up to the product leaders to take on those recommendations and tailor it to suit their needs, providing feedback to the product ops team to further refine existing standardizations.

The operating model of product ops is critical, extending beyond just the product team. It crosses boundaries to collaborate with business, engineering teams and more, ensuring the operating model’s efficacy. It’s been a fascinating journey, it really has. And the slightly misunderstood role of product ops is gradually gaining the recognition it deserves.

How does product discovery work and how has that evolved?

Product discovery works through a process we’ve implemented called “Stage Gating.” It essentially provides our product managers with a mechanism to run through various stages or milestones while working on a product. This journey typically begins with an idea, which could come from anywhere. The critical question then becomes, when does that idea escalate to a priority? Have we understood its value? Have we linked it to a business problem to solve? Because, let’s face it, every idea can seem cool, but not every idea aligns with business needs. 

The discovery phase of product development involves using our stage gates to align ideas with business value. This even allows our product managers to question and analyze random ideas from business partners, to figure out what it provides for the customer and what it would drive. 

Once a priority is established, we move on to the next stage where we discover solutions, evaluate options, build, plan, deploy, and finally measure the results. We come full circle to ask: Did we achieve the value we aimed for? If not, why not, and did we actually solve the problem we had set out to? 

This Stage Gating process is still maturing. It’s not perfect everywhere, but in certain contexts, it’s been highly effective. I like to compare this to an InstantPot – anyone can own one, but the quality of the recipe you produce is extremely dependent on the cook. Similarly, product ops provides the tools, in this case the “instapot”, and then it’s up to the manager to “cook” the best product solution with it.

How do you capture customer feedback?

We have two dedicated teams paying intense attention to this responsibility. Firstly, we have a business strategy team, which operates closely with the product team. Although the product team isn’t fully responsible for value tracking and such, they do collaborate with this business strategy team on it. 

Secondly, we have a research team, with a sub-section who go out, interact with our customers, and conduct all kinds of research. They are in close collaboration with our customer service team, striving to fully understand customer complaints and any other feedback they may have. 

Our primary aim with collecting this feedback is to cohesively bring together the quantitative and qualitative factors when we are faced with a decision-making process. If our research team identifies a problem, they bring it to our attention and suggest a fix based on the customer complaints. Instead of rushing to a solution, we take the time to evaluate the value, and understand the quant and qual components of it before we decide how to address it. 

At times, we also have a large research group involved in more complex customer feedback. For instance, my team recently engaged in a ‘size to fit’ initiative, which is often overlooked in our line of work. Both quantitative and qualitative aspects came into play as they worked with customers to understand their perceptions and feelings on the matter. Once again, this is a situation where we bring both types of data together to provide the best response and solution for our customers.

How do you go from feedback to roadmap?

When we receive feedback, initially it’s just an idea. Remember the stage gate process we discussed. Every idea has two opportunities to be considered. One in the internal road mapping session, which happens weekly, and the other one at the partner roadmap ceremony. 

In the internal road mapping session, we actively discuss the ideas that have come in. It’s a bit of a rumble, really – myself and my leaders assess the potential value of the idea. The objective here is to see whether it holds enough weight to move forward.

If it does, we bring it to the partner roadmap. Here, we collectively agree if this is substantial enough to determine its value and start planning on how to pursue it in the upcoming sprints or quarter. We try not to disrupt a quarter as much as possible, unless of course, the idea is absolutely amazing and needs immediate execution.

So in terms of defining the roadmap, it’s all about evaluation and timing. If an idea is of substantial value and aligns with our priorities, we incorporate it into our roadmap. A critical part of this process is the internal and the partner roadmap meetings or ceremonies as I prefer to call them. They’re crucial to moving through the stage gates effectively.

How are these ideas and feedback organized?

We use Confluence and we often remove stale ideas and focus on the ones that have significant influence. The key thing about ideas is that while there are no bad ideas, not every idea needs to be worked on. We evaluate ideas based on key performance indicators (KPIs). We ask, what is the business value? Is it worth the investment? If an idea passes this test, only then does it get to the discussion stage. For instance, say we have an idea to change a button color from green to blue. We first identify what KPI it affects and whether it ties with our high-level strategic goals for the year. If it does, we’ll pursue it through established validation stages. This approach allows us to prioritize effectively.

How do you manage feature/project rollouts with other departments?

This is an area where we’re always maturing and improving. We have standardized templates for launch communication to work with our business partners that are expected to be part of the rollout. We want to make sure everything checks out before we proceed.

Our approach is typically tiered. We start with a small group, and then we gradually expand it. This way, we can spot potential issues and fix them before rolling out to a broader audience. Admittedly, this can result in us being in reactive mode at times. There can be instances where we’re surprised by unexpected results and we’re left to troubleshoot and fix the issue after it has been launched. 

I’m comfortable with this approach though because if we become too proactive, it can lead to a process-heavy system which slows down the movement. I find it more efficient to launch, observe the impact, and then refine as needed, as compared to holding back the feature or project to anticipate and avoid every potential hiccup. 

However, I do exercise caution when it comes to changes that directly impact the jobs of our internal team. We deal with a lot of internal and external products, and we certainly don’t want to cause unnecessary disruption or confusion. In these cases, careful change management becomes super critical. Here, I’d rather slow down and think through every role and possible pitfall before rolling out the change.

What do you look for in product hires and what does your interview process look like?

I look for learners. I find that the world of corporations often holds the misconception that a person should already be doing the job before they get the title. I believe in giving people a chance to prove themselves and work towards something. It’s not about whether they’re already doing it, but rather, can they do what the position needs?

My first step in the interview process is to determine if the candidate is customer-oriented. 

I accomplish this by asking them to talk about what they did in their previous job. I don’t say, “tell me what problem you solved.” 

If they start their answer with technology or something unrelated to the customer, it’s a no from me. If they begin discussing a customer and a problem, then they’ve piqued my interest. 

Within the first few minutes of the interview, I can assess whether or not the candidate is right for us. I believe a core understanding of our customers and the ability to root oneself in their problems is critical to product roles. Thus, this understanding becomes a deciding factor in my hiring process.

You have a technical background. How important do you think it is for product managers to have a technical background?

I don’t believe it’s important at all for product managers to have a technical background. Now, there might be situations where it could be beneficial, but that’s not always the case. 

For example, suppose we have a data product. Most of the time, you need to engage in thoughtful consideration around the governance and longevity of the data. But this does not require a technical background, instead what it demands is critical thinking. 

People often mistake the need for logical reasoning with the necessity for technical acumen. But the truth is, what a product really requires is a stronger sense of logic rather than technical understanding. Logical thinkers, regardless of their background, can excel as product leaders. I’ve had mentors who were English teachers and yet they were great product leaders simply because they had the logical thinking ability. 

Therefore, I believe the question we should be asking is, are they logical enough to be able to think through the problem? Can they stair step the problem tree and figure out a solution? These qualities are far more critical than having a technical background. Even if someone had to write a requirement for an API, there would be more logic involved than technicality. 

As a product manager, you won’t be asked to write or change code, your job will be to define how things are supposed to work. So, while I don’t see the need for product managers to be overly technical, they absolutely need to be hyper logical.

You’re also a UX enthusiast. How important is it for a product manager to have a UX background?

Oh, I love the rare unicorn who has both UX experience and product expertise. It’s such a unique blend because the UX role is quite similar to an artist’s role. Now, I’m not suggesting that everyone is either artistically inclined or not, it’s just that people’s skills tend to lean towards one or the other. 

What I’ve observed is that there are remarkable product leaders and managers who are also stellar UX folks. They typically hail from an analytical background, possessing the ability to objectively examine a problem or solution. They ask, “What percentage of the customers used it?” “How did we resolve this for them?” This demonstrates a knack for product thinking, which extends beyond the role of a product manager. Even my CEO and the Chief Customer Officer, to whom I report, possess this product thinking knack.

Yet, being labeled as a product person requires more than just a title; it’s about the mindset. From this standpoint, I’ve noticed that an exceptional UX person excels at addressing a specific, industry-wide problem. However, there’s room for growth in our UX practices. They should evolve to address more journey-based problems. This journey-thinking approach, albeit rare among UX professionals, can catapult them into more product-oriented thinking.

What are fun or unique traditions or rituals on your product team?

We have a committee that orchestrates all sorts of enjoyable activities to keep meetings interactive, notwithstanding our all-remote setup. We usually kick off meetings with the first 15 to 20 minutes focused on understanding how each one feels. 

We also give ourselves breaks on “no meeting” days and schedule cafe breaks. We encourage open conversation within our team through things like our buddy system. This system is inspired by my daughter’s middle school exercise, where she pairs up with a kindergartner. We’ve mirrored it in our team by having each member pair up and become friends with someone they’ve never really talked to before. There’s a fun incentive too – a coffee treat for the duo!

We can also find joy in our work sessions. For instance, our design thinking sessions can be quite lively. You see the team coming out of these sessions brimming with energy and excitement.

Huge thank you to Vanathy Lakshmi for her time and expertise! You can follow along with Vanathy on linkedin.

The post Deep-Dive: Behind the Product: JCPenney appeared first on Product Collective | Organizers of INDUSTRY: The Product Conference.