One time-tested tool for gathering real-time user insights is A/B testing. A/B testing in product management allows PMs to test specific elements of the user experience, such as copy, user interface, or design. A/B tests are popular because they provide immediate and easy-to-analyze insights, and they reflect actual user interaction (instead of theoretical actions).

What is A/B testing in product management?

In this article, we’ll first break down what A/B testing is, then we’ll focus on how to learn A/B testing for product managers.

A/B testing definition

A/B testing (also called split testing) is an experiment meant to determine which variation of a variable drives the most impact. For example, you might run an A/B test to determine which landing page layout drives a higher click-through rate (CTR) or lower bounce rate. 

A/B tests require:

• Two or more variables (e.g., design, call to action, messaging) to test
• A randomized audience (so as to get the most unbiased results)

A success metric (e.g., conversions, clicks, time on page)

Why is product A/B testing important for product managers?

For the most part, A/B testing has been used in marketing and advertising to find the ideal combination of variables that perform the best. But A/B testing can be used by product managers as well. 

Often, product experimentation happens in the design phase. Research participants are shown a mockup and asked to interact with it or provide feedback. With A/B testing, product managers can launch multiple versions of a layout or design to figure out which works better. Not only does this save time from one-on-one research, it also provides objective feedback on which performs better. With design-based research, people are reacting to a concept or idea, not a live product. With A/B testing, you gain insight on what works when the design or layout is put into the hands of real users.

How to do A/B testing in product management: 5 tips to nail A/B test in product management

Trying to find the right design for your product but running into conflicting opinions? An A/B test can provide you with tie-breaking user insights. Here are five steps to leverage A/B testing for product management:

1. Develop a hypothesis

The first step to successful A/B testing is to develop a hypothesis. Think back to when you learned the scientific method. A hypothesis is what you expect to happen. A good hypothesis is something you can prove either correct or incorrect.

Three questions to ask yourself when developing a hypothesis:

• How do you think users will interact with each variation? 
• What do you expect the results to be?
• Why do you think that?

Let’s use the example of call-to-action (CTA) button color. If your current button color is green, you might want to test that against a different color, like orange. This is what your hypothesis would look like: “If we use an orange CTA button, we’ll see an increase in conversions since many other SaaS companies are seeing orange buttons outperform other colors.”

If you don’t have a clear hypothesis or rationale for your hypothesis, it might not be an element you’re ready to test.

2. Decide if the test is worth it

Now that you know your hypothesis, think about the potential impact. Will the results of this test drive significant results? If not, is there a different test you could run that might make more of an impact?

While the motto “Everything is worth testing” is a great one, it’s also worth thinking about effort vs. impact. If the test will take time and effort, but not measurably move the needle, it’s probably not worth it.

3. Design the test

Once you’ve established your hypothesis and decided it’s an impactful test, it’s time to design your test. 

You’ll need to determine the following:

• Who is the audience for this test? This could be a subset or all of your user base or prospects.
• How long will you run the test for? You may be short on time, which can be a consideration for determining the length of your test. But you usually want to run a test until you see a difference that has statistical significance.
• What will the test look like? 
• What tool will you use for testing? (Check out our A/B testing tool recommendations below.)

With this criteria in mind, you can build and deploy your A/B test.

4. Run the test

This is the fun part! It’s time to push your test live. Once the test is up and running, you can start thinking about other tests you might run and get those in your backlog.

5. Measure the results

Once you’ve hit the predetermined length of time—or have a large enough sample size—you can measure your test KPIs.

Since A/B tests are only two variables and drive equal traffic to both, you’ll simply compare one to the other. Which one drove better results?

In the case of our CTA button, we might want to look at two different measures of success: Clicks and subsequent conversions. You may find that just because the initial measure of success (a click) was higher doesn’t mean that the downstream impact (conversion or interaction) is higher.

7 product management A/B testing tools

A/B testing tools can do a lot of the heavy lifting when it comes to running the actual split test. 

When it comes to A/B testing for product managers, here are some of the most popular tools you can use:

VWO

VWO is a conversion optimization tool that’s great for A/B testing. VWO lets you run multiple tests at the same time by allocating traffic to each test. This reduces interference and allows for cleaner data. VWO also allows you to run tests based on a user’s on-site behavior. For example, VWO can automatically enroll someone in a test once they click on an element of your site or based on the amount of time they’ve spent on each page.

A key benefit to VWO is its asynchronous code, which reduces loading times while tests are running. 

Google Optimize

Google Optimize is a great free option for A/B testing. If you have Google Analytics set up on your web-based application, you can use that data to identify areas of your product that need improvement. From there, you can test and measure different versions. Google Optimize uses Bayesian statistical methods to model the real-world performance of your experiments.

Why would you not use Google Optimize? To get the most out of the platform, you need a certain amount of technical prowess. 

Optimizely

Optimizely is a powerful tool for A/B testing that was built for enterprise organizations. Optimizely uses a visual editor to make getting your A/B test up and running simple—even without developer support. Exclusion groups allow you to safely run multiple experiments at the same time. Optimizely’s program management functionality allows you to scale experimentation and collaboration by collecting ideas, prioritizing projects, and managing experiments across your organization.

Omniconvert

Omniconvert is a conversion rate optimization tool for developers, startups, and ecommerce companies. It uses CSS and Javascript to give you complete control over variation code. You can also reuse code between variations to make it easier to run new experiments, or leverage cookies, data layer attributes, or UTM parameters for advanced segmentation. 

A big differentiator for Omniconvert’s CRO tool (Explore) is that it uses a CDN cache bypass, allowing you to run live tests immediately. Due to its technical nature, Omniconvert works best for product teams that have a developer who can help create and deploy A/B tests.

LaunchDarkly

LaunchDarkly allows you to continuously improve your software by gathering data on the impact of new features.” 

Something that’s unique to LaunchDarkly’s model is that they make experimentation part of the normal delivery process. LaunchDarkly builds A/B tests into feature flags. Each time you launch a new feature, it’s wrapped in a feature flag. You can launch different variations and LaunchDarkly will identify the winning variant, then you can roll out the ideal version to all instances. 

Inspectlet

Inspectlet is a lot like the other A/B testing tools in that it’s easy to install via a short Javascript snippet, you can build tests using a visual editor, and it tracks your experiment parameters. What’s neat about Inspectlet is it also includes session recordings and dynamic heatmaps. These tools will help you understand the answers to these questions:

• Are there sections users are spending a lot of time on? 
• Where are they clicking? 
• Where are they scrolling without stopping? 
• Where are people getting confused or frustrated?

These insights can help you pinpoint areas for testing. 

AB Tasty

AB Tasty is another A/B and multivariate testing tool. The functionality is a lot like the other A/B testing tools above, with two standout features:

1. AB Tasty sends experiment performance data directly to your analytics tool for easy analysis.
2. AB Tasty offers an ROI dashboard that tracks the impact of your tests compared to the original. This helps you attribute gains in revenue to your experiments—making it easy to get buy-in from stakeholders for the next experiment.

Key Takeaways

If you’re a product manager, run experiments regularly. A/B test results can help you improve the customer experience in your current product—and make sure new product development meets user’s expectations. Try leveraging an A/B testing tool to make experimentation quick and simple. 

The post A/B Testing in Product Management: Definition & Overview appeared first on Product Collective | Organizers of INDUSTRY: The Product Conference.

With the shift to a digital landscape that was accelerated by the pandemic—along with the increasing number of technology startups—the need for people who know how to collaborate with product managers is growing. Successful product managers work with cross-functional teams, which means the need for people who understand how to work with product managers (PMs) is increasing along with the rise in PMs. 

How to work with product management

If you work for a tech company, it’s helpful to know how to work with product management. While the product management profession has been around for a while, it’s still not as clearly defined as other roles. Product manager responsibilities often depend on the size of the organization, the makeup of the product team, and available resources. 

What do you need to work with a product manager?

When working with product management, it’s necessary to understand the role of a product manager. A product manager is responsible for creating products that meet customer needs and business objectives. 

Product managers, or PMs, sit at the intersection of business, marketing, and engineering teams. Due to their unique position in the company, they take in a lot of information and feedback to help them develop better products. When working with a product manager, one of the best ways to get their attention is to bring data. This includes quantitative data (such as product and feature usage) and qualitative data (such as Net Promoter Score, support tickets, and customer feedback given through account management). These data points help them to make informed decisions about their product roadmap.

How should product designers work with product managers?

Let’s start by looking at the job description of product designers vs product managers. Product designers ensure customers have the best experience possible and focus on usability (i.e., is it easy for people to use this product?). Product managers try to balance meeting users’ needs and meeting the needs of the business. While there is overlap in how these roles think, it’s important for product designers to keep in mind that product managers are thinking about the bigger picture. As a result, they may not be as focused on improving the user experience of one particular screen or flow (that’s where you come in!). 

Here are a few suggestions for how product designers can work harmoniously with product managers:

1. Get a deep understanding of the customer.

When you work with product management as a UX designer, one of the greatest value adds you can offer is a deep understanding of the customer. Through customer interviews, user research, and usability studies, you can derive answers to the following questions:

• What are their pain points?
• What problems are they trying to solve?
• How do they interact with your product? 

These customer insights make product designers great representatives for the customer in the product development cycle, helping inform critical product decisions.

2. Know the product vision.

The product vision is something the product team, its stakeholders, and the company as a whole are striving toward. This vision gives you insight into the future of the product, allowing you to design with the future in mind.

Knowing the product vision can also help you focus ideas. If you know a certain feature or function isn’t part of the core product strategy or vision, you can filter out that feedback and direct your efforts on the functionality you know will contribute to the long-term vision of the product. 

3. Leverage mockups and prototypes.

As a designer, you may think in images. But many on your product team may struggle to visualize what you’re envisioning—or to put their own thoughts into a visual format. Mockups and prototypes can help you convey ideas and solutions, giving people something to respond to. It also helps ensure your team is on the same page and thinking about the same thing. 

You can support your product lead by translating their vision into sketches or wireframes. This will help them circulate ideas with stakeholders and receive actionable feedback that informs how the product is developed.

How should project managers work with product managers?

If you’re a project manager working with product management for the first time, you may struggle with how to work with product management. This is because of a few critical differences between the product manager role and that of a project manager.

Here are some ways you can navigate these differences and become a better partner to your product manager:

4. Understand their priorities.

As a project manager, you might be responsible for gathering user requirements related to your product. But the product manager will decide how these get defined as user stories and prioritized in the backlog. When you’ve worked so hard to gather the details, it can be frustrating to not see them prioritized the way you might have prioritized them. Take some time to understand what matters most to your product manager and how they think about prioritization. This will help you minimize friction and improve your working relationship.

5. Keep a firm grasp on scope.

One of the hardest tasks for product professionals is scope. As project manager, you’re balancing time, budget, and quality. In an ideal world, everything would go as planned—meeting the timeline, producing a quality product, and staying within budget. But, if you’ve worked in software development for more than a minute, you know that things hardly ever go according to plan.

As a project manager, you’ll need to be flexible to adapt to the changes—and firmly communicate scope to ensure quality products ship on time and within budget.

6. Consider dependencies and unknowns.

If you’re used to working in traditional (non-software) project management, you’re probably used to projects that are clearly defined. The opposite is true for products: they don’t usually start with a lot of definition. Product development starts with a problem to solve—and there are usually multiple solutions to a problem. Once you choose a possible solution, you still may not know everything that goes into developing that solution. Hence why products become more defined over time. These differences are often why working with product managers can be difficult for project managers at first. 

If you want to be successful in project managing a product, be patient and understanding. Things will change—that’s part of the job. Remain flexible and stay in constant communication with your product manager to make sure you both can release a product you’re proud of.

7. Help them understand and communicate risks and blockers.

A key part of the project management role is risk and issue management. As a project manager, you’re looking for potential problems that will impact your product development and release timeline. When issues arise, consider the benefits and tradeoffs of each possible solution. These will help your PM make the best decision possible about how to move forward.

How should product marketers work with product managers?

One of the biggest differences between product managers and product marketers is who they focus on. PMs think about the user, the person actually using the product. Product marketing managers tend to keep buyers front and center. There can be overlap in these two audiences, which is why product marketers can be such great partners for product management.

Here are a few tips to keep in mind when working with a product manager as a product marketer:

8. Remember that product roadmaps change.

As a product marketer, you’re responsible for the go-to-market strategy for new products and features. This puts pressure on you to determine how to position new features or products, as well as establish a timeline for release communications. 

This can create tension—you want a clear idea of what’s being shipped and when so you can communicate to customer-facing teams, but product managers don’t always have that level of clarity in advance. As their development teams work on the product, they’re learning more about unknown complications and limitations. Understanding that this is a natural part of the product development process can help you establish more reasonable expectations when preparing for a release. 

9. Focus on the problem being solved.

You may have heard about the Jobs to Be Done Framework. The basic principle of this framework is that people don’t buy products, they buy solutions to jobs they need done. So you could say that people don’t buy shelves, they “hire” shelves to organize their home so it’s not overwhelmed with clutter. 

Product teams may use this approach to position the problems their product is solving—and it can help you, as a product marketer, to think about how to frame new product features. Instead of focusing on the feature itself, ask yourself, “What problem is this feature solving?” The product manager you work with can help answer this question, informing your messaging and positioning for the release.

10. Keep your customer front and center.

A big part of the product marketing role is an understanding of the customer and market. Product marketers perform market analyses to get a feel for the current landscape—the buyers, competitors, and potential partners. This information helps PMs with decision-making about which projects and features to prioritize.

Product marketing managers also do market research to determine the appetite of the market for new product development, and to gather insights into consumer needs and preferences. They turn this information into buyer personas, which are leveraged by marketing and sales teams as part of their outreach and campaign development. Great product managers will familiarize themselves with these personas to better understand potential customers and what they’re looking for from a product.

Some more tips for collaborating with PMs

No matter your role in an organization, there are some key principles you can keep in mind when working with product managers.

11. Understand their product development process.

Take some time to learn about the methodologies they use (e.g., Kanban, scrum, agile). Some good questions to ask might be:

• How long are your sprints?
• When do sprints start and stop?
• What does the product development lifecycle look like?
• What do your release cycles look like?

The answers to these questions will help you understand how they work, so you can determine the best way to collaborate. For example, if you know that their release cycles happen once a quarter, you’ll have a better idea of when to propose an idea or when to expect a new feature to ship.

12. Be okay with improvements going on the backlog.

One of the core responsibilities of a PM is prioritization. They know they can’t do everything—and they need to stay focused on the product vision and the most high-impact opportunities.

Product managers (or their product owner counterparts, in larger organizations) will keep a backlog of improvements and ideas. If you suggest a new initiative or product improvements, they may end up on the backlog. Keep in mind that this doesn’t mean they won’t ever get implemented. To make a case for why something is important, gather metrics and data that show the impact of the problem/potential solution. This will aid in getting your suggestion prioritized.

You could also write user stories for your ticket, which will make it easier for them to pull the item out of their backlog and into a sprint when bandwidth frees up.

Final thoughts on working with the product manager role

At the end of the day, everyone wants to develop and release good products. Achieving this goal relies largely on how you work with product management. And to better partner with this function, you need to understand the product manager role. This understanding will help you tailor your interactions and expectations. Product management isn’t always an easy job—but by implementing these tips, you can make it a much better experience. 

The post 12 Tips On How To Work With Product Managers appeared first on Product Collective | Organizers of INDUSTRY: The Product Conference.

But how do you decide whether to foray into B2B vs B2C product management?

In this article, you’ll learn:

• What B2B and B2C product management are
• The differences between B2B and B2C product management
• How to decide whether B2B vs. B2C product management is right for you

B2B & B2C product management: what are they?

Before we dive into the differences in product management for B2B vs. B2C, let’s briefly cover what business-to-business and business-to-consumer product management is. In both B2B and B2C companies, product managers might develop physical or digital products. For the purposes of this article, we’ll focus on digital products, or what we call software product management.

What is B2B Product Management?

Business-to-business (B2B) product management is the development of products that are sold to other businesses. Think: products like marketing technology, human resource information systems, etc.

In B2B companies, product managers (PMs) focus on market trends and customer feedback to build their product roadmap. They work with a variety of stakeholders around the organization—including customer success, sales, and product marketing—to understand customer needs and bring new products and features to market.

What is B2C Product Management?

Business-to-consumer (B2C) product management is the development of products sold directly to end users or individuals. This could be a mobile app (like Mint), downloadable software (like Microsoft Office), or web-based application (like PayPal).

In B2C companies, PMs spend a lot of time experimenting and using product analytics to study user behavior. This is because the user experience (UX) is paramount to a successful B2C product. If a product is too difficult to use, the customer leaves and buys something different. But because UX is so critical, small changes to it can result in a lift in engagement.

Product management B2B & B2C: do they have anything in common?

The core responsibilities for a product manager remain the same, regardless of industry. Product managers of all kinds need to conduct market research, lead cross-functional teams, coordinate go-to-market strategy, and develop and refine a product roadmap.

7 differences between B2B and B2C product management

You might be wondering: If so many product management responsibilities are similar, what are B2B vs. B2C product management differences? Because there are a few key differences worth pointing out.

1. Different buyers and end users

In B2B sales cycles, the person purchasing the product is usually different from the person using the product. That means that while you need to design your platform with the end user in mind, you also have to consider the features and functions that will get the people with purchasing power to buy your product. This also means that the personas your marketing team develops may be different from the persona your product team uses for research and development.

In B2C, the end user and the buyer are usually the same person. Teams across the organization can typically align around a few personas for both product development and marketing.

2. Prior industry experience

Many believe that to successfully build products in B2B, you need previous industry experience. This is because so much of product development in B2B depends on an understanding of market needs and trends. If you haven’t been in the industry for long, it can be hard to understand and stay on top of changes to the market.

That’s not to say you can’t be successful in B2B without industry experience. In fact, some organizations have found that bringing in B2B PMs from other industries has helped to cross-pollinate ideas and best practices.

For B2C, industry experience doesn’t matter quite as much, because you have a larger customer base to work with and learn from. Product analytics and market data can help you understand your customer’s needs without doing frequent discovery calls.

3. The role of sales in product development

Sales plays a larger role in B2B roadmaps than B2C because the product is influencing deals. If a sales rep is working a large deal and the customer asks for a feature that could make or break the deal, that request is going to be prioritized higher than other items on your roadmap. The same goes for renewals of existing clients—if the dollar value is high enough, those requests will take precedence over the development of other features.

Product managers in B2B companies may also be more involved in the sales process. If an account executive is working a deal where a customer has more technical questions, the PM might get pulled into those communications. The same could go for customer success managers working a renewal or upsell.

4. Whether customers focus on features or usability

While each buyer is unique, and there’s no hard-and-fast rule on what people prioritize, B2B buyers do tend to focus on features, whereas B2C buyers tend to focus on usability.

For B2B products, buyers tend to look for certain features. For instance, does this product integrate with other software in my tech stack? Can they export data to pull into their analytics dashboards? As a result, there may be some things you build where the KPI is influenced pipeline, rather than product adoption.

For B2C customers, usability is everything. If they can’t adopt the product because it’s not easy to use, they’ll stop using it—and stop paying for it. That’s why B2C product managers focus more on UX and quality-of-life improvements than on complex, net-new features.

5. The number of release cycles

B2B products often have fewer release cycles than B2C products. This is because updates require significant changes around the organization. The marketing website needs to be updated to reflect the new features, onboarding needs to be updated to reflect UI changes, etc.

B2B product managers also need to consider how a release will impact a customer’s workflow. For example, a product built for a sales team (like Gong.io) wouldn’t want to release updates at the end of the month, end of the quarter, or end of the year, when salespeople are trying to meet quota.

6. Access to customers for research

When it comes to customer research, B2B product managers may have the upper hand. Because using a product is part of a customer’s job, they may be more willing to participate in user research and provide feedback. After all, improvements to the product make it easier for them to do their job!

On the other hand, while B2C product managers may have more difficulty getting ahold of customers for research, they might have an easier time with user testing. Because B2C products tend to have more users, B2C PMs can derive results from experiments much faster than their B2B counterparts.

7. How people use the product

In B2C, the number of users in the product is critical to the company’s success. Why? Because each user is a customer. The more customers they have, the more profitable the business is. As a result, B2C PMs focus heavily on the customer experience. They want customers using the product as much as possible, because that’s what will retain customers.

Compare this to B2B companies, where there are fewer customers and users make use of the platform differently. Take Salesforce as an example: Revenue ops people might use Salesforce to route leads. Data analysts might use Salesforce to create, pull, and share reports on open opportunities or closed won and closed lost deals. BDRs and account executives might use Salesforce to track deal progress and pipeline. Even still, account managers might use Salesforce to store notes on a customer following a quarterly business review. These are all users on an account, but what they’re trying to do with your product is different.

As you can see, B2B and B2C product managers have to take different approaches to looking at customer journeys and product usage.

B2B or B2C product management: Which one is right for you?

Trying to decide whether to take a B2B or B2C product management role?

If you:

• Like to connect with users one-to-one or conduct offsites
• Don’t mind demoing products as part of a release
• Have a lot of industry experience
• Are comfortable leaning on customer support or account management for client feedback

B2B product management might be a good fit for you.

On the other hand, if you:

• Want to build for the individual instead of different use cases
• Love A/B testing and data analysis
• Care deeply about usability

B2C product management might be for you.

Of course, that’s not to say each of those things are unique to B2B or B2C, but those are some of the key elements of each role. If you’re still unsure, try dipping your toe in the waters of each to see what works best for you!

B2B vs B2C product: can a product be both B2B and B2C?

A product can absolutely be both B2B and B2C. There are two ways this could show up.

The first is B2B2C products. In the B2B2C business model, businesses sell to other businesses, who then go sell directly to consumers. A great example of this model can be found in manufacturing and distribution. Businesses create products and pitch them to a distributor like Amazon or Target. The distributor then sells the product. In this case, the manufacturer needs to market to both a business (to sell their products) and the consumer (people have to want to buy the product).

Another example is businesses that have B2B and B2C arms. Take, for instance, Udemy, an online learning and teaching marketplace. Udemy’s product was initially marketed to businesses (teachers who could provide courses) and consumers (people who wanted to learn new skills). Udemy then expanded their B2B arm with the addition of Udemy Business, selling employee learning to companies. LinkedIn is another business that has both B2B and B2C components. LinkedIn is built for and marketed to consumers as a social platform, but tools like LinkedIn Learning, Sales Navigator, and LinkedIn recruiter are largely built for and marketed to businesses.

Key takeaways about b2b vs b2c product management

While the job description of a product manager may not change much from industry to industry, there are some key differences between B2B vs. B2B product managers.

Looking to start a career in product management (or change jobs)? Check out our jobs board.

The post B2B vs B2C Product Management: What’s the Difference? appeared first on Product Collective | Organizers of INDUSTRY: The Product Conference.