New product introductions fail at an amazingly high rate – about half of all new products commercialized fail to meet sales targets. In some cases, customers like the product features, but the cost is too high. In other situations, companies miss the mark in developing and designing a product that will meet customer needs.
Across the spectrum, CEOs and new product development practitioners are frustrated with the success of new product launches. The answer doesn’t seem to be in adding more resources or in creating splashier ad campaigns. Instead, study after study shows that effective new product development (NPD) comes from understanding customer needs, following repeatable NPD processes, and executing a customer-centric innovation strategy.
New product development is inherently risky because industries, competition, and technologies all change – and today are changing at a dizzying pace! While risk is part of every NPD project, as new product development practitioners, we can harness requirements uncertainty and technical uncertainty to choose an appropriate NPD system. Not every project should be planned and executed in an identical fashion. So, when we tailor NPD processes to the individual project risk, we can find greater success in completing commercial innovations.
Three Common Innovation Systems
Institutionalized in new product development is the traditional, staged and gated system. This process is designed to manage telescoping risks and benefits throughout a project’s life cycle. Investment matches risk from the idea generation stage through product launch. These types of processes are often called “waterfall”.
Next, agile processes have been introduced to manage highly uncertain technical developments, primarily in the software and IT industries. Rapid experimentation, iterative design, and frequent customer feedback build the core characteristics of an agile NPD system, like Scrum.
Finally, a hybrid process, called “wagile,” attempts to collect the advantages of both the waterfall and agile systems. Wagile NPD is neither staged and gated nor is it fully iterative. Many companies working with tangible products in fast-moving industries are implementing agile processes to speed innovation and customer feedback while managing investment risk in a more conventional manner.
Let’s look at each process in turn.
Traditional NPD Processes
New product development is most successful when a systematic process is utilized to create repetitive product or service commercialization. “Success” means hitting a sales volume target or profitability goal. In benchmarking studies, “success” also means that a firm is among the top performer for innovation in its industry.
The benefit of a systematic NPD process is that everyone throughout the firm, and including customers and suppliers, understands the status of a project in progress, the steps required for technical and market development, and the necessary market approvals. NPD processes do not have to be complicated and certainly should not be onerous, but the must ensure that each project is reviewed consistently and fairly. Moreover, NPD processes must be applied to every innovation idea, concept, and prototype within the firm. The most common NPD process is the Stage-Gate™ model, designed by Robert Cooper and documented in his famous book, Winning at New Products. (You can see a book review of this here.)
Benefits of a Stage-Gate Model
In a traditional stage-gate model for new product development, work on the product is done in “stages” and decisions are made at “gates”. About 80% of US companies use a stage-gate model for innovation today. Various incarnations of phased and gated models are utilized for traditional engineering and construction projects as well.
A key benefit of a phased process model, like a stage-gate system, is that investment risk is minimized. Relatively few dollars are committed in the early stages to study market opportunities and ideas. If the options prove out, then more money is invested to build prototypes and conduct customer tests. Then, if these experiments are also positive, further investment occurs to build or enhance manufacturing facilities and to formally commercialize the product. At any point in the process, a project can be killed if the tests do not return expected outcomes. In this way, the investment in any given idea is minimized and risk-adjusted.
Steps in a Traditional NPD System
A traditional NPD process, like a stage-gate system, is considered a “waterfall” approach to project management. As water flows downhill, it cannot flow uphill to return from whence it came. Likewise, once a step in the stage-gate framework is completed and funding is consumed, it is not easy to go back and repeat a prior step. However, the purpose of the gate decisions is to validate prior work on the project and to approve future plans. Assuming that all work is accurate, there should be no need to repeat earlier steps. Waterfall processes require that upfront planning and requirements are correct at the beginning of each phase. Traditional phases in an NPD process are as follows.
- Stage 1 – Opportunity identification
- Stage 2 – Concept generation
- Stage 3 – Concept testing
- Stage 4 – Technical development
- Stage 5 – Product launch and commercialization
In the “fuzzy front end,” investment is not so high as there are typically no physical assets involved. The work done in these early phases (Stages 1 through 3) involves gathering market data and customer insights, testing proofs and prototypes, and narrowing design characteristics of the new product.
Cautions in Deploying Stage-Gate Models
As with any waterfall process, the biggest disadvantage of stage-gate systems is the upfront planning. In theory, a traditional NPD process is designed to test customer feedback and gather end-user insights during each phase of work. Market attractiveness and customer need are major criteria reviewed at gates, and a project must deliver positive results for a project to pass a gate and move to the next stage. Customer interactions are built into the work and approvals of each phase of NPD work.
In practice, however, many firms are sloppy in customer testing. After all, a lot of smart people work in the R&D department and have the greatest knowledge of technical advances in their field. Further, just asking a customer what s/he wants in a next generation product does not yield insights into disruptive innovations. And, finally, we often work on NPD projects that flow through the system because a high-level manager thinks the idea is great, even though there is not one iota of supporting data.
A traditional stage-gate system works for NPD when requirements uncertainty is low, and the technical approach can be pre-determined. But, senior management must make tough and honest decisions at the gates. Projects that won’t deliver expected commercial value or solve a customer’s needs must be killed. Oddball product ideas that show promise must be nurtured, even if they appear to be outside the standard operating mantra of the firm.
Airbnb is an example of a new platform that pressed forward when faced with multiple innovation and sales challenges. Customer insights were positive in concept tests and the market need was genuine. Tweaking how properties were photographed allowed Airbnb to move out of the technical development stage and into widespread, successful commercialization.
Traditional Stage-Gate Models in NPD
Traditional waterfall processes are successful in bringing new ideas to market. A stage-gate system minimizes investment risk because each stage of work is carefully matched to escalating goals and objectives. When customer insights and feedback are held as sacred gate pass criteria, a traditional stage-gate system yields repetitive market successes. When both requirements uncertainty and technical uncertainty is low, a traditional NPD process is appropriate. NPD teams should consider what extensions and incremental improvement projects to be managed by a phased and gated NPD system.
Note that a low degree of uncertainty does not mean that the technology is simple. For instance, new refineries and petrochemical plants are built according to waterfall processes but because the investment risk must be minimized, requirements uncertainties and technical risks are optimized through upfront design and planning.
Scrum in New Product Development
One of the hottest topics in project management and product development today is Agile. Agile is a set of values that generate several different project management frameworks to increase productivity, customer satisfaction, and team morale. While agile methodologies are widespread in software and IT, they are only recently being adapted to the development of physical products.
The Agile Manifesto
The core values of the agile philosophy are reflected in The Agile Manifesto. This proclamation was produced by a group of software developers in 2001 in an attempt to improve speed-to-market and accuracy of product delivery. The Agile Manifesto compares a preferred way of doing things in a project to the traditional way. So, while conventional project management tools, techniques, and procedures are not rejected outright, the agile philosophy recognizes a better and more efficient way to accomplish project tasks. These are shown on the left-hand side of the comparison statements, while conventional policies are shown the right-hand side.
- Individuals and interactions over processes and tools
- Working products over comprehensive documentation
- Customer collaboration over contract negotiation
- Responding to change instead of following a plan
While there are literally dozens of implementations of the agile philosophy, the most commonly adopted framework for physical product development is Scrum. Scrum is an adaptative, flexible approach to projects that allows for iterative development and improves communication across a team and with the customer. Elements of Scrum fall into three categories: artifacts, tools, and roles. Please refer to other posts and papers for more detail on the Scrum project management approach.
A key artifact or process in Scrum is the idea of time-boxing. This produces a rhythm and cadence in the work and, for product development, helps to minimize risk. Several events are time-boxed or limited. The first is the sprint.
A sprint typically lasts two to four weeks and is a period of intense work for the product development team. Only a few, priority tasks are completed in each sprint so that the team maintains focus. For example, a sprint may be designed to gain customer insights through market research by conducting focus groups or testing a particular concept in a real environment. Sprint tasks are prioritized to bring the highest value to the project as early as possible. Thus, by keying in on a few, critical items early in the project, a new product can be designed with appropriate features that customers want and need.
A disadvantage of the sprint in physical product design is in the definition of “done”. Whereas a software project can deliver completed lines of code at the end of a sprint, physical product development sprints may deliver test results or qualitative market research. It may be useful to view the sprint as a learning period.
Other artifacts in Scrum include the daily stand-up meeting and retrospectives.
As indicated, the new product development (NPD) team works on gaining customer feedback or delivering a working feature during each sprint. A crucial decision-making tool for Scrum is the product backlog. This is loosely similar to a list of project requirements in a traditional phased and gated product development process.
The product backlog is developed at the beginning of a project and is constantly reviewed and updated (“groomed”) during the project life cycle. Product features and attributes, along with the most critical customer experiments and feedback loops are listed in rank order. Items from the product backlog are worked during any given sprint. The NPD team commits to only work on a product backlog item during a sprint which can be completed in the timeframe (e.g. 2 to 4 weeks). In this way, the highest business value items are worked first.
In physical product development, it is of high value to determine the business case for a new product as well as to test a minimally viable product (MVP). Market studies and technical experiments are often conducted int eh early sprints to determine customer need and product feasibility. Such knowledge-building activities are designed to eliminate uncertainty in the product development effort.
NPD projects that utilize an agile process, like Scrum, may be complex or complicated with expected variations in requirements or technical approaches. For instance, developing a new lane departure warning system for an automobile will require significant technical development yet the project requirements are known upfront. NPD work, in this situation, requires iterative design, testing, and feedback – all are best approached with an agile mindset.
Three important roles in a Scrum project are the team, the Scrum Master, and the product owner. A cross-functional, co-located team does the work of the project during the time-boxed sprints. Close collaboration among team members is often cited as a reason for improved productivity in an agile project versus traditional staged and gated processes. Using a co-located team of specialists-generalists improves creativity and collaboration.
The Scrum Master is a bit like a project leader yet works in a service role more than a directional one. The team largely decides how to accomplish tasks during any given sprint while the Scrum Master interfaces with the customer, and removes roadblocks and obstacles facing the team in their daily work. A Scrum Master will interface with other project leaders as necessary to ensure effective system design as well.
The product owner is a unique role in Scrum and a role that is frequently overlooked in practice of traditional NPD and project management processes. Product owners create the prioritized product backlog, making the decisions of which features are most important – and valuable – to the business and to the customer. It is the product owner who approves features and applications at the end of each sprint. NPD projects benefit from creating personas for the product owner to assume in this role.
Some NPD project may literally use customers as product owners, such as with lead customer boards or user panels. In other situations, the company may ensure that the product owner is an outward-facing position (sales or marketing) to ensure accurate market feedback for the prioritized product backlog.
Agile for NPD
While initially designed for software development, agile methodologies are gaining traction in NPD for physical, tangible product development. Often, the agile processes, like those in Scrum, are overlaid on a traditional NPD process. Learning cycles are especially important in the market and technology development of a new product. Both requirements and technical approaches are expected to vary during the project life cycle.
A product like the Amazon Echo™ in one in which an agile development process is effective. Technical requirements would be fluid during product development as the team resolved what features customers needed or desired. Prototype testing and rapid experimentation and feedback allowed the project team to identify the highest value features quickly.
“Wagile” in NPD
As indicated, most successful businesses today employ a new product development (NPD) process to manage innovation. Studies indicate that over 80% of leading US companies use a phased and gated framework to convert embryonic ideas to commercial products. Yet research also shows that about two-thirds of all new products fail within the first two years of market introduction. So, it is not surprising that companies are looking for other systems to manage NPD projects. Because Agile has been successful in software development, many firms are considering project management frameworks, like Scrum, for physical new product development.
A disadvantage of Scrum in physical product development is that while features can be designed during a typical two- to four-week long sprint, it is often difficult to test a feature without the availability of the whole product solution. Consider, for example, testing automatic collision avoidance on a vehicle without knowing the weight, horsepower, or tire diameter of that vehicle.
We know that NPD projects must be faster and more responsive to customer needs than we can normally deliver in a traditional phased and gated process. Both NPD practitioners and senior executives are frustrated with the pace and success of innovation projects. We also know that going fully agile is not practical in many situations. What’s the answer? “Wagile!”
“Wagile” is a hybrid of waterfall and agile processes, adopting the positive aspects of each project management framework to physical product development. Wagile processes are faster and more flexible than conventional staged and gated processes but recognize the whole system as a product. Moreover, wagile processes interface with customers at key intervals to determine functional needs and to garner important design insights at the right times in the project life cycle.
Iterative wagile processes are often used for new product development projects that have a few, higher risk technical or market uncertainties. The market is known and developing so speed-to-market is a critical factor in commercial success. Frequently the firm has significant technical competency in the product category and is skilled at quality production in this product arena.
In this variety of wagile NPD project management, sprints are applied early in the process to address specific technical questions or to gain customer feedback for a particular product feature. The business case is documented upfront and project requirements are known within a wide bandwidth. The iterative sprints are used to answer some specific questions so that the product design can be locked. Once those design requirements are determined, through a series of iterative technology or market experiments, a traditional waterfall process is followed for prototype generation, technical development, and commercial launch.
An example of a product for iterative wagile development is the iPad™. The market was generally known and growing as the use of eReaders was expanding in 2010. However, the iPad touchscreen required technical design beyond the smaller iPhone™ screens and some user experiences needed further testing. However, once these design specifications were frozen, technical development and product manufacturing followed Apple’s traditional project management models – the same used for other existing products.
In other cases, new products are really new platforms built to serve customers and users with new technologies and with novel applications. Consider, for instance, wearable fitness trackers. The goal of the NPD project is to deliver quality and to meet customer satisfaction objectives. Customers needed to be educated on how the product worked yet were knowledgeable about the intention and utility of the product.
For the commercialization of a fitness tracker, an incremental wagile NPD process is appropriate. While technical and market uncertainties were both high, it was important to test proofs and prototypes in the marketplace. A minimally viable product that simply counted steps was a first version of the product. NPD teams gained market insights from the niche customers using an MVP and develops a second version of the product, purchased by a growing customer base.
Again, technical requirements are developed based on customer insights and feedback from using the real product. Another, more sophisticated version is released based on this new market information, and the cycle repeats itself again and again. Fitness trackers started with the minimal functionality of counting steps then added customer-desired features incrementally: step length, heart rate, and Bluetooth integration. Each feature was tied to a new product release resulting in many different product versions in the marketplace, so a wide variety of customers can be satisfied with the overall brand.
Incremental wagile is an especially useful project management approach for a new product category. Fitbit™, like Kleenex™ and Xerox™, identifies the product category for fitness tracker today, yet functionality is radically more complex than the original step counters released as MVPs just a few years ago. An advantage of this methodology is that customer insights are used in feature prioritization and development, yet a disadvantage is that the company must support a large number of different commercial products.
Applying Wagile to NPD
Each NPD project is unique in some way. Thus, applying the “right” NPD and project management process approach requires evaluating several variables for the project. Some of these factors include the following.
- Technical uncertainty
- Market uncertainty
- Customer availability
- Company culture
- Team structure
- Competitive threats
However, most NPD projects can be successfully implement using a waterfall (staged and gated), agile (Scrum), or hybrid (wagile) approach depending on the project characteristics. Firms also must be accepting of the fact that no single NPD or project management approach is one-size-fits-all. Instead, each NPD project should be executed based on it own unique characteristics and risk profile.
We invite you to learn more about “wagile” new product development in an upcoming Agile NPD or New Product Development Professional (NPDP) certification workshop. Check out our full class schedule at Simple-PDH.com. Please contact me if you’d like a free pdf copy of the Scrum Body of Knowledge (SBOK) and check out our current course list for Agile NPD and Scrum here. Feel free to contact me at firstname.lastname@example.org or 281-280-8717.
Some great references on traditional and agile project management include the following. I also dedicate an entire chapter to NPD processes in NPDP Certification Prep: A 24-Hour Study Guide, and you can find additional references at https://globalnpsolutions.com/services/npd-resources/.
- Winning at New Products by Bob Cooper
- New Product Forecasting by Ken Kahn
- Essential Scrum by Kenneth Rubin
- The Lean Startup by Eric Ries
- Effective Project Management by Robert Wysocki
- Being Agile by Leslie Ekas and Scott Will
- Making Sense of Agile Project Management by Charles Cobb
© 2018 Global NP Solutions, LLC
Your Training Partner for NPDP Certification, Project Management, and Engineering Leadership