3. 3
Abstract
Non-flexible product development methods have been outdated since the 1980s. Yet, they are
still one of the most widely used methods for large corporations, due to the ease of control and
extensive documentation requirements. Companies will need to shift to agile product
development if they seek to gain a sustainable competitive advantage in today’s fast-paced,
highly technical business environment.
Utilizing an agile business model (or even merely utilizing some agile practices) will allow a
firm to increase lean ideologies, productivity, and team cohesiveness. Some sacrifices will need
to be made in order to achieve this, such as a flat business structure, cross-trained teams, and less
documentation.
We have constructed two iterations of an Agile Report Card that quantitatively measures how
agile a firm is. The report card is meant to show companies where they are currently utilizing
agile business practices and where they can improve. Seven case studies were analyzed in order
to show the methodology behind the Agile Report Card.
4. 4
Introduction
Overview of Agile Development
Agile development methods have been around for a long period of time, with incremental
software development seen as early at 1957 by Gerald M. Wineburg when he was working for
IBM. (Basili, 2003) Wineburg states in an interview that, “All of us, as far as I can remember,
thought waterfalling of a huge project was… ignorant of the realities. I think what the waterfall
description did for us was make us realize that we were doing something else, something
unnamed except for 'software development.’” (Basili, 2003) These words by Wineburg do a
great job of framing why agile development is so important: because traditional (non-iterative)
development methods were already viewed as outdated as early as 58 years ago.
Agile development has been adopted by many firms and individuals over the years in order to
make software development, team projects, and new product development more simple and
flexible. Many companies choose agile development methods because they promote change, rich
communication channels, and lean practices. All of these aspects make agile development
methods increasingly important, because technology is forcing firms to deliver successful
products and adapt to new consumer values in a very short period of time.
Before any agile methods are described, it is important to note that they are not meant to be
limiting, but are meant to be a productive way to assist a firm in achieving various goals. (Fowler
and Highsmith, 2001) Therefore, agile methods are meant to be changed and sculpted in order to
meet the productive criteria for specific firms, projects, or teams. Fowler and Highsmith (2001)
illustrate this nicely when they state:
In the turbulent world of business and technology, scrupulously
following a plan can have dire consequences, even if it's executed
faithfully. However carefully a plan is crafted, it becomes dangerous
if it blinds you to change.
The Agile Manifesto
The Agile Software Development Alliance was founded in 2001. The alliance was created in
order to create clarity and unity for various agile methods and to inform others about the
underlying values involved in agile development. Martin Fowler and Jim Highsmith state in their
2001 article, “We embrace modeling, but not merely to file some diagram in a dusty corporate
repository. We embrace documentation, but not to waste reams of paper in never-maintained and
rarely-used tomes. We plan, but recognize the limits of planning in a turbulent environment.”
There are various values in the agile manifesto that place an emphasis on the ability to efficiently
respond to unpredictable events, rather than assuming enough planning has been done to respond
to these kind of events. The goal of the manifesto was to have set principles, which recognize
that each team and project are different and therefore no traditional development model can serve
as a solution for all projects.
5. 5
The principles of the Agile Manifesto (Fowler and Highsmith, 2001) are:
Our highest priority is to satisfy the customer through early and continuous delivery of
valuable software.
Welcome changing requirements, even late in development. Agile processes harness
change for the customer’s competitive advantage.
Deliver working software frequently, from a couple of weeks to a couple of months, with
a preference for the shorter timescale.
Business people and developers work together daily throughout the project.
Build projects around motivated individuals, give them the environment and support they
need and trust them to get the job done.
The most efficient and effective method of conveying information with and within a
development team is face-to-face conversation.
Working software is the primary measure of progress.
Agile processes promote sustainable development. The sponsors, developers and users
should be able to maintain a constant pace indefinitely.
Continuous attention to technical excellence and good design enhances agility.
Simplicity – the art of maximizing the amount of work not done – is essential.
The best architectures, requirements and designs emerge from self-organizing teams.
At regular intervals, the team reflects on how to become more effective, then tunes and
adjusts its behavior accordingly.
An taxonomical overview of all of the agile development methods can be found in Appendix A.
The goal for this research report is to analyze the strengths of agile development and determine
how they can be used to assist with new product development. Literature on various agile
development methods and types of business models will be analyzed and a quantitative agile
evaluation model will be produced from this research.
Literature Review
The Waterfall Model
The Waterfall model is one of the oldest known development methods and was introduced
officially by Winston W. Royce in 1970. This model utilizes design, implementation, testing, and
deployment as a linear sequence of processes. The processes, as explained by Royce (1970) are:
6. 6
All of these steps require heavy documentation and the first four steps are often performed by
upper-management. This method is easy to control and manage, since the process is very rigid
and there is little autonomy for low-level employees. Waterfall is very widely used for large and
complex development projects for these reasons. This method has also been adopted for medium
and small-scale projects, because it is often viewed as a safe choice by management.
This method is described as the “stereotypical traditional method” of software development by
Feng in his 2011 article. Feng identifies the Waterfall method as being effective for large-scale
and complex projects, but also delves into the drawbacks of the method. For instance, the rigidity
of the method leads to high levels of inflexibility. This is seen late in the process and is one of
the reasons why the method is named after a Waterfall. Each step directly leads to the next and
the process is always moving forward. If a change must be implemented toward the end of the
process, it is very costly and time-consuming. High levels of documentation can also be a
hindrance to change, since it is not necessary in all projects and tends to slow the process down.
The process is illustrated below.
The Waterfall Model has recently gone through some scrutiny, despite its widespread use. For
instance, in his 2012 article David Dischave questions whether a true Waterfall model has ever
been implemented without some form of iteration. Most diagrammed uses of the waterfall
method have some form of iteration present, just like the example in the graphic below.
1 Conception
2 Initiation
3 Analysis
4 Design
5 Construction
6 Testing
7 Maintenance
7. 7
In this model, the same original framework is present, but various feedback loops have been
added for flexibility. With this in mind, most recent literature discounts the pure and non-
iterative form of the Waterfall Model to be unrealistic. In fact, Dischave (2012) suggests that this
model’s only contemporary use is as a straw-man to advocate for iterative and agile design
methodologies. This sentiment was also stated by Barry Boehm, who viewed non-iterative
development models as incomplete and needing improvement in 1988.
The Spiral Model
The Spiral Model was described by Boehm (1988) as a risk-driven approach to software
development rather than a document-driven or code-driven process. In practice, a project being
managed under the Spiral Model is divided into portions and each portion is evaluated for
alternative means of implementation relative to the objectives and constraints of the project. As
seen in the figure below, the Spiral Model consists of four phases which are revisited constantly
until a project is complete.
The four phases of the Spiral Model (Boehm, 1988) are:
1. Determine objectives, alternatives, constraints
2. Evaluate alternatives, identify, resolve risks
3. Develop, verify next-level product
4. Plan next phases
As seen in the graphic below, the first phases begin with the concept of requirements, concept of
operation, and requirements plan. As each phase is completed, the process moves along its spiral
path and continually loops through each phase all over again. This is repeated until the product is
released.
8. 8
Within each spiral cycle, certain identifications need to be made in order to ensure that risks are
discovered. At the beginning of each cycle, the goals for the portion of the product, various ways
to implement the portion of the product, and constraints are placed on alternative applications.
Then, the generated alternatives are reviewed to see how well they relate to the constraints and
objectives. If there is a perceived imbalance between the constraints, objectives, and alternatives,
it is considered to be an area of uncertainty and will be considered as a potential risk. If a risk is
found, the next step should be tailored to find an inexpensive and effective way to resolve the
source of the risk. (Boehm, 1988)
Each spiral cycle has the possibility of resulting in new prototypes in order to correct any risks
that were identified. This portion of the spiral method is very flexible, as it depends on if a firm
is “specification-oriented, prototype-oriented, simulation-oriented, automatic transformation-
oriented” (Boehm, 1988) or if the firm has any other type of focus for the product. The
orientation of the firm or specific product will affect how many prototypes are made and how
much time and effort should be dedicated to producing prototypes.
The Spiral model is a very costly way to do product development, as the initial and continuous
research, along with the building of prototypes are very costly for a firm. Management will also
need to be extremely competent in risk-assessment in order to correctly recognize and respond to
risks (the potential to go over budget, physical problems with a product, or major bugs in a
program.) The steps in the process are also fairly vague and can be changed in order to match
with the desires of upper-management. This can be a positive aspect for a firm, but can
potentially result in miscommunication if proper documentation is not completed by the firm.
Most of the difficulties with using the Spiral model can be addressed only if the firm has enough
resources to spend on training, research, and prototypes.
9. 9
Extreme Programming
The reason Extreme Programming (XP) exists is because Chrysler’s internal payroll system in
1996 and 1997 benefitted immensely from this method. Beck (2000) states that the “common
sense” approach using the 12 rules above in the figure led to a “Tried and trusted system that
established an orthodox way for conducing software engineering.” (Beck, 2000) The XP
feedback loop process is pictured below.
The first phase that takes place in XP is the “exploration” or research phase. This phase takes
anywhere from a couple of weeks to a couple of months to complete, depending on the size of
the project. In this phase, the team will familiarize themselves with the task at hand. This
includes experimenting with different pieces of technology and tools to see what could prove
useful for the current project. Research is the key for this step to be productive, because
management will need to know which kinds of technology to experiment with.
The next phase is the “planning” phase. In this phase, the team will spend time working with the
customer in order to prioritize the capabilities that need to be met. Once the capabilities are
prioritized, management will use the information to create a schedule for the project. Most
managers try their absolute best to keep the time for projects under two months. Obviously some
projects may require more time to complete, but two months is the general rule of thumb for
agile project development. (Beck, 2000)
After the research and planning have taken place, the next phase is the “iterations” phase. This
phase requires testing to be done on various prototypes that have been developed. Each test can
take 1-4 weeks depending on the amount of testing that will be needed to complete the final
product. This phase is the most productive when the product development team has many
10. 10
prototypes to test and can help quickly narrow the options down to the most promising options.
Once testing has been completed, the team is then ready to start the “productionizing” process.
The first course of action that a product development team wants to take in the productionizing
phase is to do a full review of the tentative final product. Multiple team members should be
involved in this in order to improve the reliability of the performance testing. This kind of testing
is meant to ensure that the final product is ready to fulfill the needs of its target market. This step
should be taken very seriously, as there are many products that are released without heeding to
the actual needs of the target customer. Any potential changes that were not used in the current
release will be documented and later used for completion. The fourth phase is finished when the
final product is delivered to the customer. (Succi, Stefanovic, and Pedrycz, 2001)
The last phase of XP is an ongoing continuation of the last phase, where the firm continues to
supply maintenance for the final product. In this phase, the team continuously reviews the
product in order to determine which positive changes can be made, based on customer feedback.
These improvements can include corrective changes, perfective changes, and adaptive changes.
This step does a great deal to help the firm improve its product and gain customer loyalty.
As the product reaches its end of use in the market, there are less changes and improvements that
can save the product. The product then becomes obsolete and will become lost in a market,
eventually replaced with a new release. Some effective XP rules to follow are:
11. 11
Scrum
The core concept of the Scrum method concentrates on “how teams can be organized to produce
software in a constantly changing environment. Modeled after the game of Rugby, the Scrum life
cycle consists of three phases: Pre-game, Development, and Post-game.” (Coram, 2005) It
consists of the following concepts: product backlogs, team roles, sprints, burndown charts, daily
Scrum meeting, and sprint retrospective meetings.
The product backlog contains various aspects of a product that need to be completed (also
referred to as user stories,) which are obtained from the perspective of a business and/or end
users. The business then needs to start planning which user stories are to be released. To do this,
the business assembles a project team which completes the following tasks:
The product owner makes sure the features that make it to the developmental stage properly
represent the needs of customers and end users. The product owner also helps set the
direction of the product.
Scrum master (or Project Manager) ensures that the project goes smoothly, makes sure
everybody has the tools necessary to complete the tasks, and acts as facilitator during the
project duration.
A group of developers will build the product.
A group of testers will test the product.
The team will find customers that agree to pay a set price and use the product.
Executives will need to support the project from the customer’s perspective.
When planning a release, the team will have to refer to the product backlog and identify the user
stories that they want to put into the release. The user release will be part of the release backlog.
Prioritization is then applied to the features and then an estimate will be developed in order to
allocate time and resources to each item. In some cases, if an item is too large, it may be broken
down into smaller subsections that are much more manageable. The given estimation will
provide a rough idea of the total amount of work involved to complete the entire release. There
are a lot of techniques in making estimation but the best method is to estimate the work
systematically in hours, days, and months if necessary.
A business can determine several sprints with the combination of prioritization and work effort
estimation. A sprint can have a short lifecycle but it generally ranges between 2 days and 30
days, depending on the product release cycle. Since there may be one or more sprints created, the
sprint backlog will be created containing one or more release backlogs so the product can be
fully developed and tested that at the end of each sprint.
The burndown chart will also be used as project visibility tool to show the project’s progress. It
provides daily measurement of the amount of work that remains in given sprint or release. The
project team will be able to see what the team needs to be in order to complete the task. This tool
12. 12
will be able to show the rate of productivity from the start to finish, which will then lead to
whether or not the project will be completed on time or if it will be delayed. As the work is in
progress, each team will update the amount of time remaining and this will indirectly change the
total amount of remaining until the sprint is complete.
Daily Scrum meetings are used as a catalyst for communication among the project team
members to let everybody know about the project progress, the development updates, and
obstacles that may be encountered. A sprint retrospective meeting is also used in order for the
project team to reflect on what has gone right or wrong and possible improvement. The Scrum
methodology emphasizes communication and collaboration and can help business owners
improve productivity, reduce project risk, and enhance software quality by the flexibility to adapt
to emerging business realities. This entire process is illustrated below.
Kanban
Kanban (Japanese for signboard) is a lean methodology developed by Taiichi Ohno as a way to
focus on and achieve just-in-time manufacturing and distribution. Kanban is often used in agile
management and mixed management methodologies, both of which utilize mechanics from both
Kanban and Agile Development. These are often referred to as “Leagile” (Mason-Jones et al.,
2000), which is the modern derivative of Kanban and Agile. At an evaluative and analytical
level, Kanban and Scrum are very similar; both methodologies promote efficiency through
simplification and breaking development into predictable chunks. It is in their application that
the differences between the two systems become apparent, because Kanban does not require
time-boxes, only addresses one backlog item at a time, and does not employ cross-functional
teams.
A key aspect of Kanban is using visual models to visualize various workflows. This is similar to
Scrum in many ways, since both methods use boards in order to help visualize tasks. However,
the boards have a couple of key differences:
13. 13
The above illustration is a generic example of a Scrum board. The board assists with visualizing
various product backlog items that need to be completed for a project, as well as setting time
constraints for each part. The board will show which backlog items are currently in progress,
being reviewed, being tested, and which are complete. This is a productive way to visualize what
kind of work is being done, what still needs to be done, and how long the teams have to complete
certain parts of the project. Once a sprint is complete, the board is reset and reprioritized for the
next sprint.
14. 14
The above illustration is an example of a Kanban board. It is very similar to the Scrum board,
except there is not a section for sprints. This is because the modern application of Kanban
methodologies does away with the time-boxed sprint-chunking of Scrum. Kanban teams limit the
number of features at each point in the development cycle instead of spending a set amount of
time on each feature. This decreases inefficiencies brought on by inaccurate effort estimates. In
Sjoberg’s 2012 case study, he identified that, “The combination of daily stand-up meetings, the
visibility of the items’ status on the board, and the personal ambitions to complete the job” were
all aspects of the Kanban board that would incur sufficient pressure and motivation on
employees, which would balance the lack of time-boxes in a Scrum to Kanban management
change.
By only addressing one backlog item at a time, Kanban teams eliminate sprint start-up meeting
waste. If a firm allocates time to every feature in development, it can cause meeting length to
inflate. This can lead to unnecessarily elongated meetings for developers, whose time would be
better spent coding. The caveat for eliminating time-boxes can be easily seen when a team is
disorganized or the vision for the project is unclear. This leads to management having to get
more heavily involved and spending more time organizing and supervising meetings, which
defeats the purpose of eliminating time-boxes in order to save resources. Therefore, management
should choose whether Scrum boards or Kanban boards should be used independently between
projects and between teams in order to decide if time-boxes are necessary for the particular
project.
Benefits of Agile Methods Withina Traditional Business Model
Now that various agile developments have been introduced, the relationship between agile
methods and traditional business models will be compared in order to see how the relationship
impacts new product development (NPD). First, it is important to understand the characteristics
of successful product development. Successful product development involves efficiently
producing items that can be sold for profit. Firms need to analyze five different characteristics of
their NPD process in order to make it as efficient as possible: Product quality, Product cost,
development time, and development cost and development capability. According to Ulruch and
Eppinger, (2012) those five can be defined as followed:
Product quality: How good is the product resulting from the development effort? Does it
satisfy customer needs? Is it robust and reliable? Product quality is ultimately reflected in
market share and the price that customers are willing to pay.
Product cost: What is the manufacturing cost of the product? This cost includes spending
on capital equipment and tooling as well as the incremental cost of producing each nit of
the product. Product cost determines how much profit accrues to the firm for a particular
sales volume and a particular sales price.
Development time: How quickly did the team complete the product development effort?
Development time determines how responsive the firm can be to competitive forces and
to technological developments, as well as how quickly the firm receives the economic
returns from the team’s efforts.
15. 15
Development cost: How much did the firm have to spend to develop the product?
Development cost is usually a significant fraction of the investment required to achieve
the profits.
Development capability: Are the team and the firm better able to develop future products
as a result of their experience with a product development project? Development
capability is an asset the firm can use to develop products more effectively and
economically in the future (Ulrich and Eppinger, 2012).
There are some general challenges that firms can face in product development as well. Ulirch
and Eppinger (2012) lay out the challenges by stating, “Developing great products is hard. Few
companies are highly successful more than half the time. These odds present a significant
challenge for a product development team.” Some barriers to successful product development
include:
Trade-offs: Most products must sacrifice something in order to gain more capabilities in
another area. For example, if a smart phone company wants to have a larger screen on
their next phone model, they may need to sacrifice battery life or phone size in order to
achieve that. Managing trade-offs in order to efficiently design products is crucial to the
success of new product development.
Dynamics: Making correct decisions in circumstances that change constantly can be a
challenge for managers as well. Economics, technology, customer preferences, and
competitor products can all affect what kind of decisions are made.
Details: Even minute details for a product can result in millions of dollars lost or gained
by a firm. Therefore, each and every aspect of a product should be carefully considered
before being involved in the final product.
Time pressure: The barriers for new product development can easily be solved if enough
time is allocated to solving each and every one. However, most firms must make
decisions and solve problems under strict time constraints. This can result in incorrect
decisions being made.
Economics: The process of developing and producing new products often require large
investments from stakeholders in the company. A new product must be able to produce
reasonable fiscal results in order to be appealing to customers, upper-management, and
any other stakeholders that are involved. This can be measured by utilizing metrics such
as ROI (Return On Investment) or NPV (Net Present Value).
Creation: The process of creatively developing new products can be long and arduous, as
it begins with an idea and ends with a physical product. This can prove to be difficult for
individuals or teams to accomplish, as the process can begin as a qualitative assessment
of needs and must end with a product that is quantitatively efficient and valued by
customers.
Satisfaction of societal and individual needs: Products are developed in order to satisfy
needs that customers have. New product development must be focused on fulfilling needs
and creating value. Firms can make incorrect decisions if they lose sight of the needs and
values that are fulfilled with a new product.
Team diversity: A firm must have a diverse pool of skills and talents in order to develop a
successful product. Development teams must consist of teams with various levels of
16. 16
training and experience in order to bring fresh ideas and diversified streams of
information to the team.
Team spirit: Product development teams should consist of highly cooperative and
motivated groups of employees and managers. The team members must be able to work
together in a cohesive manner and keep morale high. If not, communication can suffer
and the product development process will not be as efficient.
The chart below gives a brief idea of the steps that NPD takes to come up with a project
review. (Ulrich and Eppinger, 2012)
This shows that traditional product development processes can be fairly rigid and tend to
resemble the waterfall method. In order to fix a problem, a firm must start back from the
beginning stage rather than being able to adapt to problems in any step of the process. Agile
systems such as Scrum, XP and spiral fix that problem by having different teams interact with
each other between steps, which gives them the ability to adapt instantaneously. As previously
mentioned, many agile methods have simple processes, steps, and phases that are repeated
several times. This can help a firm make any necessary changes according to feedback that is
provided by the teams.
It is logical for companies to be more interested in using agile development methods, because
shorter development cycles and process flexibility are crucial for many firms. (Duran, 2013) An
article by Eric Graves (2014) gives a keen insight on the challenges of agile development and
how to successfully integrate agile development within a traditional business model in order to
achieve lean philosophies. Graves (2014) states, “Just as lean manufacturing has some
commonalities with Lean Product Development, Agile development has some commonalities
with Lean hardware product development. Some examples include the value of faster feedback
and earlier learning enabled by smaller batches, test drive development, daily standups, and
17. 17
visual work management. In many cases, a hardware team striving to be more agile is moving in
the right direction.”
In order to successfully integrate agile methods with traditional business models, it is natural that
firms must first understand the differences between the two methods. Graves (2014) uses the
differences between software and hardware development as an example of the relationship
between agile and non-agile practices:
1. Lead Time: Software teams have a relatively short “compile” step, which resides within
the Design-Build-Test cycle. New product development teams have a relatively long
“procure” step. Driving down the length of the procure steps is a key initiative in Lean
NPD today, but NPD is not even anywhere close to procuring parts and building
assemblies in the few minutes it takes to compile software. From the first day of a new
project through its last minute changes, lead-time is ever present and highly impactful.
2. Component Cost: Software development costs are overwhelmingly allocated only to
labor. It is not difficult or expensive to give users full access to the latest and greatest
version of software for testing. However, in product development, the cost of providing
users with working prototypes is substantially higher. Like lead time, component costs
change what a firm should do in order to achieve maximum profit.
3. Non-homogenous Work: Software teams are comprised of individuals that have several
different skill sets such as marketing, design, a few different fields of development, and
quality. In addition to the software skills that are increasingly necessary for the
development of new hardware products, there are often molded components, optics, sheet
metal, castings, cabling, piping, circuitry, assembly, and packaging skills that are needed
by different people who must stay in sync during product development. There are other
additional skills needed in product development as well, such as the research scientists,
supply chain, manufacturing, receiving/inspection, and field service. These are not
typically involved in a software projects. (Graves, 2014)
The keys to successfully integrating agile development methods involve many of the principles
in the Agile Manifesto. These principles tend to be very different than traditional viewpoints and
can result in pushback from employees and management. For example, a company must learn to
embrace individuals and interactions over processes and tools when using agile development
methods. (Fowler and Highsmith, 2001)
Communication and integration of talent needs to be valued over processes and tools,
because talented employees and rich communication channels will enable a
development team to respond and adapt to changes frequently.
Rigid processes are a much more traditional way to manage the development of new products,
but lack the ability to change or garner more creativity from a team.
Changing the way a firm views an accomplishment is something that must be changed in order to
effectively integrate agile product development. With agile development, comprehensive
documentation is replaced with iterative releases of useful software and/or prototypes in order to
gain feedback from employees and customers. Because comprehensive documentation is
replaced in agile development methods, agile teams must focus on breaking down tasks into
18. 18
small, manageable chunks. Companies will need to view potential failures within these chunks as
opportunities for new prototypes rather than as a negative event.
Also, customer collaboration must be valued over contract negotiation. The Agile Manifesto
makes feedback an absolute necessity because customer feedback is crucial to the success of an
agile product. This is compounded when hardware development is involved, because replacing
physical parts of a product is very time consuming and costly. A firm must learn to balance the
costs of collaboration with the amount of money that is saved by reducing documentation. The
amount of valuation that is placed on customer feedback should also be discussed carefully by
members of management. If a firm spends too much time and resources on customer
collaboration, the resources that were saved by reducing documentation will become obsolete.
The final and most important aspect of successful agile integration is the ability to adapt to
changes rather than rigidly following a plan. As previously stated, firms must be able to have
flexibility and tailor each agile practice to the needs of each project. If a firm, its employees and
management are not able to critically solve problems as they occur then it will be very difficult
for the firm to adapt agile development methods. The ability to adapt does not occur naturally
and often requires extensive training. Some resources may need to be spend in order to
communicate and train employees with core-competencies and agile philosophies.
Stage-Gate®: An Innovative Business Model
As mentioned in the previous section, integrating agile development methods into traditional
development models can be done, but there are many obstacles that must be overcome. In many
cases, a solution to these obstacles is utilizing an innovative business model in order to achieve
agile product development. The most successful innovation model is known as Stage-Gate® and
is currently the most popular innovation model in use. According to the Product Development &
Management Association, 69% of the best new product developers in the U.S. use some form of
the well-known Stage-Gate® Process to develop new products.
The original Stage-Gate® was created in 1980. The first documented iteration of Stage-Gate®
was derivative of the waterfall model with a bit more flexibility. Because of the similarities,
many users have complained about the traditional Stage-Gate® model being too linear and too
rigid. The market place is now extremely fast-paced, more competitive, more globalized, and
less predictable. Therefore, the Stage-Gate® model has gone through various changes in order to
make it much more agile.
Dr. Robert Cooper is the creator of this innovative and agile version of Stage-Gate®. He did a lot
of research with leading firms in order to re-think and re-invent the next generation of an idea-to-
launch system. (Cooper and Edgett, 2012) He called the next generation of idea-to-launch system
the “Triple A System,” which stands for Adaptive – Agile – Accelerate. (Cooper and Edgett,
2012) Dr. Robert Cooper describes this process in his 2014 article:
19. 19
Adaptive and Flexible:
o The next-generation idea-to-launch system is adaptive. It incorporates spiral or
iterative development to get something in front of customers early and often
through a series of build-test-revise iterations. The product may be less than 50
percent defined when it enters development, but it evolves, adapting to new
information, as it moves through development and testing. The system is also
flexible insofar as the actions for each stage and the deliverables to each gate are
unique to each development project, based on the context of the market and the
needs of the development process. This is the opposite of an SOP (standard
operating procedure) approach to product development, which prescribes
standardized actions and deliverables. There are also fast-track versions of the
process for lower-risk projects. And in the next-generation system, a risk-based
contingency model dictates that appropriate activities and deliverables be
determined based on an assessment of project assumptions and risks. Finally,
Go/Kill criteria are flexible—there are no standard sets or universal criteria for
each gate—and gates are integrated with portfolio management.
Agile:
o The next-generation system also incorporates elements of Agile Development, the
rapid development system developed by the software industry. For example,
sprints and scrums—short time-boxed increments in which the deliverable is
something that can be demonstrated to stakeholders (rather than
documentation)—are part of the new system. Equally, these new systems
emphasize moving quickly and nimbly from milestone to milestone and rely on a
much leaner system with all waste removed—no bureaucracy, no unnecessary
activities anywhere in the system.
Accelerated:
o The next-generation idea-to-launch system is focused on accelerating the
development process. Projects in the system are properly resourced, especially
major projects, and fully staffed by a dedicated cross-functional team for
maximum speed to market. Activities within stages overlap, and even stages
overlap; the notion of a “stage” is less relevant in this new system. There is more
emphasis on the fuzzy front end, making it sharper and less fuzzy, so that the
project is clearly scoped and key unknowns, risks, and uncertainties identified as
early as possible. Finally, robust IT support is provided to reduce work, provide
better communication, and accelerate the process (Cooper, 2014).
20. 20
In this figure, all five Time-Sequence Stages and Decision Gates are shown.
In today’s fast-paced world, companies can no longer make a stable and rigid product
that meets the customers’ constant needs in the midst of constant changes and high
expectations.
Sometimes, customers are not clear on what they want (or need) and it can be very difficult to
get a 100% accurate product definition prior to development. As Steve Jobs, never a proponent
of traditional market research, famously said, “People don’t know what they want until you show
it to them.” (Isaacson, 2011) Often, requirements simply change in the time that passes between
the beginning and end of development. New customer needs, new competitive products, or new
technological possibilities can emerge and render the original product definition invalid. Smart
firms, especially those doing risky and bold projects, have made the idea-to-launch system much
more adaptive. The product may be less than 50 percent defined as development begins, but it
comes together during development; the product’s design and definition adapts to new
information, customer feedback, and changing conditions on its way to launch. Such firms have
built in multiple spirals or iterations of development that permit experimentation with users (as
shown in the figure above) with each spiral consisting of the following steps:
Build: In each iteration, build something to show the customer—a rapid prototype, a
protocept, a crude working model, an early beta version.
Test: Test each version of the product with customers—let them tell you what they like
and what value they see.
Feedback: Gather feedback on that version of the product from the customer or user.
21. 21
Revise: Reset how the firm thinks about the value proposition, benefits sought, and the
product’s design based on the feedback, and start again. Each loop moves the project
closer to the final product design. This spiral approach promotes experimentation,
encouraging project teams to fail often, fail fast, and fail cheaply, a principle that Jobs
applied throughout his development career at Apple (Isaacson, 2011). Corning has
adopted this approach with 60–90-day plans that include numerous iterations that yield
testable versions of the product as a deliverable at key milestones (often weekly or
biweekly). This looping or spiral development is consistent with two core tenets of the
Agile Manifesto for software development: a focus on quick response to change, and
continuous customer or stakeholder involvement in the development of the product.
(Cooper, 2014)
Many firms have now developed lighter versions of Stage-Gate® with faster tracks to handle less
risky, better-defined, or less complex development projects. Recent benchmarking studies show
that 75% of top-performing businesses use a scalable idea-to-launch process. (Cooper and
Edgett, 2012)
Unlike the traditional Stage-Gate® system, lighter versions embrace the fact that one size does
not fit all in terms of which business method is most practical for each specific project. This
scalable Stage-Gate® system has three different versions of the original Stage-Gate® process.
The first version (as seen in the graphic below) is for large, complex projects that contain high
risk. This is known as “Stage-Gate® Full.” The next is meant for relatively less-risky projects
that require less complexity and are known as “Stage-Gate® Lite.” The last one is for minor
projects that carry the least risks, this is known as “Stage-Gate® XPress.”
Companies are able to save resources and develop products more efficiently by adapting the
Stage-Gate® process to match levels of risk and complexity. The scalable Stage-Gate® models
are flexible and act as a guide to suggest best practices, recommended activities, and likely
deliverables. The product development team has discretion over which activities they execute
and which they choose to not to do. At each gate, the product development teams present their
proposed “go-forward plan,” which is their best attempt at defining what needs to be done in
order to make the project a success. At these gates, the gatekeepers analyze the report and
necessary resources. If it satisfies their requirements, they will approve the go-forward plan and
the product development team is allowed to move onto the next phase.
22. 22
Using a scalable Stage-Gate® system is very important for firms that are looking to improve new
product development. When scalable Stage-Gate® systems are utilized, firms are able to adapt to
different circumstances and even shorten lead time by using Stage-Gate® Lite or Stage-Gate®
XPress. This leads to a more lean development process and more communication between
employees, developers, and management because an innovation model must be decided on
before each project. This is a very big improvement compared to traditional methods that treated
each project equally and relied on documentation rather than rich communication channels to
convey messages between employees and management.
Dr. Robert Cooper’s improvements on the Stage-Gate® model make it truly innovative and
agile, as it was built with agile principles in mind. Putting the Stage-Gate® model into place help
a firm’s new product development more than the mere application of a few agile principles,
because the Stage-Gate® model is an all-encompassing innovative model that helps product
development teams communicate more, be more organized, create and release new products
constantly, and encourages the hiring of creative and motivated employees.
23. 23
Research Questions
The research questions in this paper are based on the literature reviews which revealed that agile
development methods assist with new product development, even when utilized within a
traditional business model. These effects are increased when a business chooses to utilize an
innovation model (such as Stage-Gate®) to promote agile product development. Therefore, since
agile product development improves new product development and innovation models can be
used to implement agile processes, the research questions that arise are:
Is there a way to quantitatively measure the agility of a company?
Can a quantitative agility model help a company move through the phases of Stage-Gate®
more efficiently?
Quantitative Agility Model (The Agile Report Card)
The framework for the Agile Report Card was built on the foundation that Fowler and Highsmith
(2001) introduce in the Agile Manifesto. All of the criteria in the agile report card are based on
the twelve principles of agile development that are contained in the Agile Manifesto. The
purpose of the Agile Report Card is to rate how agile a company is and identify how the
company can improve its agility. The first iterations of the report card had over twenty-five
sections. Each section was formatted as various questions and the answers to the questions would
determine the amount of points the company would receive. Each question was weighted
differently depending on the importance of the topic the question was covering. The weights for
each question ranged from 50 to 500 points. After each question was answered, the points would
be added up and would determine the final score. The most points possible was 2,400.
The original rating system was able to determining if a company is agile, but lacked the ability to
point out problem areas for a company. The first iteration of the report card determined how
agile a company was, but did not show why a company is agile nor where it was succeeding or
failing. In order to achieve more depth with the report card, significant changes needed to be
made to the scoring method.
The first adjustment to be made was the introduction of three main sections: (1) Team Members
and Developers, (2) The Company, (3) Products and Services. Topics were then added to each
section. Each topic still contained a question that determines how many points the company will
24. 24
receive for that section. Every topic within each section is worth the same amount of points. This
was decided on because agile development often includes separation of work into “chunks” and
the avoidance of unnecessary details and documentation. Therefore, the Agile Report Card is
meant to emulate the chunking used in agile development and each chunk is weighted based on
risk, which is a risk-avoidance technique similar to what is used in the Spiral method. (Boehm,
1988) The total amount of points possible is 2,400. Companies scoring 2,400-1,900 are very
agile, 1,800-1,450 is mediocre, and scores below 1,449 are not very agile. An example of the
report card is below.
25. 25
Each section of the Agile Report Card is associated with a different color. Each color represents
the amount of risk involved for each section. Red represents the most amount of risk involved,
yellow represents a moderate amount of risk and green represents the least amount of risk for a
firm. In addition, each section also has a percentage that weighs the importance of the topics.
Since the success of a business heavily relies on the products and services it provides, the
products/services section is weighted the heaviest at 50%. The company serves as a middle
ground for risk because the aspects of that section such as culture change and communication
can be resolved before a project is completed and therefore do not pose as much risk as having a
bad product. The company section is weighted at 33.3%. Having motivated and well-trained
team members and developers is an important element, but employees and developers can
always be replaced or receive heavier amounts of training. Therefore the team members section
is weighted the least at 16.7%.
The purpose for dividing the report card into three sections was to target the reasons why a
company was having trouble being agile. Based on the literature, it was determined that
problems could arise from the product, company, or team members.
26. 26
Overall, the Agile Report Card was made to be simple and easy to understand. Agile companies
focus on being straightforward and reducing unnecessary work and the report card follows the
same principles.
Methodology
Section 1 – Team Members/Developers
This section highlights the team members and developers within the company. It consists of four
topics each worth 100 points each, totaling 400. This section applies the least amount of points.
If team members do not buy into the agile system, they can be replaced with members who do. In
addition, many agile companies tend to reduce the amount of members hired and choose to
cross-train existing employees.
Topic 1.1
Cohesiveness between managers, developers, and Team members.
In order for a company to earn the most points possible in this section, managers, developers and
team members must communicate well and act as a cohesive team. 50 out of the 100 points are
based on the manager’s performance. The manager’s job is to be a mediator who distributes
information, set up team meetings, and works alongside his or her team members. What
managers want to avoid is being too hierarchical. This leads to team members having little
control or autonomy. This is a determent to team cohesiveness, and will likely lead to employees
being less motivated. The other 50 points are based on the coherence of the employees and
developers specifically. An example of a high-scoring company in this area would include cross-
trained employees and developers that constantly seek out new streams of information and
communicate often.
Topic 1.2
Team Member/Developers Performance Rating.
Along with having strong cohesiveness among each other, team members and developers must
also be able to efficiently adapt and demonstrate agile development practices. 50 out of the 100
points are earned by the team members and developers’ ability to quickly adapt to change. They
must be accustomed to working in an agile environment where constant changes are anticipated.
Changes in the product development, cycle time, company goals, and team objectives could
change any time, and every member needs to be able to positively respond to these changes. The
last 50 points is based on how members handle feedback and how well they minimize errors.
Receiving and decoding feedback is important for any agile development method and
minimizing errors is important for implementing lean development practices. Team members
should constantly be examining how each iteration of a product could be altered or corrected for
the preparation of future iterations.
27. 27
Topic 1.3
Team Members/Developers Morale
All points in this section are based on the confidence level and morale of the team members and
developers. In an agile system, members should be empowered with the ability to make their
own decisions. If the members have a frail mindset and lack the qualification to perform well in
an agile system, this could result in the company getting a zero for this entire topic. Agile
development requires having motivated, talented employees and developers that have a large
amount of autonomy.
Morale may not seem as important to many critics. However, research shows that team morale is
vital for a company to succeed. (Fowler and Highsmith, 2001) Team members need know that
their input and effort are important for the company’s success. To make sure this happens,
managers are responsible for conducting team-building exercises. If members are not committed
to the agile approach, the company may need to consider replacing their current members of
management that lack confidence in the agile system.
Topic 1.4
Team Meetings
To keep everyone updated on the progress of development, team members and developers must
conduct meetings on a daily basis. It is important that managers have a fixed schedule that
specifies where the meeting will be held, and the time it will start. 50 points are earned by
managers simply conducting team meetings on a constant basis (more than two meetings per
month,) while the other 50 points are earned by the contents of the meeting. Contents of the daily
meetings should include responsibilities for employees, what has been completed, and what
needs to be completed (creating a burn-down chart is highly recommended.) (Sjøberg, 2012) The
contents of the meeting must be informative, lead to progression of development, and clarify any
lingering problems that employees may have. Uninformative and/or vague meetings will result in
a lower score.
Section 2 – The Company
This section consists of four topics each worth 200 points, totaling 800. The reason why section
two is weighted more that section one is because the company has larger responsibilities such as
hiring team members and developing products. The company must implement changes that give
more power to its members and make sure everyone is well informed.
Topic 2.1
Culture Change
To earn the most points possible for this topic, the company needs show they are capable of
adapting their business model to accommodate agile development methods. The Agile culture
requires that all members and managers to work in a flat business model, where employees have
28. 28
more autonomy while managers take on a supportive role. Companies that are unable to
transition from a tall, hierarchical structure will likely get a low score. The most important factor
to consider is if the company culture emphasizes creating products at a low cost, reducing cycle
time, and giving team members large amounts of autonomy.
Culture change can be a very broad topic with many variables to consider. The aspects of a firm
that are judged are pertain to the culture embracing individuals and interactions over processes
and tools, working software over comprehensive documentation, customer collaboration over
contract negotiation, and responding to change over following a plan. (Graves, 2014)
Topic 2.2
Communication Levels
One of the duties of the company is to relay information down to its members and managers. 100
out of 200 points are earned by how well the company can do this. This means that the company
must be dedicated to sharing pertinent information when it is received. This kind of information
may include tests results from a prototype, feedback from customers, or methods on how to
improve product functionality. Overall, the content of the information must be informative.
Managers must relay the company’s mission and objectives constantly to team members in order
to remind them of the overall goal and vision of the company. The last 100 points are based on
how rich the communication channels are within the company. If face-to-face communication is
the preferred method of communication, the firm will get a high score in this section. If the
company prefers to communicate through memos or e-mail primarily, the firm will get a
relatively low score.
Topic 2.3
Lean Integration
To earn the most point possible in this topic, the company must show that it can successfully
reduce costs of new product development. This is because agile development methods focus on
reducing various costs such as overhead, data space, and the amount of personal needed for a
project. A firm will obtain a high score in this topic by making proper decisions on which agile
development methods to choose from, based on what each specific project requires. A firm will
receive a relatively low score, for example, if the Spiral model is used for a project that does not
need that level of research or number of prototypes.
Topic 2.4
Tools Instituted
A firm needs to make sure that all of its members are equipped with the tools necessary to
complete various jobs. Some examples of tools provided are manuals, product specification
sheets, idea selection scoreboards, and burn-down charts. 100 out of 200 points are gained based
on the usefulness and pertinence of the tools the company provides. The amount of tools used
29. 29
does not necessarily matter, as long as employees are properly educated on how to do their jobs.
The quality of the tools are judged rather than the quantity.
The last 100 points are gained based on the communication channels that are used to distribute
knowledge to employees. A firm will gain a high score by utilizing digital means paired with
face-to face communication in order to train employees. This is because a rich communication
channel is paired with an agile communication platform, which enables training information to
be changed quickly and efficiently. If a firm is using paper books to communicate all training,
this can lead to a low score because changing the training regimen will be difficult to execute
and will result in wasted resources.
Section 3 – Products and Services
This is the most important factor in agile development. A firm may have great employees and a
stable company with superb management, but all of that will not matter if the firm cannot design
and release a product that is valued by consumers. It is important that developers find errors and
mistakes before a product launches by conducting several tests and looking at previous iterations.
This section consists of four topics each worth 300 points, totaling 1,200.
Topic 3.1
Customer Satisfaction
To earn the most points possible for this topic, the product or service must satisfy the needs of
consumers. This can be measured in the short term by analyzing all customer feedback. This
section is graded by how well a company’s product or service has fared in the market in the eyes
of the consumers. If consumers like and value the product or service the company has produced,
this will result in a high score. If company’s product or service has not done well in the market,
this generally leads to having a lower score. However, lost points can be regained if a company
uses a lackluster product release as a point of feedback in order to make improvements to the
product.
Topic 3.2
Product Performance
This topic measures the long-term use of a product or service. 150 points out of 300 are earned
by how well the firm’s product or service performs over a long time period, such as three to five
years in use. If the product breaks down before the consumer has gotten full perceived value out
of the product, the consumer will likely look elsewhere for a replacement. Having products that
are perceived to have a low price-to-perceived-value ratio will devalue the company and brand.
The second 150 points depend on how well the company manages any recalls or defects the
product has. Many of these issues don’t come up till later in the product life cycle, but it is the
responsibility of the company to address these issues if they come up. A firm will receive a high
score if no defects or recalls are necessary,
30. 30
Topic 3.3
Quality of Product or Service
150 points out of 300 are earned by the quality of the product. If a firm is successfully using
agile development to assist with new product development, released products will be perceived
as high-quality products that create value for consumers. The last 150 points are based on how
well the quality of each iteration of the product or service is improved. In order to gain a high
score in this area, a firm must focus on each iteration having better quality than previous
iterations. The best way to measure quality is examining product performance from previous
iterations. This is achieved by customer collaboration and utilizing feedback. (Fowler and
Highsmith, 2001)
Topic 3.4
Product and Service Preparation
In order to earn the most points possible for this section, product designs must be on time, meet
all product requirements, and exploit any defects to be fixed before launch. Products and services
must also pass all final testing for safety and any other requirements. If a product or service
launches with little attention paid to detail and is plagued with glitches, it will result in a lower
score. The important factor here is making sure the products are fully ready for launch when the
time comes. A firm will gain a high score in this area by being able to pay attention to details,
have an effective product preparation process, and be able to produce a quality product/service
that will be able to effectively perform on the day of release.
Case Studies
Case Studies
Various business case studies were selected in order to test the veracity of the Agile Report Card
and to test how agile the companies were. The criteria for selecting case studies included a focus
on new product development, a structural representation of the model, and a balanced
presentation of agile development. Because the Agile Report Card model is targeted toward
judging the agility of a company’s new product development as opposed to project management,
it was important to seek out business case studies that directly discussed new product design.
Quantifying project management agility has been described before by Vinodh et al. (2008), so
the focus of the Agile Report Card was to specifically quantify the agility of new product
development processes.
The analyzed business case studies needed to structurally display agility in order to comply with
the Agile Report Card. Applying the Agile Report Card to case studies that did not directly
discuss agile ideas would not provide useful information at this time, because the goal was to
prove the usability of the Agile Report Card evaluation model as opposed to gathering
information about agility in the larger business ecosystem. Case studies were sought out that
31. 31
displayed a balanced presentation of agile concepts. Some case studies that were produced by
agile consulting companies did not address shortcomings of agile practices and therefore could
not be used.
Twelve case studies were initially gathered. From these twelve the case studies, Plannon, Dell,
and Daimler were discarded. The Plannon case study dealt with Software Product Management
as opposed to New Product Development. The Dell case study did not contain specific enough
information about agile practices and was focused on supply chain logistics instead of New
Product Development. The Daimler case study was also focused on logistics instead of New
Product Development.
Applying the Model
The seven remaining case studies were read and examined for language directly relating to the
use of agile concepts. These case studies all held the characteristic of describing the application
of agile concepts to firms that did not previously utilize agile development processes. High value
was given to quantitative language in the case studies. Examples of numerical changes in
productivity were always used for considering the agility of the companies due to the objective
measure that numerical changes present. Subjective language dealing with vague or ambiguous
references to agility were not considered when applying the model to the case studies.
In between these was a grey area where it was up to the individual applying the model to
determine if a reference to agility was strong enough to merit consideration. Once the
determinant language was extracted, it was considered with reference to the criteria laid out in
the Agile Report Card and a numerical score out of the total possible points was assigned. The
maximum value for each subject was interpreted as a maximum theoretical display of the
concepts described in the Agility report card.
A hybrid of subtractive and additive score assignment was used due to the unpredictability of
applicable language in each case study. Negative references to stated concepts caused a
subtraction from a maximum score under a subject while positive references increased the score,
but not beyond the maximum score. A truly additive methodology where scores started at zero
and increased based on positive displays of agility would have been preferred over this hybrid
approach.
A total agility score was assigned to the companies and then the application of the model to the
different case studies was considered. A discussion of the specific case studies, their
idiosyncrasies relating to the Agility report card, and examples of the language extracted for
application follows.
Danube Technologies: Intel Silicon Chip Product Development
This case study was produced by Danube Technologies, which is a firm that acted as the agile
consultant for Intel. Intel was implementing agile methods for the development of a new silicon
chip and brought on Danube to oversee their implementation. Because of Danube’s role as an
32. 32
overseer, they directly address problems with Intel’s agility experiment which made this case
study a candidate for inclusion.
Examples of Extracted Language:
Cohesiveness: After initial skepticism the Intel teams embraced cross-functional teams working
under Scrum.
Team Member/Developer Performance Rating: No employees lost during transition.
Team Member/Developer Morale: Huge backlogs caused some teams to feel bombarded, some
micromanagement from project owners.
Team Meetings: Predictable and timely, but could be more frequent (weekly vs daily).
Culture Change: Story points very seamlessly integrated, agile development still used after two
years.
Communication Level: Across the board transparency increased, revealing bugs, weak tools, and
poor engineering with some isolated problems.
Lean Integration: 66 percent decrease in cycle time, velocity reduced when sprints interrupted.
Tools Instituted: Cross functional teams, Scrum, Incremental development and review, Story
Points.
Customer Satisfaction: Increased quality of products but no documentation of customer response
in case study.
Product Performance: Silicone development quality increased.
Quality of Product/Service: Increased quality of products but no specific details in case study.
Product/Service Preparation: Increased quality of products but no specific details in case study.
VersionOne: Siemens Health Services
This case study was produced by VersionOne, which is an agile consulting company. While the
case study contains “a success story” in its title, the case study still referred to barriers that
occurred while implementing a fully agile model and discusses hybridizing agile with the
produce development model previously employed by Siemens. VersionOne introduced digital
Kanban cards into Siemens’ software new product development and then discussed their
effectiveness and the transition from a time box model to the continuous flow of Kanban.
Examples of Extracted Language:
Cohesiveness: 40+ teams on 3 continents switched to Kanban, Enterprise level shift in product
development model.
Team Member/Developer Performance Rating: Initial 15 team test only needed one year to prove
effective.
33. 33
Team Member/Developer Morale: Workers quickly adapted to new model but less than one
hundred percent retention during switch.
Team Meetings: None addressed in case study.
Culture Change: 4 years of continuous use of Kanban model.
Communication Level: Increased communication between departments and between senior
management and production.
Lean Integration: Reduced administrative maintenance 70 percent.
Tools Instituted: Continuous flow process, limited WIP, Kanban boards.
Customer Satisfaction: Development improved but customer related specifics not present in case
study.
Product Performance: Continued use of agile gives evidence for product improvement.
Quality of Product/Service: First release after Kanban delivered on time and 10 percent under
budget.
Product/Service Preparation: Increased throughput by 33 percent.
Universal Credit
This case study details the attempt to introduce agile new product development to Universal
Credit: the English Government’s welfare reform project. The reformation of a tall, hierarchical
government project proved too much for the program and agile practices were eventually
abandoned. However, the company did continue to show some variations of agile practices even
after the methods were discontinued.
Examples of Extracted Language:
Cohesiveness: Required organizational culture change failed to take shape, “Never agile in the
first place.”
Employee/Developer Performance Rating: Employees had trouble being productive while
changing their work styles.
Team Meetings: Command and Control culture.
Employee/Developer Morale: Waterfall contracts favored waterfall methodology, developers
reluctant to change.
Culture Change: All agile software development ceased due to existing command and control
culture. Up to 37 percent unwilling or unable to adapt.
Communication Level: Training occurred but did not stick, waterfall methodology maintained
for project.
34. 34
Lean Integration: Bloated government bureaucracy favored along chain of control from
production to administration.
Tools Instituted: attempted but abandoned cross functional teams, scrum and sprint
methodologies.
Customer Satisfaction: Product development was not improved by agile, no benefit to customer,
end result was increase in project cost
Product Performance: Performance was poor.
Quality of Product/Service: Failed
Product/Service Preparation: Failed
Alias Sketchbook Pro
Under guidance of an agile systems consultant, Alias developed the Sketchbook Pro product for
Autodesk. User-centered design was central to their new development methodology.
Examples of Extracted Language:
Cohesiveness: Agile only applied to one initial product.
Team Member/Developer Performance Rating: Some friction in understanding roles at different
points in cycles such as feature capture and design implementation.
Team Member/Developer Morale: No record of lost employees or morale problems.
Team Meetings: Standard Scrum Meetings.
Culture Change: All subsequent releases developed with agile.
Communication Level: Customer communication increased, cross-team communication adopted
as standard practice.
Lean Integration: Very targeted market and audience, final development model improved but
more complex than intended.
Tools Instituted: Adaptive Development, Scrum, Extreme Programming, User stories.
Customer Satisfaction: Increased customer input and improved management of customer input,
number one provider of product.
Product Performance: Maintained position at head of field.
Quality of Product/Service: Very High.
Product/Service Preparation: Development improved but could have improved feature capture,
role clarity, design implementation processes.
35. 35
Marel
Marel introduced agile concepts under the guidance of a student writing a report on
implementation of agile practices. The inexperience of the student consultant resulted in many
concepts not sticking and employees favoring a skeptical view of new ways of working.
Examples of Extracted Language:
Cohesiveness: Mechanical product development investigatory team used agile. Reliance on non-
agile suppliers affects NPD agility.
Team Member/Developer Performance Rating: Rookie scrum master, stubborn team members
unable to handle detailed work breakdown, reject cross-functional teams.
Team Member/Developer Morale: Team members openly skeptical and critical of new product
development framework, reject many agile ideas, uninvolved product owner.
Team Meetings: Standard scrum meetings adopted.
Culture Change: Some mechanical product development scrum aspects maintained such as stand
up meetings, planning, consistent questioning of agile and scrum methods.
Communication Level: Communication increased, but Transparency required for Agile not
achieved. Scrum master acted as security guard to framework.
Lean Integration: Education/training cut by half, team sizes reduced, scope creep.
Tools Instituted: Stand up meetings, sprint.
Customer Satisfaction: Work items built on technical abilities, not customer needs.
Product Performance: Low effect on final product
Quality of Product/Service: Successful product delivery, on time on budget. Suppressed
creativity potentially decreases final quality
Product/Service Preparation: Decreased in-house Product Development bottlenecks, suppression
of creativity, met final release goal.
SAAB
Before explicitly introducing agile methods to new product design SAAB already operated under
a matrix organization that utilized cross-functional teams. This case dealt with product
development on a single team as the firm tested the application of a modified Scrum method
under internal guidance. Modified schedules for team meetings and sprints decreased the agility
of this application of agile product development.
Examples of Extracted Language:
Cohesiveness: Scrum applied to one design project, no outside consultation.
36. 36
Team Member/Developer Performance Rating: Poor adherence to prescribed agile methods, PM
values modifying agile to fit their comfort zone.
Team Member/Developer Morale: Thought framework worked, liked the agile aspects that were
used. Increased team member happiness and engagement.
Team Meetings: Management only, top-down, planning only.
Culture Change: Only a single one hour meeting devoted to agile onboarding. Visual planning
adopted into company culture.
Communication Level: Team members told what to do, not involved with planning.
Lean Integration: Bulky management structure maintained through agile integration.
Tools Instituted: Cross functional team, “Scum-like” method, visual planning, long sprints,
burndown chart.
Customer Satisfaction: At-risk deadlines met.
Product Performance: Not significantly affected.
Quality of Product/Service: Not significantly affected.
Product/Service Preparation: Improved planning, cohesive and deliberate planning methodology.
Andritz
In this case study, a company-wide switch to agile methods included the teams involved with
new product development. Customer-oriented design was a central idea in the new agile model
and the consultant employed for overseeing the conversion proved to be very effective.
Examples of Extracted Language:
Cohesiveness: Company-wide, small specialized team focus, used consultant.
Team Member/Developer Performance Rating: Strong agile management passed on to team
members throughout organization.
Team Member/Developer Morale: Strong leadership focus on agile, occasional need to “pull”
team members.
Team Meetings: Operational and planning meetings, daily team meetings.
Culture Change: Scrum and Agile based organization, strong reliance on agile principles, self-
improvement process.
Communication Level: Very high communication levels due to effective meeting organization.
Lean Integration: Company-wide restructure, very clear team organization and roles.
Tools Instituted: Cross-functional teams, modified scrum (no project teams), and modified
sprints.
37. 37
Customer Satisfaction: Decreased product development time and cost to customer.
Product Performance: Increased product quality attributed to agile structural changes and
increased team member touch on specialized tasks.
Quality of Product/Service: Instead of developing resources engineering produces products.
Product/Service Preparation: Improved requirements capture, continuous improvement model.
Summary of Agility Ratings
Statistical Analysis
Minimum: 110
Maximum: 2380
Mean: 1722.86
Variance: 695690
Standard Deviation: 834.081
95% CI: 1105 >1722.86> 2341
AR
(out
of
2400)
2280 2280 110 2250 1320 1440 2380
Notes Consultant:
Danube
Technologies
Consultant:
VersionOne
Unable to
overcome
organizational
bureaucracy
Scrum
implemented
with heavy
consultant
guidance and
input
Inexperienced
consultant
Heavily
modified
agile
principles
to fit
existing
company
culture
Strong
managerial
guidance
following
consultant
training
38. 38
The statistical analysis of the Agile Report Card ratings showed a mean rating above 50% of the
maximum possible score, suggesting that there is a high level of agility displayed in the selected
case studies. The outlying low score from Universal Credit introduces very high variance and
standard deviation results and results in a 95% confidence interval failing to capture the highest
possible score.
Discussion of Findings
The Products/Services section was given the greatest weight in the model because rating a
company’s agility is not meaningful if the final product is not improved through application of
agile principles. Unfortunately, no case studies could be found with a scope requiring the
dissemination of information regarding Customer Satisfaction or Product Performance. Ratings
in this section were largely based on inference. Other sections also proved to be difficult to rate
on a case-by-case basis.
The overall applicability of the Agile Report Card model was very high. An overview of
opportunities for increasing agility in new product development proved to be presentable in a
quantitative model. Increasing the specificity of gathered information would significantly
improve the quantification of agility and evaluation of opportunities that can be used to increase
it. Using the model to judge agility based on language found in case studies was effective, but
could be improved by either presenting the model to someone inside the company for them apply
to their new product development processes or by preparing a questionnaire with questions
relating to the rating criteria contained in the report card. This would bypass the potential bias
contained in consultant-produced case studies as well as assist with finding information relating
to new product development agility not readily available in the case study.
A specific problem with the Agile Report Card, as pointed out by Dr. Robert Harmon, was that it
only measured the agility of a company and did not apply to the company’s use of Stage-Gate®
in order to quantify the new product development more accurately. In order to achieve this, an
extra dimension was added to the Agile Report Card so the company’s use of an innovative
model could also be evaluated.
39. 39
In order to design this new model, the Agile Report Card was applied to each phase of the Full
Stage-Gate® process and the topics were trimmed and re-weighted in accordance to what was
pertinent to the current phase of Stage-Gate®. This allows a firm to determine the most crucial
Stage-Gate® phase that needs work and provides specific information on how to improve agility
within the Stage-Gate® model.
Agile Report Card 2.0
As previously stated, the Agile Report Card 2.0 is meant to specifically assist a firm with
determining how agile they are within each phase of Stage-Gate®. This is achieved by creating a
tailored report card for each phase of Stage-Gate®. This will result in a total of five custom
report cards. Firms will then be able to choose a specific report card based on which stage of
Stage-Gate® that is giving them the most trouble. In order for firms to know which phase they
are having trouble with, a quantitative criteria was made so firms could easily analyze which
phase of Stage-Gate® was causing the most trouble (also known as the Critical Phase.) The
criteria is pictured below.
Once the Critical Phase has been identified, the firm will then choose the pertinent Agile Report
Card 2.0 in order to approve agility within the Critical Phase. The five versions of the Agile
Report Card 2.0 are pictured below. The grey topics signify irrelevant topics to the current phase.
Trouble with creative thinking or idea
scoping
Phase 1
Trouble defining the product or planning
the project
Phase 2
Trouble executing the product plan,
producing prototypes, or identifying
correct product components
Phase 3
Trouble with beta testing or gathering
feedback from customers
Phase 4
Trouble with low percieved product
quality or dissatisfied customers
Phase 5
Criteria For Identifying the Critical Phase
40. 40
Phase 1 deals with idea scoping. This phase requires a great deal of focus to be placed on the
agility and cohesiveness of team members and managers. If a firm is having trouble with this
phase, it should investigate the team members/developers section in order to become more agile
in that area. Some topics in the company section should be investigated as well, as
communication levels, culture change, and tools instituted can be a hindrance to idea scoping.
(Cooper, 2014)
Sections Topic
Cohesiveness b/w managers,
developers, and team members
Team Member/Developer performance rating
Team Members/Developers
Team Members/Developer moral
Team Meetings
Culture Change
Communication level
Company
Lean integration
Tools Instituted
Customer Satisfaction
Product Perfomance
Products/Services
Quality of Product/Service
Product/Service Preparation
Agile Report Card 2.0 (Phase 1)
41. 41
Phase 2 deals with building business cases. Similar to the last phase, this one requires
investigation into the team members/developers section and the company section. The most
important topics to investigate are cohesiveness, team morale, team meetings, and
communication levels. Planning the project should be done with ample feedback from
developers and employees and developers should also be heavily involved with defining the
product. A company may need to use richer communication channels or have more team
meetings in order to increase cohesiveness.
Sections Topic
Cohesiveness b/w managers,
developers, and team members
Team Member/Developer performance rating
Team Members/Developers
Team Members/Developer moral
Team Meetings
Culture change
Communication level
Company
Lean integration
Tools Instituted
Customer Satisfaction
Product Perfomance
Products/Services Quality of Product/Service
Product/Service Preparation
Agile Report Card 2.0 (Phase 2)
42. 42
Phase 3 deals with development. By this step in the process, culture change should already be in
place. Customer satisfaction is also greyed out because that is not a problem area until after the
product is launched. However, many topics in the team members/developers, company, and
products/services sections must be investigated in order to see where agility needs to be
improved. The problem could lie anywhere from non-cohesive team members to ineffective
product/service preparation. A firm must carefully analyze these areas in order to see where
agility improvements can be made.
Sections Topic
Cohesiveness b/w managers,
developers, and team members
Team Member/Developer performance rating
Team Members/Developers
Team Members/Developer moral
Team Meetings
Culture change
Communication level
Company Lean integration
Tools Instituted
Customer Satisfaction
Product Perfomance
Products/Services Quality of Product/Service
Product/Service Preparation
Agile Report Card 2.0 (Phase 3)
43. 43
Phase 4 deals with testing and validation. In order for proper testing and validation to be
completed, the quality of the product should be inspected and its preparation should be
scrutinized. However, team cohesiveness and communication levels must be high as well in
order for positive changes to be made. (Fowler and Highsmith, 2001)
Sections Topic
Cohesiveness b/w managers,
developers, and team members
Team Member/Developer performance rating
Team Members/Developers Team Members/Developer moral
Team Meetings
Culture change
Communication level
Company Lean integration
Tools Instituted
Customer Satisfaction
Product Perfomance
Products/Services
Quality of Product/Service
Product/Service Preparation
Agile Report Card 2.0 (Phase 4)
44. 44
Phase 5 deals with the successful launch of a product. If there is trouble in this area, every aspect
of the product section should be analyzed in order to determine the specific problem area. Team
cohesiveness and daily meetings are essential in this phase, as quick changes may need to take
place. This is especially true after a software launch, as consumers tend to discover bugs that
were missed in the testing and validation phase.
Conclusion, Recommendations, and Future Research
This research paper has established that agile development methods greatly assist with new
product development and innovation models (such as Stage-Gate®) assist with new product
development in an even greater measure. The Agile Report card was able to place previously
qualitative aspects of agile development (taken from the Agile Manifesto) into quantitative topics
that could be used to measure the agility of a company. The Agile Report Card 2.0 takes the
evaluation a step further by helping firms identify which phase is the Critical Phase and helps
identify which topics need agile improvement.
Based on the complexity of this research report, it is obvious that integrating, executing, and
analyzing agile development practices can be a complicated matter. Based on the research and
case studies that were done, it was uncovered that firms succeeded with agile integration and
execution when an outside agile consultant was utilized. Utilizing the Agile Report Card or Agile
Sections Topic
Cohesiveness b/w managers,
developers, and team members
Team Member/Developer performance rating
Team Members/Developers
Team Meetings
Culture change
Communication level
Company
Tools Instituted
Customer Satisfaction
Product Perfomance
Products/Services
Product/Service Preparation
Team Members/Developer moral
Lean integration
Quality of Product/Service
Agile Report Card 2.0 (Phase 5)
45. 45
Report Card 2.0 would be recommended before consulting an agile consultant in order to know
where problem areas lie.
The Agile Report Card (and its successor) are good ways for firms to judge the agility of their
development processes, but the rating models do have some flaws. The original Agile Report
Card should be applied to actual businesses as opposed to case studies in order to get more
accurate and less-biased information. The report card could then be improved upon by seeing
which topics and sections prove to be the most important for firms within certain markets or
industries. The Agile Report Card 2.0 needs to be re-weighted and re-assigned with a point
system in order to present companies with a more quantifiable score. As of now, the second
iteration of the Agile Report Card mostly serves as a method for finding the Critical Phase for a
firm.
Research on improving the Agile Report Cards is heavily encouraged, because the report cards
are based on agile practices and should have large amounts of feedback in order to be improved.
Improvement and agility within the phases of Stage-Gate® should also be more heavily
researched in order to determine what a firm should specifically do in each problem situation.
This research report is also meant to encourage non-agile firms to consider utilizing agile
practices and possibly even consider switching to Stage-Gate® for a truly innovative approach to
new product development. Agile development may seem like a method that is difficult to control,
but research and case-studies show that it can be controlled and a company can thrive while
using it.
In order to truly sum up all of the research, case-studies, quantitative models, and suggestions,
here is a quote from the Agile Manifesto: (Fowler and Highsmith, 2001)
“Facilitating change is more effective than attempting to prevent it.
Learn to trust in your ability to respond to unpredictable events; it's
more important than trusting in your ability to plan for disaster.”
46. 46
Bibliography
Ballard, Mark (2013), “Why Agile Development Failed for Universal Credit,” Computer
Weekly, (Accessed March 15, 2015), [available
at http://www.computerweekly.com/news/2240187478/Why-agile-development-failed-for-
Universal-Credit].
Basili, Victor R. (June 2003). "Iterative and Incremental Development: A Brief
History," Computer, 36 (6), 47–56.
Boehm, Barry W (1988). "A spiral model of software development and enhancement," ACM
SIGSOFT Software Engineering Notes, 11 (4), 14-62.
Dischave, David (2012). “A Waterfall Systems Development Methodology … Seriously?” GET
- Global Enterprise Technology. (Accessed January 26, 2015), [Available at:
http://get.syr.edu/news_alt.aspx?recid=401]
Duran, Kelley, Gabbie Burns and Paul Snell (2013), “Lehman’s Laws in Agile and Non-agile
Projects,” Reverse Engineering (WCRE), October (2013), 292-300.
Elwer, Pat (2008), “Agile Project Development at Intel: A Scrum Odyssey,” case study, Danube
Technologies, Portland, Oregon.
Evans, Ian (2006), "Agile Delivery at British Telecom," Methods & Tools, 14 (Summer), 20-27.
Feng, Ji and Todd Sedano (2011), "Comparing extreme programming and Waterfall project
results." Software Engineering Education and Training, May (2011), 482-486.
Fichtner, Abby (2012), "Kanban Is the New Scrum." The Hacker Chick Blog. (Accessed
February 19, 2015), [Available at http://www.hackerchick.com/2012/01/kanban-is-the-new-
scrum.html].
Fowler, Martin and Jim Highsmith (2001), “The Agile Manifesto,” Software Development,
August (2001).
Graves, Eric (2014), "Applying Agile to Hardware New Product Development Environments."
(Accessed March 1, 2015) [Available
at http://playbookhq.co/blog/agileinhardwarenewproductdevelopment/]
Gunasekaran, Angappa (2001), “Agile Manufacturing: The 21st Century Competitive Strategy.”
Oxford: Elsevier.
Hanfield, Rob (2003), “IT’s Contribution: Information Visibility and Systems Implementation,”
North Carolina State University, SCM Supply Chain Resource Cooperative (SCRC).
Ipek, Ozkaya and Michael Gaglihard (2013), “Archetecting For Large Scale Agile Software
Development: A Risk Driven Approach.” Cross Talk, February (2013), 17-23.
Isaacson, Walter (2011) Steve Jobs: The Exclusive Biography. New York: Simon & Schuster,
567.
47. 47
Kniberg, H. and Matias Skarin (2010), Kanban and Scrum: Making the Most of Both.
C4Media:InfoQ.
Larman, Craig (2003), “Agile and Iterative Development: A Manager's Guide.”
Boston: Addison-Wesley Professional.
Meyer, Geoff (2012), “Scaling Agile @ Dell Real-life Problems - and Solutions.” Austin, Agile
Austin.
Miller, Lynn (2005), “Case Study of Customer Input for a Successful Product,” Proceedings of
agile conference, Alias, Toronto, Canada.
Rizwan Jameel Qureshi, M. (2012), “Agile Software Development Methodology For Medium
and Large Projects.” Software, IET, 6 (4), 358-363.
Reynisdóttir, Þórdís (2013), “Scrum in Mechanical Product Development,” master of science
thesis, Department of Product and Production Development, Chalmers University of
Technology, Gothenburg, Sweden.
Sanjiv Augstine, Bob Payne, Fred Sencindiver, and Susan Woodcock (2005), “Agile
Project Management: Steering From The Edges,” in Communications Of The ACM, Vol.
48, New York, Association For Computing Machinery, 85-89.
Schneider, Joan (2005), “Improving the Stage-Gate® Process”, Frozen Food Age, 53 (10), 38.
Schwaber, Ken and Mike Beedle (2002), Agile Software Development with Scrum. New Jersey:
Prentice Hall.
Sjøberg, Dag .I.K., Anders Johnsen and Jorgen Solberg (2012), "Quantifying the Effect of Using
Kanban versus Scrum: A Case Study," Software, IEEE, 29 (5), 47-53.
Succi, G., Milorad Stefanovic and Witold Pedrycz (2001), “Quantitative Assessment of Extreme
Programming Practices,” Electrical and Computer Engineering, May (2001), 81-86.
Talya Bauer and Berrin Erdogan (2014), Organizational Behavior. Washington DC: Flat
World Knowledge, Inc.
Ulrich, Karl T. and Steven Eppinger (2012), Product Design and Development. New York:
McGraw-Hill.
Vallet, Bennet (2014), “Siemens Success Story,” case study, VersionOne.
Vinodh, S., S.R. Devadasan, B., Vasudeva Reddy and Kusuma Ravichand (2009) “Agility Index
Measurement Using Multi-grade Fuzzy Approach Integrated in a 20 Criteria Agile
Model,”International Journal of Production Research, 48 (23).
48. 48
Vlaanderen, K., S. Brinkkemper, T. Cheng, and S. Jansen (2009), “Case Study Report: Agile
Product,” Technical Report UU-CS-2009-005, Department of Information and Computing
Sciences, Utrecht University, The Netherlands.
49. 49
Appendix A
Taxonomy of Agile Development
Author (Year) Development Method Advantages Drawbacks Advantages for NPD
Ji, Fang and Todd Sedano
(2011)
Royce, Winston W. (1970)
Waterfall
Very traditional and
organized.
Good for large projects.
Inflexible in the face of
changes.
Adaptating to change is
very costly.
Can assist with NPD as
long as minimal
flexibility is required
and high levels of
documentation are
required. Not ideal,
even for very large and
very complex projects.
Ji, Fang and Todd Sedano
(2011)
Rizwan Jameel Qureshi,
M. (2012)
Extreme Programming
Very flexible and easy to
change.
Delivers business values
early and continues to
improve.
Difficult to organize and
execute.
Documentation is weak.
Scaling is expensive.
Feedback loops help
refine and focus the
design of new products.
Heavy user feedback
and continuous
improvement assist with
current and future new
products.
Boehm, Barry W (1988) Spiral
Risk (avoidance)-driven
approach.
Highly organized and data-
driven.
Costly to implement. Not
useful for small projects.
Requires high levels of
research and expertise.
The repition of phases
and emphasis on using
prototypes to refine a
new product helps
developers innovate
and design more
effectively.
Sjoburd, Dag I.K. (2012) Scrum
Requires cross-functional
teams.
Promotes high levels of
communication and
creativity.
Difficult to scale. Time-
boxed sprints are rigid.
Difficulties can arise
when communication is
weak.
High levels of
communication and
cross functional teams
will create a rich stream
of information that will
help developers design
new products more
efficiently. Sprints assist
with keeping
development on track.
Kniberg, H. and Matias
Skarin (2010)
Fitchner, Abbey (2012)
Kanban
Promotes lean
philosophies. Breaks
development into
predictable chunks.
One breakdown can
cause system-wide
problems.
The goal can be lost if the
kanban board is not
updated.
Kanban boards help a
development team
visualize what items are
left on a backlog.
Kanban cards can asssist
with organization
throughout software
development
