Going Agile with Ca Clarity PPM & Agile Vision1. Going Agile with CA Clarity PPM & Agile Vision
Going Agile: CA Technologies, Clarity PPM Division’s Transformative Journey
By Bill Reagan, Innovation Architect for Digital Celerity and former Director of Product Management for CA
Technologies, CA Clarity PPM Division.
Introduction
I recently worked with an innovative organization that went through four phases of transformation to a fully
Agile product development lifecycle. That organization was the Clarity PPM Division of CA Technologies, a
$4.4B IT management software and solutions company. CA Clarity PPM, the leading product used across the
world for IT Project & Portfolio Management, New Product Development and Governance, has a complex
development journey of its own: namely to innovate, build new products and maintain current products on a
regular, timely schedule, and to lead and compete in a rugged landscape.
Having led CA’s Clarity PPM Innovation Roadmap as Director of Product Management for Clarity for nearly 10
years, I was intimately involved in all phases of the move to agility, and directly experienced the challenges,
lessons learned, and benefits of going Agile.
These are the four phases we went through on our journey.
The Setting: Big, Complex Waterfall
The Seed: The Lone Agile Project
Growing: Agile But….
Full Bloom: Scrumming Along Beautifully
The Setting: Big Complex Waterfall
Three years ago, the CA Clarity PPM Division was a traditional, waterfall software development shop. We
didn’t admit we were waterfall because everyone knows you have to be iterative, open to change, and
constantly re-evaluating your projects as conditions evolve. And while we tried to be all those things as much
as possible, in essence, we were developing using a modified waterfall method. What does this mean? We
spent a large amount of time and effort on upfront requirements, validation, design, specifications, prototypes,
and estimations. Once we were sure we had the right set of features - exactly what our customers wanted
most - fully specified and estimated, we published a detailed project schedule and began marching, confident
that we’d succeed in building what we had planned, on-time, and that it would satisfy our customers most
critical needs.
CA Clarity PPM is a complex product. In fact it is a number of products, and growing all the time. The
development team is large and distributed across multiple locations – two separate locations in Northern
California and a large professional team in Hyderabad, India.
Digital Celerity LL All Rights Reserved © Guiding Your Best Path to PPM, ITSM & Governance Best Practices Sponsored by DC Education
2. Going Agile with CA Clarity PPM & Agile Vision
Clarity PPM is a very successful product in an expansive PPM industry. Clarity PPM has over 1,500
customers world-wide, and is constantly adding more, including many Fortune 500/G2000 companies. Clarity
PPM is delivered both as an Installed on-premise application and SaaS on-demand. Competition in the PPM
space is fierce – with large established leaders and newer, fast-moving, up-and-comers. These intensive
market forces place extreme pressures on a development organization to Build it Right, Build it Well, and
Build it Quickly.
Here are the fundamental challenges we experienced using a modified waterfall development method within
this extreme setting:
First and foremost, requirements change. The requirements and features we were so sure about when
we initially planned the development were no longer right by the end of a 12-18 month development
cycle. Conditions change rapidly in the highly competitive and dynamic PPM world. We tried our best
to change with them, but it’s hard to alter course when you’re being swept down a large waterfall,
even a modified one. Too much up-front planning means too much change management downstream.
Fundamentally, we were not executing on our first mandate, Build it Right.
Secondly, by over-committing early to features and schedule, we ended up building incomplete
features when, as is typical, most work takes longer than expected. Rather than building small,
complete features, we built towards a big, long-term goal. When delays occurred, we had to cut pieces
out of features, which weakens them, or move the schedule. Further, sharing large specifications on
large features among distributed teams leads to communication issues. We weren’t doing a great job
of Build It Well.
Lastly, we encountered inefficiencies in our development process. Large features, distributed teams,
long schedules, long feedback loops, large amounts of time spent on re-estimating, re-prioritizing, and
re-planning led to a slow, cumbersome development cycle. We weren’t doing a great job of Build it
Quickly.
The Seed: The Lone Agile Project
Agile development has been growing for some time, often fueled by engineers themselves who are excited by
its promise of leaner, more flexible development. Similarly, at CA Clarity PPM, a project was initiated that was
driven primarily by engineers – a technology project – and it was run using Agile. The engineers were excited.
They created a backlog, formed scrum teams, ran sprints, held daily scrum meetings, and maintained burn-
down charts. A lot of truly great innovation and engineering was accomplished. And the team morale was
great. But there were problems: mainly the team was largely cloaked from the rest of the company.
Executives and Program Managers had no clear idea of where the project stood. What had been
accomplished? Was it on track? What could we see? As the larger demands of the organization changed, it
was hard to make meaningful decisions on how the Agile project fit in, or if and how it should change to align
best with overall strategy.
Digital Celerity LL All Rights Reserved © Guiding Your Best Path to PPM, ITSM & Governance Best Practices Sponsored by DC Education
3. Going Agile with CA Clarity PPM & Agile Vision
For many reasons the project was ultimately shelved, but we had learned a lot about Agile.
An Agile project does not mean little or no documentation is needed.
An Agile project does not mean you can make changes with no impact.
An Agile project must be tied in with the larger program. Status and progress must be visible and
understood by all stakeholders.
Growing: Agile but…
It was clear that Agile development had a lot to offer and we needed to embrace it. Understanding how to run
Agile projects was one thing, but understanding how to fit Agile projects into a stage-gate development process
was quite another. So we did what a lot of companies do, we invented our own version of a modified
agile/waterfall/stage-gate process. Ours was based on the SCRUM method and it picked up various
nicknames along the way – Water Scrum, Scrum Fall, Scrum Gate – you get the idea. So we were using Agile
but…we were doing it our own way. I won’t go into particulars on our method, but I will make a few general
observations.
Making up your own version of a process takes time and effort. And as you proceed through the
development lifecycle, you discover places where your new process doesn’t work, has holes in it, etc.
so you have to spend more time and effort thinking up new versions, variations, and patches to your
current process. Every organization needs to adapt process to their culture and specific needs, but it’s
best to start with best practices and only adapt when necessary.
Tools can help. If you have tools based on best practices, and you use the tools to help you follow best
practices, you can save time and avoid a lot of pot-holes along the way.
Full Bloom: Scrumming Along Beautifully
So finally we took the plunge and decided to fully embrace Agile using the SCRUM method right down the line.
And we also decided to use the right tools to help us follow best practices. We already had the right tool to
help us follow a stage-gate development lifecycle, and manage projects, resources, and costs. We had been
using our own product, Clarity PPM for years to manage every CA project. It works beautifully for automating
workflow, guiding best practices through the organization, and standardizing reporting, prioritization, and
project gate approvals. It’s a great tool for executives, the PMO, and individual project managers. But we
needed a tool for the Agile developers too. Around this time, CA purchased an Agile tool from Salesforce.com.
Salesforce has been an Agile shop for years and had built an internal tool to support the SCRUM method. CA
purchased the tool, productized it, and integrated it with Clarity. It was called Agile Vision. CA also built a tool
for Product Managers (Product Owners in Agile terminology) and called it Product Vision. So, fully armed, we
bravely ventured into the new world.
Digital Celerity LL All Rights Reserved © Guiding Your Best Path to PPM, ITSM & Governance Best Practices Sponsored by DC Education
4. Going Agile with CA Clarity PPM & Agile Vision
Here’s how it worked.
Product Owners fill out a product backlog in Product Vision. The backlog consists of features needed
by customers to compete in the market, or to open up new markets. The backlog items are described
briefly and succinctly. The backlog is prioritized by the Product Owner with input from customers,
sales, services, support, etc.
Product Owners move features into a Release in Product Vision. These are the features that are
planned for the next release of a product.
The PMO creates a new master project in Clarity.
Scrum teams are formed. A scrum team consists of the people who will build out a specific product or
feature area. The team consists of a product owner, scrum master, and engineers who will build, test,
and document the features. The team is co-located as much as possible. This means that as much as
possible, the entire team sits in the same office. This encourages communication, collaboration,
understanding, teamwork, and results.
A new sub-project is created in Clarity for each scrum team. The project is staffed in Clarity and
therefore enables broad-based resource and capacity planning in Clarity. The project is synced to Agile
Vision, and the scrum team is automatically populated from Clarity.
The scrum team does high-level estimation on high-level descriptions of features in their release
backlog. Based on this, a set of deliverables for each sprint is built by the scrum team. This is all
captured in Agile Vision, and the sprint plans are synced back to Clarity so that the project plans show
task plans, estimates, and assignments.
Clarity workflow kicks off acceptance criteria at each stage gate and the project proceeds through the
lifecycle.
The scrum team uses Agile Vision solely. The product manager builds user stories, the team creates
tasks, and the engineers build estimates and specifications, and record their time in Agile Vision. Time,
tasks, and estimates automatically sync back to Clarity for tracking and reporting purposes. Projects
look, and act the same, and use the same approval workflows in Clarity regardless of whether they are
Agile or Traditional.
At the end of each sprint, a retrospective is held. A virtual meeting is called, and the scrum team
demonstrates what they have accomplished so far. As much as possible, complete features (small
though they may be) are demonstrated. All stakeholders attend the meeting.
Digital Celerity LL All Rights Reserved © Guiding Your Best Path to PPM, ITSM & Governance Best Practices Sponsored by DC Education
5. Going Agile with CA Clarity PPM & Agile Vision
We held two meetings. The first was an internal one for other scrum teams, development stakeholders, and
representatives from sales, services, and support. The second was an external one for customers. Yes, we
had customers viewing incomplete code all along the development cycle. This created an ongoing
collaborative environment. The feedback loop was very tight. Comments and changes were incorporated
quickly. Sprint plans could be changed and updated as needed. The Product Owner constantly scrubed the
backlog, and could add new items in and take others out without causing alarm. The features were not fully
specified or estimated prior to inclusion in a sprint, so no onewas overly invested in a given feature. Change
was expected and understood.
That’s a high-level picture of our overall process. In general, I personally found it to be very effective. The
scrum teams were engaged, empowered, and energized. Collaboration and flexibility was enhanced. The
tools largely assisted, rather than detracted, from development because each tool was designed for the person
using it. The PMO used Clarity. Product Management used Product Vision and Agile Vision. Engineers used
only Agile Vision.
The project was successful and Clarity PPM version 13 was the result.
Conclusion
There is no silver bullet in software development. Everyone is excited about Agile; there is lots of hype and
hope surrounding it. But Agile is not perfect and it won’t solve all problems. To build successful products, you
need smart people, good ideas, hard work, teamwork, and maybe a little luck too. But it helps to have guide
posts to help you along the development path. In my experience, Agile provides a good set of guide posts,
and a good method backed by good tools can offer constant reminders to an organization on how development
should be done. The Agile Alliance has published a manifesto that sums it up well.
To improve our ways of developing software, we value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.
Digital Celerity LL All Rights Reserved © Guiding Your Best Path to PPM, ITSM & Governance Best Practices Sponsored by DC Education
6. Going Agile with CA Clarity PPM & Agile Vision
About the Author: Bill Reagan, Product Innovation Architect, Digital Celerity LLC
Bill Reagan is recognized as a thought leader in in Project, Portfolio, and Resource Management systems with
a unique depth of competency and experience, having led CA’s Clarity PPM innovation roadmap as Director of
Clarity PPM Product Management from ’98 to ‘01 and ‘05 to ’11.
Bill has considerable domain expertise in business applications, processes and industry leading enterprise
software, with experience in gated waterfall and Agile SCRUM development processes. As an accomplished
leader in solution definition, design and development, with extensive experience and expertise in engaging with
customers to capture their requirements and recommend best practices, he is uniquely prepared to design,
develop and implement innovative solutions for Digital Celerity’s customers.
Digital Celerity LLC
10 Fernwood Drive, San Francisco, CA 94127
Phone: (408) 812-9999 | Fax (408) 516-8069
Contact Us: Sales@DigitalCelerity.com
Additional best practices content may be accessed at:
www.digitalcelerity.com
Digital Celerity LL All Rights Reserved © Guiding Your Best Path to PPM, ITSM & Governance Best Practices Sponsored by DC Education
7. Going Agile with CA Clarity PPM & Agile Vision
Sharing Knowledge, Exchanging Ideas, Building Community
Digital Celerity LL All Rights Reserved © Guiding Your Best Path to PPM, ITSM & Governance Best Practices Sponsored by DC Education