Web & Social Media Analytics Previous Year Question Paper.pdf
Pragmatic Marketer Volume 8 Issue 3
1. Volume 8 • Issue 3 • 2010
The journal for technology product management and marketing professionals
®
productcamp
ProductCamp
Best Practices
Auditing Product Management
in Early-Stage Companies
Is Product Management Needed
In a Software Startup?
How Many Data Points
Do You Really Need?
3. Inside this issue: Volume 8 Issue 3 • 2010
5 ProductCamp Best Practices
By John Milburn and Paul Young
Creator of the world’s most popular ProductCamps are taking the world by storm and
product management and marketing seminars have sprouted up in Silicon Valley, Austin, Toronto,
Amsterdam, Sydney and more. If you haven’t
participated in one, you might wonder what all the fuss
About Pragmatic Marketing® is about. If you have participated, you already know!
Since 1993, Pragmatic Marketing has
conducted product management and
marketing training for 5,000 companies
in 23 countries. Our team of industry
thought-leaders produce blogs, webinars,
podcasts, and publications read by more
than 100,000 every year. 12 Auditing Product Management
in Early-Stage Companies
By L. Kurt Reiss
How do you deal with the “mess” of moving
The Pragmatic Marketer ™ targets and reference points? The tool that
8910 E. Raintree Drive has not been broadly used but can be of
Scottsdale, AZ 85260 great help is the product management audit.
Pragmatic Marketing, Inc. What is a product management audit?
Craig Stull / Founder and CEO
Kristyn Benmoussa / Editor-in-Chief 20 Is Product Management Needed
Graham Joyce / Managing Editor In a Software Startup?
————————————————— By Gopal Shenoy
Interested in contributing an article? What are the benefits that product
PragmaticMarketing.com/submit management can bring to the table during
the early days of a fledging business? Let’s
No part of this publication may be reproduced, stored in look at the typical genesis of a startup
any retrieval system, or transmitted, in any form or by any company and what transpires between
means, electronic, mechanical photocopying, recording
or otherwise, without the prior written permission of the start and first release of the product.
publisher.
Other product and/or company names mentioned in this
journal may be trademarks or registered trademarks of
their respective companies and are the sole property
24 How Many Data Points
of their respective owners. The Pragmatic Marketer, Do You Really Need?
a Pragmatic Marketing publication, shall not be liable
regardless of the cause, for any errors, inaccuracies, By Ken Allred
omissions, or other defects in, or untimeliness or
unauthenticity of, the information contained within this
Because statistics teach us about minimum
magazine. Pragmatic Marketing makes no representations, confidence levels, confidence intervals, and
warranties, or guarantees as to the results obtained from margins of error, people often struggle with
the use of this information and shall not be liable for any
third-party claims or losses of any kind, including lost the idea they can start leveraging the data
profits, and punitive damages. from their win/loss program from the very
first data point.
The Pragmatic Marketer is a trademark of Pragmatic
Marketing, Inc. Printed in the U.S.A. All rights reserved.
ISSN 1938-9752 (Print)
ISSN 1938-9760 (Online)
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 3
4. Product Launch Essentials ™
Plan and execute a successful product launch
Are your product launch efforts focused on deliverables rather than results?
Launching a product is more than following a simple checklist. A successful product launch is the
culmination of many, carefully planned steps by a focused, coordinated team. Even good products can
fail because of organizational issues, misunderstanding of roles and responsibilities, and a lack of a
strategic approach to guide efforts.
• Learn a repeatable product launch process to shorten the launch planning cycle, get the resources
needed, and know what to expect at every step.
• Understand the seven product launch strategies your team can use
to maximize sales velocity.
• Measure product launch progress with indicators that identify unforeseen
issues before they become big problems.
Get a free e-book at Daniel
s
PragmaticMarketing.com/launch
id
By Dav
Download a complete agenda and register at PragmaticMarketing.com/seminars
Call (800) 816-7861 to conduct this seminar at your office
5. ProductCamp
How did ProductCamps originate?
ProductCamps are based on the concept of
Best Practices
BarCamps, the first of which was held in Palo
Alto, California in 2005. It’s a reference to the
hacker slang term, foobar. BarCamp arose
as a tongue-in-cheek spin-off of Foo Camp,
By John Milburn and Paul Young an annual invitation-only participant-driven
conference hosted by open source publishing
luminary Tim O’Reilly. Where FooCamp was by
invitation only, BarCamp would be democratic
ProductCamps are taking the world and open to all. BarCamps have been held
in over 350 cities around the world, and even
by storm and have sprouted up virtually over the Internet.
in Silicon Valley, Austin, Toronto, The first ProductCamp (aka P-CAMP) was
held on March 15, 2008 in Mountain View,
California. Word-of-mouth spread fast about this
Amsterdam, Sydney and more. If you “unconference” (no registration fee, no agenda,
no selling) and about 170 people attended on a
haven’t participated in one, you might Saturday to discuss topics of interest to product
managers, product marketers and a host of
wonder what all the fuss is about. If you related roles.
have participated, you already know! Following this event, many participants
commented via blogs, wikis, and tweets.
The idea went viral, and other cities started
ProductCamps of their own. Austin was first
out-of-the-gate with others following over
subsequent months in 2008, 2009 and into 2010.
Given the proliferation of ProductCamps, we
thought it would be useful to summarize some
best practices and hopefully these guidelines
will help you organize or participate in an
unconference in your local area.
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 5
6. ProductCamp Best Practices
Why attend ProductCamp?
Has your travel budget been cut? Do you want to
learn from peers outside your company? Are you
looking for informal ways to “meet-and-greet”
others? Do you enjoy presenting or leading
roundtable discussions on timely industry topics?
Are you unemployed or under-employed, and
want to increase your network? Do you want to
meet others who are passionate about product
management and marketing? If the answer is
“yes,” to any of these, you should participate
in ProductCamp!
Why organize a ProductCamp?
Have you ever attended an industry conference
or company event and felt like you could
have organized it better? Do you feel industry
conferences have become too “sterile” or
dominated by big vendors? Are there a lot
of knowledgeable product management and
marketing professionals in your area, but no
good forum to get together? Would you like
recognition in your community as someone that Many ProductCamps use an
“gets things done”? Dozens of your peers around
the world are planning ProductCamps for these “Open Grid” to set the agenda. Presenters
same reasons.
and roundtable facilitators submit topics
Since ProductCamp is free to attend and typically prior to the event, or sometimes even on
held on a Saturday, it’s a low-risk way to expand
your skills and personal network. Plus, since the morning before the event starts.
it’s “user-sponsored,” it will not be a thinly A final agenda is created real-time,
veiled sales event—participants are strongly
discouraged from selling or promoting products. by and for attendees.
6 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
7. ProductCamp Best Practices
ProductCamp guidelines A typical day begins with attendee registration and
breakfast. Then participants are led to a “main-tent”
Several common themes have emerged from session that explains the format, rules, and introduction
ProductCamps held to date, though the flow and of ProductCamp planners—but, more importantly, the
tenor may change from city to city. Since it’s an opening session sets the tone that ProductCamp is by
unconference, there are very few rules. The general and for attendees, open and discussion-oriented, and
guidelines (adapted from BarCamp) are: most importantly, fun.
• ProductCamps are free, but there is a “cost” to In today’s economy, several ProductCamps have offered a
attend. There are no passive attendees. Instead, few minutes for anyone who has job openings to present
everyone is referred to as a “participant” and expected a very short overview of the position/company.
to contribute in some way: presenting a session,
coordinating sponsors, managing the venue, ProductCamps use an “Open Grid” to set the agenda.
volunteering for setup and teardown, or sitting Presenters and roundtable facilitators submit topics prior
in a discussion panel—there are many ways to the event, or sometimes even on the morning before
to participate. the event starts. A final agenda is created real-time,
by and for attendees. Session leaders are sometimes
• Information sharing is expected and encouraged. given 30 seconds in the welcome session to introduce
Everyone is urged to share event information and themselves and their topic. Then it’s time to vote! One
experiences, both live and after the fact, via blogging, popular technique is to give each participant three sticky
photo sharing, social bookmarking, tweeting, and notes as they arrive. Volunteers post session topics in a
wiki-ing. This open encouragement is in deliberate central area and ask everyone to place the sticky notes
contrast to the “off-the-record” and “no recording” under their top three choices. Sessions receiving the
most votes are plotted on an “agenda sheet” in such
rules of many conferences.
a way to minimize topic conflicts. Based on interest
• The “2-foot” rule applies: ProductCamp typically level and available time slots, some topics may not
features concurrent sessions. If you feel you’ll get make the agenda. It’s open, it’s participant-driven,
and it really works!
more out of a different session, use your own “two
feet” to move—no one should feel tied down.
Sessions usually run 45-50 minutes with a break for
• Networking is good! ProductCamps are held on lunch and plenty of “slack time” for informal networking.
a weekend and go all day, so participation is a Most ProductCamps reserve a few meeting rooms for
commitment. The sacrifice in time and energy means parallel sessions and list four to six time slots. The
only those who are really interested attend. best ProductCamp sessions are facilitated roundtable
discussions—leveraging participant knowledge.
Venues typically provide basic services. Free internet Notes, videos and photos are posted or linked
access and Wi-Fi are crucial. Sponsors provide funding to the ProductCamp website.
for “nice-to-haves” such as t-shirts, meals/snacks, signage,
lanyards, etc. Attendance is free and usually restricted It’s always fun to bring everyone together at the end
only by venue capacity. Due to the popularity of these of the day for a quick summary and to award prizes for
events (and to assist in planning for meals, t-shirts, etc.) best sessions. By this time, the group is comfortable and
advance registration is often required. can get rather lively celebrating the hard work and start
planning the next ProductCamp.
ProductCamps have been held at university-
donated facilities, professional learning
centers, and corporate offices.
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 7
8. ProductCamp Best Practices
Tips for running ProductCamp Marketing • Have name tags printed for those
who pre-registered, blanks for
Don’t over-plan! Think of it as a • Create a website and Twitter walk-in registrations (remember to
fun way to get together and meet hashtag. Optional: create a leave space for your participant’s
others to learn and network—not a Facebook page and LinkedIn Twitter ID!). If you can, add
typical conference where events are Group. If you have a local Product
pre-planned. Structure is good, but session break and lunch times
Management Association, ask to the back of the name tag.
too much defeats the purpose.
them to post the event on their
site and in newsletters • Parking directions/assistance
Make registration very easy. People (if needed)
won’t register if they have to jump • Provide regular e-mail updates
through hoops—email address is to registrants • Cleanup—after meals and at the
the only personal information really end of day
needed at this stage. • Get mentions in relevant
newsletters, blogs and websites. • Hall monitors—between sessions
If you have trouble getting sponsors, Contact local media (newspapers, to answer questions and help
sacrifice the t-shirts and breakfast. business and tech journals, shepherd people where they
TV, radio) need to go
Assign a “go-to” person who is very • Session monitors—make sure there
organized and able to delegate— Volunteers is a timekeeper and note-taker in
not to do all the work. The time each session (doesn’t have to be
commitment varies (more time is • Initially just collect names but a
one of the organizers, have the
needed closer to the event). critical role on ProductCamp day
presenter/moderator ask for
• Set up registration table for sign-in, volunteers)
There are five key areas that need
t-shirt, sponsor give-aways, any
leadership, coordination, and • Encourage people to take pictures
accountability to ensure a successful information sheets, etc.
(a camera phone is often good
ProductCamp: enough) and post to Twitter or the
event website or a Flickr page.
Use the Twitter hashtag as a
“keyword.” A dedicated
photographer is a plus.
Don’t over-plan! Think of it as a fun way
to get together and meet others to learn
and network—not a typical conference
where events are pre-planned.
8 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
9. ProductCamp Best Practices
Sessions Venue If you need help organizing
• Encourage people to sign up to • Coordinate building open/close a ProductCamp, please let
present or host a roundtable times, security and parking Pragmatic Marketing know. We
• Ensure everyone understands • Visit the facility the day/evening have been involved with every
the “Open Grid” scheduling before the event to ensure ProductCamp and can provide
everything is ready sponsorship, promotion and get
• Collect and post presentations
and notes from the sessions to • Set up rooms including projectors, involved with planning teams
the event website or a site like chairs, whiteboards, etc.
as a resource for questions
SlideShare
• Manage details about the food— and guidance.
• Coordinate the “best presentation” set up, trash cans, location,
voting and awards cleanup
• Conduct a post-ProductCamp
survey
Budget John Milburn has “walked the walk” in technology product
management and is an instructor at Pragmatic Marketing.
• Get financial commitment from
Throughout his 20+ year career, he managed or delivered
sponsors over 40 hardware and software products and implemented
• Manage budget and pay bills the Pragmatic Marketing Framework at countless companies.
(ideally, set up a 501(c)(3) and
open a checking account) John is on the planning team for the Austin ProductCamp
and has helped leaders of many ProductCamps. Contact John at
• Order and pay for food, name jmilburn@pragmaticmarketing.com
badges, signs, t-shirts, lanyards
and trophies
Paul Young is a product management professional with
more than a decade of experience in hardware, software,
and services product management and marketing. He
is an instructor at Pragmatic Marketing. He has also
held a variety of product management and marketing
roles in technology startups and Fortune 50 companies
such as Dell, NetStreams and Cisco Systems.
Paul is founder of ProductCamp Austin and has helped leaders at other
ProductCamps. Read his blog at productbeautiful.com. Contact Paul at
pyoung@pragmaticmarketing.com
productcamp
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 9
11. Seminars
Living in an Agile World™ Requirements That Work™
Strategies for product management Methods for creating straightforward product
when Development goes agile. plans that product managers can write and
developers embrace.
Practical Product Management®
Principles of the Pragmatic Marketing Framework, Effective Product Marketing™
the industry standard for managing and marketing Repeatable, go-to-market process to design,
technology products. execute, and measure high-impact marketing
programs.
Pragmatic Roadmapping™
Techniques to plan, consolidate and communicate New Rules of Marketing™
product strategy to multiple audiences. Reach buyers directly, with information they
want to read and search engines reward
with high rankings.
Product Launch Essentials™
Assess organizational readiness and define team
Executive Briefings responsibilities for a successful product launch.
Designed specifically for senior management,
Executive Briefings discuss how to organize
Product Management and Marketing
departments for optimal effectiveness
and accountability.
In addition to the extensive published schedule, training can be conducted
onsite at your office, saving travel time and costs for attendees, and allowing
a much more focused discussion on internal, critical issues.
Pragmatic Marketing’s seminars have been attended by more than
60,000 product management and marketing professionals.
12. Auditing Product Management in
Early-Stage Companies By L. Kurt Reiss
Many product managers like to work in startup
companies. I’ve been one of them. Why? Without
doubt, a product manager in a startup has
an opportunity to make broad-ranging and
meaningful contributions to product success,
fast. But, life in that lane is not easy.
Early product definition tends to be quite
elastic, leaving room for creativity. Corporate
structure in a startup tends to be flat, allowing
exposure to serious and far-reaching business
and technology problems that impact decision
making. That’s all good. However, one of the
major difficulties is confusion. It’s all in flux.
Yes, broad-stroke formulations of product vision
and business concept do exist, the ones used to
raise funds for the startup. But almost always,
even these fundamental statements change
with time. Flexibility is good but it’s hard to
succeed without a firm grasp of reality and clear
articulation of objectives, decisions, success
factors and progress metrics.
How do you deal with this “mess” of moving
targets and reference points? The tool that has
not been broadly used but can be of great help
is the product management audit. What is a
product management audit? Let’s take a closer
look, but first, I’ll highlight key issues related to
product management in a startup. Then, we’ll
define the specific problem we propose to
address here and explore an example of an audit
meant as a remedy for this problem.
12 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
13. The problem we aim to bring to market, and when. And, it’s a matter
of mutual understanding among all the managers in the
The three issues that hold keys to product success and company what these requirements are. Are we building
frequently, create problems in a startup are strategy, what we think we’re building?
market and customer needs, and product requirements.
So are these three issues the problem we address here?
Strategy is extremely important for any endeavor, and Not quite. Entrepreneurs usually know these issues
especially for a startup, both the corporate strategy are important and create problems if not addressed.
and the product strategy. The issue here is two-fold: However, in a startup, people are very busy doing a lot
Is our corporate strategy really sound and clear—as of different things. Moreover, entrepreneurs want to be
articulated—and is its impact on product strategy optimistic, driving to success by focusing on the possible
well understood? Do all key people in the company rather than on the difficult. This combination doesn’t
understand and agree on the current articulation of leave much room for rigorous self-assessment.
product strategy?
The problem often is these three key issues are glossed
Target Market and Customer Needs define, almost over. We tend to think all three are in good shape,
completely, product requirements. Thus any lack and justify these thoughts by superficial considerations
of clarity here will soon lead to difficulties with and wishful thinking. We also want to believe what we
the product. Have we fully decided who our target preach to the point of tweaking the facts to fit our needs.
customers are and truly verified the scope of Although drinking your own Kool-Aid may be enjoyable,
overlap between what they need and what we it usually leads to an upset stomach.
plan to deliver? It’s amazing how often the answer
is: “Sort of, but not really.”
The solution is a product management audit—a practical
way to counter neglect and false optimism and focus on
Product Requirements are key for a startup, thoughtful decisions and a justified positive outlook.
driving both development work and early product
promotion. No it’s not a matter of a polished
text document. It’s a matter of having thought
through and decided on the kind of product
Key Product Management
Strategy
Issues in a Startup
Market Requirements
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 13
14. Auditing Product Management in Early-Stage Companies
Facts
Key Factors in the Audit
Con
fid bi lity
enc
e p o nsi
Res
The solution: a product management audit
What is a product management audit? First, it’s a
review. An analysis of the state of everything related
to product management. The goal is to step out of the
daily grind and take an unbiased look at issues, and
grade the progress achieved towards solving each of
them. It has some similarity to a financial audit. It’s a Can an audit yield easy-to-understand-and-track metrics?
step by step, “by the book,” analysis of everything that Yes it can! And just two metrics will do. An “average”
impacts product management. It’s a short, intense, and score (call it the A-score) and the “fix-it-now” score (the
repeated effort. F-score). The A-score is a geometric average score of all
factors under consideration. And the F-score is the lowest
What’s the typical framework of a product management score of all individual factors.
audit? For each issue related to product management,
one should take a critical look at: Why is geometric average the right one to use? Think of
a car’s tire pressure. If three wheels are at 100% but one
• Facts: What’s known / not-known? is at 0.01%, the arithmetic average (75%) says “it’s still OK
• Confidence: How confident are we to handle problems to drive” but the geometric average (10%) says “fix the
tires now!” Clearly, you need all wheels in good working
related to the issue?
order to succeed.
• Responsibility: Who’s responsible for resolving
each of these problems? The good thing about having numerical scores is they
help track progress. They also help identify the bad
That’s the minimum, but a fundamental one. Of course, outlier factors that need fixing immediately.
there are ways to augment the review by considering
objectives, teams, resources, timelines, plans, etc. I would .
argue against taking it on. Always make sure the basics
are sound before diving into the sea of details. Address
details as a separate exercise.
14 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
15. A-Score
Completed
Reviewed
Auditing Product Management in Early-Stage Companies
Aligned
0 20% 40% 60% 80% 100%
Good Bad
Example scoring of Strategy Total score for Strategy in this example: A=47%, F=10%.
Facts: Consider three strategy-related factors that can
be scored with the good-bad percentage metric. For Total Strategy Scores
example, our strategy statement is 80% completed and
articulated (80% good, 20% bad). And, 75% of our F-Score
external strategy reviewers approve of the risks we’ll be
exposed to and bets we have made. And, 20% of our A-Score
managers understand and accept our strategy. That yields
A=49% and F=20%. That’s very bad. 0 20% 40% 60% 80% 100%
Good Bad
Strategy Facts Scores
These are dismal results, butStrategy Scoresthing is: they
Total the good
F-Score
clearly point to areas where improvement is needed.
A-Score
Completed Example scoring of Target Market
Reviewed Facts: We know user needs of 80% of the target market
segment. We have articulated competitive win scenarios
Aligned for 20% of anticipated competitive bid types. 100% of
our identified target adopters are committed to do beta
0 20% 40% 60% 80% 100%
testing of our product. A=54%, F=20%.
Good Bad
Confidence: We are 90% confident that our value
proposition is compelling to our target customers. We are
80% confident customer needs will not change before we
Confidence: We’re 80% sure our strategy will not need ship the product. We are absolutely (100%) sure of our
to be modified before we ship the product. And, we’re target market size. A=90%, F=80%. Excellent result—and
50% sure our business can survive bad outcomes of our therefore solid scrutiny is needed to verify this score.
bets. And, we’re 100% sure we will not run into any
technology showstoppers. A=74%, F=50%.
Responsibility: We have individual owners for 50% of
win scenarios that are still missing. A=F=50%
That’s not too bad but not good enough.
F-Score
Total score for Market: A=67%, F=20%. Better than
Responsibility: For 10% of our still-open strategy issues
A-Score Strategy, but still far, far below target.
we have an individual identified who’s responsible for
closing them. A=F=10%. 40%
0 20% 60%
Really bad! 80% 100%
Good Bad
Total Strategy Scores
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 15
16. Auditing Product Management in Early-Stage Companies
Example scoring of Product Requirements Audit template
Facts: The Product Requirements Document (PRD) Whether you decide to do it or arrange for an
is 100% completed and reviewed. 90% of individual independent audit, it’s good to have a solid
requirements have defined priorities associated with understanding of how it works. There are three key
them. The review found 80% of these requirements aspects of an audit: First and foremost, it’s the scope:
are clear to reviewers. A=90%, F=80%. Good. the list of questions or factors to examine and review.
Second, it’s the mechanics of organizing results and
Confidence: We are 60% confident that requirements computing metrics. And finally, but perhaps most
define all factors critical for winning against competition importantly, it’s the veracity and validity of the audit.
(Does this agree with the estimate for competitive A template shown on the next page may be of assistance
wins in the Market section? What if it doesn’t?). We for the first two. As for the third, it’s a matter of trust
are 90% confident that priority-1 requirements must be in the person(s) doing the audit.
satisfied before the product ships (Which ones are the
questionable 10%?). A=73%, F=60%. Still needs fixing. Questions can be many and varied, but not all of them
are the “right” questions for a product management audit.
Responsibility: We have one owner for 100% of the What are the right questions? They are largely defined
requirements that need better prioritization and/or by the intersection of the two sets of topics discussed
clarification. For 20% of items that need clarification, earlier: “strategy-market-requirements” and “facts-
we have identified reviewers that will decide if they confidence-responsibility.” The ones in the preceding
have been sufficiently clarified. A=45%, F=20%. Bad. section could be a good starting point. How many
questions to ask? As few as possible that are the most
meaningful in your situation and answerable with a
Total score for Requirements: A=69%, F=20%. Well, this numerical (percentage) metric.
is embarrassing.
Some questions can be answered with a precise
And finally, aggregating all scores, the total Audit (countable) number. For example: “What fraction of
score for Product Management in this example is: individual requirements in the PRD have priority
A=60%, F=10%. assigned to them?” Others can’t. For example: “What is
our level of confidence in factors that will assure our
Total Audit Scores wins in competitive bids?” Still, a rough percentage
estimate can be defined and be useful. Just make sure
the audit captures and considers not only the estimate
but also its source.
F-Score
Target
A-Score Going beyond the audit questions used in the example
above, other fact-related questions may include:
0 20% 40% 60% 80% 100%
• What fraction of our value proposition is uniquely
Good Bad
differentiated?
• What fraction of our efforts brings a distinctly new
I wouldn’t invest in this example startup until these value to the market?
numbers are much, much higher. Can they get better?
Total Audit Scores
Of course they can. Once you know—as illustrated by • What fraction of our target market could materially
the audit results—what needs fixing, it’s easier to focus benefit from the full set of features of our product?
your energy and improve the situation.
• What fraction of the cost of the solution implementation
by our customer will be due to the purchase of
our product?
16 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
17. Auditing Product Management in Early-Stage Companies
• What fraction of our product requirements directly map find it difficult to deal emotionally with someone
to our value proposition? else “checking” the validity of their own vision
and perception. Weak entrepreneurs may even fear
• What fraction of our product requirements represents less-than-stellar results of an audit may negatively
new and differentiated value? impact investors’ view of the company. It is possible
that some companies would prefer not to do such
All of these questions are the “right” questions for the audits. But, the value of an audit is not in finding fault,
audit. That’s because for each of these questions, a high it is in confirming that everything is fine and if not, in
percentage-score answer represents a “good” answer. identifying areas that need attention. The audit results
The higher the score, the better. present an independent opinion, a tool facilitating good
decision making for product management purposes.
Mechanics of the audit can be supported by a
spreadsheet, like the one shown below. The auditor You can do an internal audit but it’s best to rely on
can complete the “Questions” fields and associated “%” someone external. Just think about a due diligence
estimates, as the audit proceeds. process a startup goes through prior to a round of
funding. Would an investor ask the company itself to
The example above and the template need a fair amount perform such due diligence? Would you rely on the
of specificity. The main thing about the Audit is doing analysis for fund-raising purposes to actually drive your
it, applying serious qualitative analysis and using the business? The #1 objective of that analysis was to raise
score as a tool, first for critical thinking, then for tracking funds and that calls for a level of optimism that may be
progress—until the next audit. dangerous in day-to-day business operations.
Audit Template
Who needs (and wants) Questions Scores (%)
the Audit?
Facts
Certainly top executives in the
Strategy
startup should want it, as should
STR
Confidence
the board and investors. If there
is one, so should the product Responsibility
manager. Best startups may not
need product management audits—
Facts
all’s working well there. But the vast
majority of startups would certainly
Market
TOTAL
MAR
benefit. What if a company does Confidence
not yet have a dedicated product
manager? Often, the founder drives Responsibility
early product definition and its
creation. However, the founder Facts
Requirements
does not usually have the time
and inclination which often leads REQ
Confidence
to unjustified optimism about the
product in the market and vague
product requirements. An audit Responsibility
can really help.
Results
POWER(PRODUCT(STR),1/COUNTA(STR)) = A-score for Strategy =
Can it possibly happen that top
MIN(STR) = F-score for Strategy =
executives of a company would
POWER(PRODUCT(MAR),1/COUNTA(MAR)) = A-score for Market =
not want a product management
MIN(MAR) = F-score for Market =
audit? Entrepreneurs are typically
POWER(PRODUCT(REQ),1/COUNTA(REQ)) = A-score for Requirements =
strong willed and may simply
MIN(REQ) = F-score for Requirements =
POWER(PRODUCT(TOTAL),1/COUNTA(TOTAL)) = Total A-score =
MIN(TOTAL) = Total F-score =
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 17
19. PragmaticMarketing.com
Stay current with industry best practices
Review 10 years of
Annual Product Management
and Marketing Survey results
Watch a webinar by one
of today’s industry experts
Read articles on product management,
marketing and leadership strategies
Subscribe to blogs by
Pragmatic Marketing thought-leaders
Participate in online
networking through LinkedIn,
Facebook and Twitter
Purchase books, clothing
and merchandise
20. Is Product Management
Needed In a
Software Startup?
By Gopal Shenoy
Startup—the early days
A startup is launched when an entrepreneur—a software engineer,
marketer, product manager or salesperson—comes up with an idea
based on past employment or personal experience. The idea could
have been triggered by observations made during sales calls, opinions
expressed by current customers, high volume of prospect inquiries or
customer support calls, personal experience using a technology,
etc. The entrepreneur gets excited about the potential of the idea
and decides to create a product to solve this problem. The typical
questions that consume the founder’s mind include “Who can help
me create the product that will bring this idea to life?”,
“How can I get this done as soon as possible
before someone else does?”, “How can I sell
this and make millions?”
There is a very high sense of urgency at
Is product management the beginning to do something. However,
needed during early stages urgency during the early days is a
double-edged sword. While needing to
of a software startup? be the first to market has its advantages,
rushing to market without doing enough
What benefits can product homework is a sure fire recipe for
management bring to the failure. There is a lot written about why
startups fail. All of them point to the
table during the early days simple fact that most startups build
something nobody wanted. The
of a fledging business? entrepreneur either gets enamored
with a new technology (especially
if the entrepreneur is a software
Before we delve into the engineer or has a technical
need for software product orientation) or a radical
new product (especially
management in a startup, if coming from sales),
let’s look at the typical but then end up never
finding enough customers
genesis of a startup company to buy the product or
service created.
and what transpires between
start and first release
of the product.
20 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
21. Fundamental questions a startup Product pivoting and What skills will this person need?
(or established companies venturing the first release
into new markets) should ask before • Ability to dig into what is stated
building a product or service are: If the early discovery process
by the customer and identify
indicates a potential market, the
startup needs to quickly build a underlying problems.
• What are the problems that exist
minimal product and get it in the • A business orientation to maintain
in the target market?
hands of a few customers who have the big picture of what the
• How many potential customers the problem. Once the customers
company is trying to do without
have this problem? start using the product, someone at
the startup needs to monitor usage, getting caught up in details of the
• How are potential customers be in constant communication product. The business model,
solving the problem today and with these first customers to get target market, and pricing are all
how dissatisfied or frustrated are their feedback and work with the undetermined at this time.
they with the current solution? engineering team to make changes
(pivot) to improve the product. All • Great communication skills to
• Is the frustration large enough that of this should happen at a very fast understand what the customer is
these customers will be willing to pace. Process is not what matters, stating and converting it into a
pay to solve this problem? flexibility is. language engineers understand.
• Is what the customers are willing • Project management skills so to
The sole objective during this prioritize and ensure things don’t
to pay large enough to build a period is to pivot enough times
profitable business? fall through the cracks.
to quickly identify the business
model and final product for the It is extremely important this person
• What should the solution target market. This speed requires be a member of the founding team.
look like? someone focused on this task and You can call this person a product
You will notice that all questions who serves as coordinator between manager but that’s less important
but the last one, are about the customers and engineering. This is than their primary responsibility
market. None of them are about a critical phase and any misstep can to focus on the market and ensure
what technology platform should be catastrophic, resulting in failure the product creates a sustainable
be chosen, what feature set should of the entire business. business. Product-oriented founders
be in Release 1, or how many are typically well suited for this role.
engineers the startup needs. They
are completely independent of
the solution but instead focus on
discovery of market problems and
determination of market size.
While those are great questions to
ask, it is typically not possible to
get complete or accurate answers to
these questions quickly, resulting in
a dilemma—if it is not possible to
find correct and accurate answers to
all of these questions before getting
started, and if the chances of failure
are very high if you do not get the
answers you need, what gives? This
is where product pivoting comes
into play.
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 21
22. Is Product Management Needed In a Software Startup?
If the founding team does not have Even if such a person exists in the • Developers have so much to do
someone with a product orientation, company, here are some reasons that piling on market research is
then it becomes critical that such that make it hard to play dual roles: not a good idea.
a person is brought on board.
This is a tough sell, especially to • Salespeople are inclined to
• Delving deep into customer
those funding the startup with no overcome objections in order
problems is an art not a science.
product. It is much easier to justify to meet their numbers. They
hiring developers or salespeople. Customers often propose solutions
are not trained to understand the
But to create a sustainable business, and it takes skill and effort to
underlying problem, relay it back
it will take product management unearth the real underlying
to the product team and get it
discipline to ensure early market problem.
fixed. Sales is all about the
success continues to create
a growing business. • It is easy to confuse “what the last “here and now.”
two users told me” with “this is
• Early customers need someone
what the entire market wants.”
Some may say the call for a product they can bounce ideas off and
management orientation in a startup • Maintaining an objective opinion someone they can engage with to
is based on false assumptions that when the customer says your build trusted relationships. It is
Development or Sales or Marketing “baby is ugly” (first version of these early customers that turn
are not capable of doing market into references and proponents
your product) is difficult,
research. After all, aren’t we
especially when you have spent when the product starts its
required to wear multiple hats
in a fledgling startup? sleepless nights building it. It is journey into the mainstream.
often difficult to listen to and
understand the customer’s Past the early success
point of view.
Once the company establishes
the viability of the product, creates
the business model and has its
first group of paying customers,
it’s time to assign someone whose
primary responsibility is to take
the early product and transform it
into one designed to appeal to a
wider market. This requires full time
effort and focus. Startups at this
stage of growth must make hiring a
seasoned product manager a priority
and long-term investment.
22 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
23. Is Product Management Needed In a Software Startup?
“Startup” product manager • Someone who has previously built necessary because often what you
qualifications teams and can grow your product discover in the market requires
management team once the you to change internal opinions
Based on my experience working in company hits the next milestone. and points of view. This requires
startups, I recommend the following Someone you can trust to manage the ability to question currently
criteria when hiring your first what justifies the company’s very held beliefs and opinions
product manager:
existence—your product and recalibrate based on
or service. market realities.
• Someone who has previously
worked for a successful startup • Someone who can balance a How important is domain
and gone through this critical strategic and tactical role. It is experience? It depends on your
“next” phase. equally important the product market segment. In highly technical
manager lead discussions on fields such as computer networking,
• Someone who has previously product/business strategy and deal virtualization, cloud computing,
worked at least one failed startup CAD, space exploration, network
with day-to-day tactics to get the
and who is able to articulate the security, pharmaceuticals, medical
product out the door. This person services, financial services, etc.,
lessons he/she learned from that should have a keen sense for domain experience will be of
experience. What mistakes were product usability, performance and paramount importance because you
made and why? I personally value functionality because eventually do not have time to teach someone
a person who has failed at least these have major influence on new to the industry the technology,
once over someone who has only product success. jargon, regulations, etc. But in other
had past successes. general areas such as ecommerce,
• Someone you can put in front hire product managers who have
• Someone who can strike the right of customers. Your “product a breadth of experience.
balance between process and manager” should be one who
flexibility. Someone who can enjoys and thrives on customer It is essential that startups have
function without regimented engagement. One who is a good a strong product management
processes, but one who can also listener, can engage in active orientation right from the start.
rein in the chaos so common in dialogue with prospects/customers It is not necessary that the person
startups. As the company grows, and your customers feel have the official title of a product
add process into the mix to comfortable with. manager, but it is of utmost
scale efficiently. importance that someone from the
• Someone your organization founding team has the primary
• Someone who shows immense will respect. Given the product responsibility of doing product
passion and drive. Someone who manager will be ultimately management. Discovering unsolved
can work with very limited responsible for bringing the problems in a target market,
information and takes the voice of the customer into the building a product this target market
initiative. You absolutely need organization, this person needs is willing to buy and figuring
someone who has the energy, will out a business model that helps
to win the respect of internal
and drive to convert early success the company create a sustainable
stakeholders. You also want business is not a part-time effort.
into sustainable growth. someone who can stand up to
the internal stakeholders when
Gopal Shenoy is the Director of Product Management at Gazelle.com where he leads the product
management effort to allow consumers to sell or recycle used electronics. He is a seasoned product
management leader with over 14 years of experience in startups and small businesses. He has led
direct report and cross-functional teams that defined and launched highly successful SaaS-based
and enterprise software products. His passion for product management led him to create a Software
Product Manager blog at productmanagementtips.com. He is also a speaker at ProductCamps and
other software product management conferences. Contact Gopal at gopal.shenoy@gmail.com
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 23
24. How
One of the common questions I am act more like a laser pointer that
asked when discussing win/loss illuminates very specific information
analysis is, “How many deals do I about very specific issues and
Many
need to analyze before I can start questions. In other words, to get the
using the data?” In the past, my most out of your win/loss program,
response has always been that you it should not be seen as a survey tool
will get value from the very first win, (which simply gathers information),
Data
loss, or no decision, you analyze. but rather a decision-making tool
However, because statistics teaches and an early warning tool.
us about minimum confidence levels,
Points
confidence intervals, and margins of If you think you can’t get value from
error, people often struggle with the your win/loss program, or believe
idea that they can start leveraging you can’t begin making decisions
the data from their win/loss program
Do You
based on win/loss data until you
from the very first data point. achieve a statistically significant
number of data points, our research
shows you risk missing as much as
Get away from
Really
75 percent of the potential ROI you
the “survey” mindset could achieve with your win/loss
analysis program. The key is to
By just listening to the terminology understand how and when to use the
Need
executives use, it is obvious intelligence gathered through your
many initially base their vision win/loss analysis.
of a win/loss program on their
prior knowledge of market
By Ken Allred research or political polls. For
instance, the term “survey” is
The decisions executives
often used to describe what is and managers make
actually an in-depth “interview.” To determine how many interviews
Unfortunately, this market research- are needed for a win/loss program
based mindset often diminishes the to be effective, it is important to
usefulness of a win/loss program understand the kinds of decisions
and gives the false impression the your organization makes and to
program will need hundreds (or understand how win/loss analysis
even thousands) of data points can help you make better decisions.
to be accurate or useful.
Professor Robert A. Harris, in his
Certainly, if you were to use a VirtualSalt blog, outlines three major
win/loss program to determine types of decisions: strategic, tactical,
the general buying habits of and operational.
your buyers (or the populace as
a whole), you would want to
interview a large number of Strategic decisions determine the
random customers in order to long-term direction of an enterprise,
achieve a representative sample product, or department. They
and a high level of statistical also tend to direct and control an
confidence. However, organization’s resources. Strategic
working with hundreds of decisions involve a high degree of
companies over the past ten uncertainty due to the scope and
years has shown that the time frames usually associated with
most successful win/loss these types of decisions.
analysis programs do not
try to cast such a wide
net. Instead, they
24 • The Pragmatic Marketer • Volume 8, Issue 3, 2010
25. Real-world examples of win/loss Operational decisions support The number of data points needed
analysis that helped clients make tactical and strategic decisions. They to support accurate decision-
better strategic decisions include: help to ensure that daily operations making increases as you move
go smoothly and that tactical from operational (which is often
• Competitive losses indicate buyers decisions are implemented. As the immediate and one data point
name indicates, these are usually can be sufficient) to strategic,
− want a vendor with strong off-
made by managers, although they which may require a large amount
shore capabilities in two specific can also be made by executives. of information to get a clear
countries picture of the entire situation. For
− had issues with key contractual Real-world examples of win/loss instance, deciding which paper
program intelligence that help clients supplier to use for a direct mail
terms the client requires of
make operational decisions include: campaign (an operational decision)
new customers would probably not require client
• Buyers in both wins and losses interviews, but if you wanted to
• A buyer in a lost opportunity change the focus of your company
describe a specific set of needs indicated from services to software (a strategic
that aren’t currently addressed
− the sales representative poorly decision), it is in your best interest
in the market and warrant to interview a representative number
development of a new product presented the solution and did
not understand how their product of current clients to determine how
would solve the buyer’s problems they would react to such a change.
Tactical decisions support or
achieve strategic decisions. In other − they didn’t select the client
words, tactical decisions are made primarily because a product Win/loss as a decision-making
to help bring about the results feature was not available, but the and decision-support tool
envisioned in the organization’s client’s product actually had the
overall strategy. As such, they tend feature From my experience over the
to be more immediate than strategic last ten years, I found that win/
− they are still evaluating solutions loss analysis provides most benefit
decisions, and can be made at
both the managerial and executive and would like to hear from at the tactical and operational
level, whereas strategic decisions the client again (this situation level, although, certainly, it does
are almost exclusively determined happens more often than provide tremendous insight at the
by executives. you think) strategic level as well. It just takes
• A buyer in a won opportunity a little more time before you can
Real-world examples of win/loss indicates they were having trouble start to lean on your win/loss
program data that led clients to with communication and it was program when making long-term,
make tactical decisions include: having a negative impact on their strategic decisions.
decision satisfaction
• Competitive wins indicate a key On average, we find for each
factor in buyers’ decisions was the win, loss, or no decision interview
presence of superior sales performed, you can expect
engineer support to identify an average of ten
operational or tactical decisions
• Buyers in competitive wins talked that can be acted upon.
about specific sales messaging and
sales collateral that helped them
make their decision, while buyers
in lost opportunities did not
reference these things
On average, for each win, loss, or no decision interview performed,
• Buyers in losses cited over-
confidence of the sales you can expect to identify an average of ten operational or tactical
representatives and disparaging decisions that can be acted upon.
the competition as key reasons
for non-selection
The Pragmatic Marketer • Volume 8, Issue 3, 2010 • 25
