A presentation to the Washington, D.C. chapter of the Project Management Institute describing prediction market technology and how it can be used in project and program management.
CBMI 2013 Presentation: User Intentions in Multimedia
Prediction Markets In Project and Program Management
1. Prediction Markets for
Program and Project
Management
PMI WDC Tools
Seminar
Tom Erickson
17 April 2012
2. Agenda
• The Wisdom of Crowds
• Prediction Markets
• An Audience Participation Demo
• Prediction Market Characteristics
• A Real Prediction Market
• Prediction Markets for Knowledge Areas
• Lessons Learned
4/17/2012 Tom Erickson, 2012 All Rights Reserved 2
3. “The Wisdom of Crowds” 1
• How Many Jelly Beans?
• Average of answers from
a large population will be
more accurate than most
of the individual answers.
• A diverse collection of
independently-deciding
individuals is likely to
make certain types of
decisions and predictions
better than individuals or
2
even experts .
1. “The Wisdom of Crowds”, James Surowiecki, Doubleday, 2004
4/17/2012 3
2. http://en.wikipedia.org/wiki/The_Wisdom_of_Crowds
4. What If. . .?
4/17/2012 Tom Erickson, 2012 All Rights Reserved 4
5. The Problem
I Need A
Prediction
Market!
4/17/2012 Tom Erickson, 2012 All Rights Reserved 5
6. 3
What Is a Prediction Market?
We can look to a popular television
show to explain what prediction
markets are:
In “Who Wants to be a Millionaire,” when
contestants get stuck, they have three
lifelines:
» 50/50
Eliminate two of the four answers
» Phone a friend
Call whoever you want to get help
» Ask the audience (93% accurate)
Use the highest percentage answer from the audience
A prediction market is like “asking the audience.”
4/17/2012 Inkling Markets, Inc. Used with Permission 6
7. The Nebit IPL
• Potentially earth-shaking new
consumer product
• Initial, simultaneous, multi-site
initial product launch (IPL)
pending
• Initial product runs successful
(more or less)
• Mahogany row has high
expectations
• What could go wrong?
Vote your confidence in the IPL‟s success.
4/17/2012 Tom Erickson, 2012 All Rights Reserved 7
8. The Nebit IPL Prediction Market
4/17/2012 Tom Erickson, 2012 All Rights Reserved 8
9. Prediction Market Characteristics
• Prediction market systems:
– Can assess the consensus of large (selected)
populations efficiently, on a real-time basis
– Are usually web-based
– Pose questions about future events
– Can look and feel like stock markets, complete with
„play money‟, buys and sells
– Can assess multiple future outcomes in parallel; i.e.,
they can include many questions on many topics at
the same time
– Often include tools for statistical and trend analysis
– Frequently allow users to pose their own questions
4/17/2012 Tom Erickson, 2012 All Rights Reserved 9
10. Surveys vs Prediction Markets3
Typically if a company wants to understand people‟s opinions they
conduct a survey or poll. A prediction market is different from a
survey or poll.
• Surveys and polls • A prediction market asks
typically ask what a what a person predicts will
person wants to happen happen whether they want
“Who will you vote for? the outcome to happen or
not
“Who do you think will be
elected?
• Surveys and polls are • A prediction market gives
snapshots in time. A an ongoing view in to what
person cannot change people think will happen
their answer once they and allows people to
have answered it change their mind based
on external events or new
information
4/17/2012 Inkling Markets, Inc. Used with Permission 10
11. Wise Crowds
Criteria Description
Each person should have private information even if
Diversity of opinion
it's just an eccentric interpretation of the known facts.
People's opinions aren't determined by the opinions of
Independence
those around them.
People are able to specialize and draw on local
Decentralization
knowledge.
Some mechanism exists for turning private judgments
Aggregation
into a collective decision.
4/17/2012 “The Wisdom of Crowds”, James Surowiecki, Doubleday, 2004 11
12. This Just In. . .
You are the shipping
manager and you heard
through the grapevine that
one of your shippers has just
won another major contract
that will use all their spare
capacity, and then some.
4/17/2012 Tom Erickson, 2012 All Rights Reserved 12
14. News Flash
During a production staff
meeting the industrial
engineers monitoring the
production statistics reported
a 20% improvement in
throughput over the initial
production run, but this news
won‟t be reported up the
chain until the next monthly
PMR.
4/17/2012 Tom Erickson, 2012 All Rights Reserved 14
15. Project/Program Knowledge Areas
The Standard for Project Management The Standard for Program Management
Integration Management Integration Management
Scope Management Scope Management
Time Management Time Management
Cost Management Cost Management
Quality Management Quality Management
Human Resource Management Human Resource Management
Communications Management Communications Management
Risk Management Risk Management
Procurement Management Procurement Management
Financial Management
Stakeholder Management
Program Governance
4/17/2012 Tom Erickson, 2012 All Rights Reserved 15
16. Prediction Market Questions
Knowledge Area Prediction Market Questions
• How many changes will the Charter go through
during the project?
•Will the Program Management Plan get signed
Integration Management
off before CDR?
• Will all class 1 changes become resolved before
IOC?
• How much will requirements increase before
PDR?
Scope Management • How many errors will the audit find in the WBS?
• How many errors will the functional configuration
audit find?
• What percent of the project activities will turn out
to be on the critical path?
Time Management • How many activities will be added to the
schedule during final design?
• Will FQT complete on schedule?
4/17/2012 Tom Erickson, 2012 All Rights Reserved 16
17. Prediction Market Questions
Knowledge Area Prediction Market Questions
• When will the overall program CPI exceed 5% of
baseline?
Cost Management • How long will it take to complete the next rolling
wave budget?
• Will the cost variance at IOC exceed 15%?
• Will the customer approve the QMP in time for
CDR?
• Will Program XYZ successfully pass Phase 2
Quality Management
PCA?
• How many process escapes will occur during low
rate initial production?
• Will we meet the project start up 100-day staffing
levels?
• What percentage of the staff will complete time
Human Resource Management
accounting training before the next timecard
audit?
• How many key personnel will leave by FAT?
4/17/2012 Tom Erickson, 2012 All Rights Reserved 17
18. Prediction Market Questions
Knowledge Area Prediction Market Questions
• How many stakeholders will attend the PDR?
• What will be the score from the next
communications survey with the team? With the
Communications Management
stakeholders?
• How many staff hours will it take to prepare for
the next quarterly status review?
• What will be the score of the next business
resiliency exercise?
• Where is the next unforeseen problem most
Risk Management
likely to occur?
• How many opportune uncertainties will the staff
identify in the next quarter?
• Will the contract get signed before the deadline
for ordering long-lead materiel passes?
•How much will the contract mod backlog grow
Procurement Management
before CDR?
• How many of our sole source suppliers will we
lose during low rate initial production?
4/17/2012 Tom Erickson, 2012 All Rights Reserved 18
19. Prediction Market Questions
Knowledge Area Prediction Market Questions
• Will this year’s program ROIC exceed
expectations?
Financial Management • How much will our cost of capital change this
fiscal year?
• Which program will exceed expected cash flow?
• Will the COO approve the stakeholder
management/ communications plan?
• How many stakeholders will decline the next
Stakeholder Management
semi-annual program review?
• Which key stakeholders will be most difficult to
schedule for one-on-ones?
• Which program benefits expected this year will
not appear?
• Which functional area will generate the most
Program Governance
audit findings this year?
• How much will quality escapes erode program
profits in the next five years?
4/17/2012 Tom Erickson, 2012 All Rights Reserved 19
20. The Leak
A member of the West Coast
marketing team overhears
one of the retailers at a
conference describe The
Nebit to a competitor. It is
one week until IPL.
4/17/2012 Tom Erickson, 2012 All Rights Reserved 20
21. Lessons Learned
• Need diverse mix of participants
– Different levels
– Different organizational areas
• Traders must have some knowledge or frame of
reference of at least 3-4 questions being asked in the
marketplace
• Maintain anonymity among peers
• Don‟t try to force organizational structure on
marketplace
• Users will self-select what to trade in
• People take the fantasy currency seriously
4/17/2012 Inkling Markets, Inc. Used with permission. 21
22. Lessons Learned (cont.)
• Prizes don‟t work that well. Here are some alternatives:
– Reward participants with exclusive access to people
and information
– Create a competitive environment between groups
– Create a community of practice
– Let people ask their own questions. Make the
marketplace feel organic, not forced from above
In addition to incentives, proper management of the marketplace is
the #1 indicator of success vs. failure
4/17/2012 Inkling Markets, Inc. Used with permission. 22
23. The Nebit IPL Prediction Market
What has been
happening?
4/17/2012 Tom Erickson, 2012 All Rights Reserved 23
25. Conclusions
• Prediction Markets can provide the capability to
efficiently engage the “wisdom of crowds”
phenomenon.
• Prediction Markets can support all Project/Program
Knowledge Areas.
• Prediction Markets are not suitable for all projects and
populations.
4/17/2012 Tom Erickson, 2012 All Rights Reserved 25
27. Acknowledegments
• I am grateful to Adam Siegel and the Inkling Markets
team for allowing me to include their material in this
presentation, as well as their past support and
assistance while using their tools for demos and
prototypes.
4/17/2012 Tom Erickson, 2012 All Rights Reserved 27
28. Contact Information
• Tom Erickson:
– terickson@apnbok.com
– www.linkedin.com/in/tcerickson
– 703-201-0021
• Adam Siegel:
– adam@inklingmarkets.com
– www.inklingmarkets.com
– (773) 856-6087
4/17/2012 28
30. Prediction Market Systems
• Commercial Systems
– Consensus Point
– Gexid
– Hollywood Stock Exchange (HSX)
– Inkling Markets, Inc.
– InTrade/TradeSports
– Lumenogic
– Nosco
– Pro:kons
• Open Source Systems
– Conversocial
– IdeaFutures
– MarMix
– Serotonin
– USIFEX
– Zocalo
4/17/2012 http://sefier.in/~irj/archives/long-review-market-prediction-software 30
Notes de l'éditeur
It all began at a county fair in 1907. There was a contest to guess how much an ox’s meat would weight after butchering. Francis Galton, a polymath of the 19th century, was surprised to find that the medianof the crowds’ answers was within 9 pounds of the actual result. Later he found that the average (mean) of the individual answers was within 1 pound of the butchered weight! James Surowiecki, in a book published in 2004 built on this and other similar observations to describe the phenomenon we refer to today as the Wisdom of Crowds.
What if you could use the wisdom of crowds to predict future events on your projects?
The problem – crowds today are not nearly so well-behaved as they were at the turn of the 20th century. Crowds are bigger, spread over very large geographical areas, and can generate unimaginable quantities of data. Today, to tap the wisdom of crowds you need tools, and prediction markets are the answer.
You thought you could just sit there and pick up a PDU? We are going to conduct a prediction market of sorts during the presentation. (This is probably the first time at a tools session a presenter has told you to keep your cell phones turned on.)When you came in you found a sheet of paper on your chair describing a future fictitious event, and during the presentation I am going to ask you to express your opinion about the outcome. You will be allowed to change your mind as many times as you wish, and you will use your cell phone, or tablet (if you can get on the hotel WiFi or have cellular access) to provide your guesses. Here is the setup in a nutshell.
We are using a polling engine called Poll Anywhere that can accept cell phone text messages to guess the outcome. I have a poll set up for the scenario described on your handout. Here is how it works. If you gut tells you the IPL “. . . Will do well. Execs will be happy” you will send a text message to 22333 containing only the code 298815.Alternatively you can direct your phone’s browser to PollEV.com/Nebit (case sensitive) and select your choice. You can change your mind as many times as you wish during the presentation.Go ahead. If you have a gut feeling about how the IPL will turn out, text your choice to 22333, or connect with a browser to PollEv.com/Nebit (case sensitive) and pick your best guess. I’ll give you a few minutes before I proceed.(Whistling the Jeopardy theme. . .)If everyone has had a chance to post their initial thoughts, let me go on to describe some of the characteristics of a prediction market.
Almost all prediction markets share common characteristics. How they may implement these features may vary, but the concepts will be similar.The last item is important. Allowing users to pose their own questions has the double benefit of raising issues or potential problems that might not otherwise occur to management or the prediction market operators as well as providing users a reason to have a vested interest in the market.
A prediction market is not another type of survey.Surveys ask “what will you do”; prediction markets ask “what will happen”.Surveys are snapshots in time; prediction markets can be continuous.
Subsequent research on the idea has shown that you need particular characteristics for accurate results from the crowd.
Bulletin!Does this change your opinion about the success of the Nebit IPL? If so, take a moment to post your choice on the poll.O.K. So much for the theory. Let’s take a look at a real prediction market.
I will give you a quick guided tour of the Inkling Markets public marketplace. (If you view these slides in presentation mode and click on the graphic it will launch your default browser and open the Inkling Markets public marketplace.)
Uh-oh – another update from the front.This is good news, right? Does it change your intuition about the success of the Nebit IPL? I’ll give you a few minutes to express your opinion before I proceed.Moving on, I promised I would talk about prediction markets in the context of project and program management. In the next few slides I will show examples of prediction market questions for all project and program management knowledge areas.
Comparing the knowledge areas for the Standard for Project Management and the Standard for Program Management, we can see they are very similar. Program Management adds three knowledge areas, and as you would expect the process stakeholders and scope for the two standards won’t be the same, but for our purpose we can address both Standards together.
Here are typical questions you might want to include in a project or program management prediction market for Integration Management, Scope Management and Time Management.
Here are typical questions you might want to include in a project or program management prediction market for Cost Management, Quality Management and Human Resource Management.
Here are typical questions you might want to include in a project or program management prediction market for Communications Management, Risk Management and Procurement Management.
Here are typical questions you might want to include in program management prediction market for Financial Management, Stakeholder Management and Program Governance.
Looks like theNebit IPL is sailing into troubled waters.What does this bit of news do to your confidence in the success of the Nebit IPL? Take a moment to express your guess on the poll.
Successful prediction markets need to be managed. Here are several points to keep in mind as you plan for and operate a prediction market on your program.
Incentives are important to keep people engaged, but prizes usually aren’t the most effective.
This was the final result of the poll taken throughout the presentation.
Thank you for your time, and for your questions and comments. Does anyone want to explore any additional thoughts before we wrap up?
I must acknowledge the support Inkling Markets has provided, with material and advice, in preparing this presentation.
If you have additional questions feel free to contact me. If you want to know more about Inkling’s products, contact Adam or the team at Inkling Markets – they are the experts.
For your further reading or research here is a list of commercial and open source prediction market systems and software compiled recently at the URL provided.