Selection bias can occur when choosing renewable energy projects if too much weight is given to the estimated net cash flow (NCF) which has uncertainty. A selection model showed that with a 5 project pool, if the NCF weight is 30% and uncertainty is 9%, selected projects will underperform estimates. To protect against this, companies should apply conservative factors transparently, invest based on a consistent risk level like P95 rather than P50, and act conservatively given the inherent uncertainty in NCF estimates.

1. Selection bias & the value of certainty
Presented by:
Matthew Hendrickson, Sr. Dir. Of Assessment

2. 3TIER is Risk Management
Renewable Energy
Risk Analysis
Powered by weather science, 3TIER helps the
global energy market manage renewable
energy risk—for the next 5 minutes through
the next 50 years.

3. What to do about Risk?

4. What is Risk?
» Risk is often described as uncertainty.
» A typical wind consultant will describe uncertainty as the
standard deviation of error around the primary estimate.
A few questions
» Do we know what this means?
» How does knowledge of risk guide our decisions?
σ=?

5. US Wind Industry Underperformance
In 2008, reports by three US consultants
show that US Wind Industry is
underperforming estimates by 10-11%.
1. “Understanding and Closing the Gap on Plant Performance” – Eric White, AWS Truewind, AWEA WindPower 2008
2. “Project Underperformance: 2008 Udate” – Steve Jones, DNV-GEC, AWEA WindPower 2008
3. “Validation of Energy Predictions by Comparison to Actual Performance, Clint Johnson, AWEA WindPower 2008

6. Underperformance explained
» No single factor described as major source. Rather a list of
possibilities has been suggested.
› Wind farm availability
› Inter-annual variability
› Turbine performance
› Wake effects
› Wind flow modeling
› Measurement bias
» Advancements in every one of the these fields has taken
place and US performance gap is closing.

7. Seems like a bad case of luck
» With so many sources, there is a nagging feeling like much
of what is wrong is luck.
» Could it be luck?
» Or something much more predictable…
» Perhaps it could be Selection Bias

8. Selection Bias Game

9. Project Selection Game
» You work for a large wind developer working in an active
market.
» As Chief Investment Officer, you are tasked to manage
your companies wind investments.
» All of your investments come in the form of winning
RFP’s to sell power in a power purchase agreement
(PPA). With a PPA, your company can build the project.
» Each project costs $100 in capital. Your companies goal
is to make a desired return of $20 for each $100.

10. Game mechanics
Windy Hill Desert Breeze Ocean Front Gusty Pass Bent Corn
True NCF (hidden) 36.3% 35.0% 37.7% 32.5% 37.9%
Uncrtainty 8.3% 9.2% 9.4% 7.7% 7.5%
UncertainNCF 34.6% 27.9% 39.6% 35.2% 42.9%
$100 56.10 68.18 47.13 55.13 41.21
$110 60.77 72.85 51.80 59.80 45.88
$120 65.45 77.52 56.47 64.47 50.56
$130 70.12 82.19 61.15 69.15 55.23
$140 74.79 86.87 65.82 73.82 59.90
$150 79.46 91.54 70.49 78.49 64.57
» Each Investor must pick a project and bid a price that they
think will both win and yield a desired return.
» Lowest price wins.
» True NCF is hidden.

11. Game mechanics
» Your desired return is $120.
» But your CEO becomes very unhappy with returns less than
$110. Since you can’t be fired, he will take investment
capital away from you for low returns and put it somewhere
else.
» He will take away $4 of capital for every $1 below the
threshold return.
» Each projects return gets added to an investors capital.
» The investor with the most capital at the end of the game
wins!

12. Game mechanics
Windy Hill Desert Breeze Ocean Front Gusty Pass Bent Corn
True NCF (hidden) 36.3% 35.0% 37.7% 32.5% 37.9%
Uncrtainty 8.3% 9.2% 9.4% 7.7% 7.5%
UncertainNCF 34.6% 27.9% 39.6% 35.2% 42.9%
$100 56.10 68.18 47.13 55.13 41.21
$110 60.77 72.85 51.80 59.80 45.88
$120 65.45 77.52 56.47 64.47 50.56
$130 70.12 82.19 61.15 69.15 55.23
$140 74.79 86.87 65.82 73.82 59.90
$150 79.46 91.54 70.49 78.49 64.57
» Watch out for uncertainty!
» It might make a project look better or worse than it really is.
» Each projects true NCF varies. That might be a windy
project or it might be a dud.

13. Play the game
Launch Game

14. Selection bias model
» Selection model created that simulated selection
» Factors included
› Energy Estimate Uncertainty (Estimator of Risk)
› Weight of NCF in Selection Decision (How much do you care?)
› Market Pressure (Forces outside of your control)
› Selection Pool Size (How many choices?)
» Model ran iteratively across range of values
» Simulated decision process of investor

15. Selection Model Results
Assumes 30% NCF Weight & 5 project selection pool

16. Selection Model Results
Assumes 9% uncertainty, 1.1x market threshold, 30% NCF Weight & 5 project selection pool

17. How to protect yourself
Conclusion – whenever project return is a consideration
(almost always), selection bias will be present.
» Act with conservatism
› Apply conservative factors where appropriate
› But these factors should be transparent to avoid double-dipping
» System must resist uncertainty
› Investing on the P50 level flawed
› Pick a risk level (PX) that is comfortable and stick with it. (e.g.
Invest at the probability level such that your P95 scenerio is at least
a break even scenerio)