Regulatory authorities use different efficiency assessment methods to support the setting of efficiency increase targets for the regulated service providers. Session 2 describes the principles of this regulatory benchmarking. Within this session the various mathematical techniques to measure efficiency and their characteristics are presented: · Uni-dimensional ratio analysis · Statistical and econometric methods · Linear programming methods · Virtual network models Furthermore, it is discussed why efficiency should be measured, what role efficiency assessment plays and how the efficiency results are applied and incorporated in the price control. The status quo of efficiency analysis in the EU is presented in a short synopsis.

1. Introduction to Network Regulation
Module 2: Role of Efficiency Analysis
Dr. Konstantin Petrov, DNV KEMA
4 November 2013

2. Agenda
1. Introduction to Efficiency Analysis
2. Methods for Efficiency Assessment
3. Application of Efficiency Results
4. International Examples
Introduction to Network Regulation
4 November 2013

3. Introduction to Efficiency Analysis
Why measure efficiency?
 Usually, competition forces companies to
operate in an efficient way
Cap Regulation
 But, in areas where competition does not work
(e.g. natural monopolies - transmission,
distribution networks) regulation is needed to
limit excessive pricing and to set incentives for
efficient performance
 In cost-based regulatory schemes, a fixed rate of
return compensates the companies and little
incentives to minimise costs are provided
 Incentive regulatory schemes are explicitly
designed to provide incentives for cost-efficiency
 Incentive regulation is based on benchmarking
which regulators use to assess efficiency of
regulated companies and to set targets
Introduction to Network Regulation
4 November 2013
3
Current price level
Current price + Inflation
Current price + Inflation – productivity growth
Actual Cost
Efficiency gains
Set by regulator
Influenced by company
Base price
for next
regulatory
period
Influenced by company
time

4. Introduction to Efficiency Analysis
What is efficiency?
EfficiencyA =
OutputsA
InputsA
+
“Correction for
Environment”
Input Factors
Output Factors
Distribution Company A
e.g. # customers, delivered
energy (kWh), peak load (kW)
e.g. # employees, fuel,
operational costs,
Environmental Factors
e.g. firm size, network topology, climate, topography,
terrain, task complexity
 Efficiency characterises the productivity of a company compared with the productivity of other
companies.
Introduction to Network Regulation
4 November 2013
4

5. Introduction to Efficiency Analysis
Why are companies inefficient?
 Inefficiency is a deviation from the optimal point on the production or cost frontier.
 Two main sources for this deviation: technological deficits and problems due to a non-optimal
allocation of resources into production
 Sources for efficiency changes:
- Technological change (frontier shift): change in production technology within the sector
- Efficiency change (catch-up): change in efficiency of production
• Change in the scale of production (scale efficiency)
• Pure technical efficiency change
- Allocative efficiency
• Input mix allocative efficiency: producing same outputs with different mix of inputs
• Output mix allocative efficiency: producing different level of outputs with same mix of
inputs
- Changes in operating environment
Introduction to Network Regulation
4 November 2013
5

6. Agenda
1. Introduction to Efficiency Analysis
2. Methods for Efficiency Assessment
3. Application of Efficiency Results
4. International Examples
Introduction to Network Regulation
4 November 2013
6

7. Methods for Efficiency Assessment
Decision Sequence
Benchmarking Process
Steps in Benchmarking Analysis
Step 1
Step 2
Step 3
Model Specification
Choice of
Approach
for Efficiency
Measurement
1) DEA
2) SFA
3) OLS
4) COLS
5) Partial
Choice of
Model
Parameters
1) Model
Orientation
2) Constant or
Variable
Returns to
Scale
3) Definition of
Inputs/ Outputs
Step 4
Step 5
Model Application
Choice of
Sample Size
and Data
Collection
1) National
versus
International
2) Comparison
Criteria
3) Data
Validation
Introduction to Network Regulation
4 November 2013
7
Model Run
Result
Validation
1) Application of Alternative
Approaches
2) Sensitivity Analysis
Related to Input and Output
Parameters
3) Check for Outliers

8. Methods for Efficiency Assessment
Choice of Benchmarking Method
 Benchmarking (efficiency performance assessment) is applied based on a variety of
methods ranging from basic indicators to more complex measures
 Methods differ in the standard of comparison
 Benchmarking influences the allowed revenue of companies and the price level
reliability of inefficiency scores and the method chosen is crucial for the regulator
 There is no consensus among regulators at to which methodology is the best
Benchmarking should not be applied mechanically
Sometimes different methods are applied simultaneously
 Frontier methods preferred by regulators (in particular DEA and SFA)
- Parametric (econometric) models (Germany, UK)
- DEA analysis (Norway, the Netherlands, Germany, several countries in CEE)
- Reference network models (Spain, Sweden, Chile)
Introduction to Network Regulation
4 November 2013
8

9. Methods for Efficiency Assessment
Overview of Benchmarking Methods
Benchmarking Methods
Partial
Methods
Performance
Indicators
Total Methods
Index
Methods
NonParametric
Linear
Programming
UniDimensional
Ratios
Total
Factor
Productivity
(TFP)
Engineering
Models
Parametric
Econometrics
Data
Envelopment
Analysis
(DEA)
Ordinary
Least
Squares
(OLS)
Introduction to Network Regulation
4 November 2013
9
Corrected
Ordinary
Least
Squares
(COLS)
Stochastic
Frontier
Analysis
(SFA)
Reference
Networks
(Virtual
Networks)

10. Methods for Efficiency Assessment
Partial vs. Total Methods
 Partial methods use uni-dimensional ratios; comparison of single performance
indicators between firms:
Productivity Indicators:
Financial Indicators:
• GWh/Employee
• Debt/Equity Ratio
• OPEX/GWh
• Return on Investment
(ROI)
• OPEX/Employee
• GWh/Line Length
• Return on Capital
Employed (ROCE)
 Partial methods produce simple, easy to calculate and straightforward indicators of
performance
 But: they fail to account for the relationships between different input and output
factors and do not recognise trade-offs between different improvement possibilities
 Total methods can capture this trade-off
…at the expense of higher computational complexity
Introduction to Network Regulation
4 November 2013
10

11. Methods for Efficiency Assessment
Index- vs. Frontier-based Methods
 Index method – Total Factor Productivity (TFP):
- Measure of physical output of a regulated company produced by a given
TFP 
quantity of inputs
- With multiple inputs (Y) and outputs (X), outputs are usually weighted by
their revenue shares (sR) and inputs are weighted by their cost shares (sC)
- Usually used for assessments of company performance over time
m
s
R
i 1
n
s
j 1
C
j
Y
i i
Xj
 Frontier-based methods:
- based on the concept that all companies should be able to operate at an optimal efficiency
level/ “frontier” that is determined by other efficient “peer” companies in the same sample
- The companies that form the efficiency frontier use the minimum quantity of inputs to
produce the same quantity of outputs (input oriented model)
- The efficiency frontier is used as a reference against which the comparative performance of
all other companies (that do not lie on the frontier) is measured
- The distance to the efficiency frontier provides a measure for the inefficiency
Introduction to Network Regulation
4 November 2013
11

12. Methods for Efficiency Assessment
Non-parametric vs. Parametric Models (Frontier-based Methods)
Input X
(Costs)
𝑋2
𝑌
Parametric Methods
Ordinary Least
Squares (OLS)
Stochastic Frontier
Approach (SFA)
Corrected Ordinary
Least Squares
(COLS)
Most Efficient
Observation
Non-Parametric Methods
Efficiency Frontier
Most Efficient
Companies
A
E
B
E’
Inefficiency
C
Output Y
 Econometric methods use cost or production
functions and regression analysis. SFA accounts
for stochastic noise in the data sample
𝑋1
𝑌
D
E
 DEA uses multi-input / output analysis
based on linear programming
x1 /y
Introduction to Network Regulation
4 November 2013
12

13. Agenda
1. Introduction to Efficiency Analysis
2. Methods for Efficiency Assessment
3. Application of Efficiency Results
4. International Examples
Introduction to Network Regulation
4 November 2013
13

14. Application of Efficiency Results
Efficiency Assessment and Price Control
Efficiency
Interface
Allowed Revenue (Tariffs)
Efficiency
Benchmarking
Price
Targets
Scores
Integration in
Improvement
Efficiency
Assessment
Efficiency
Control
Conversion
Integration
 Approach
 Convergence Time
 Chargeable Basis
 Sample
 Convergence Profile
 Capex Treatment
 Model Orientation
 Inefficiency Caps
 Revenue Requirements
 Data Collection
 Efficiency Bands
 Regulatory Formula
 Data Validation
Introduction to Network Regulation
4 November 2013
14

15. Application of Efficiency Results
Defining Efficiency Increase Targets
 Once calculated efficiency scores should be converted into efficiency increase
requirements (X-factors).
 X-Factor ensures ex-ante
Measures of relative
sharing of anticipated efficiency
inefficiencies towards best
gains
Efficiency
performance
 X-Factor can be calculated:
Score
Conversion
(definition of
efficiency
increase
targets)
- Indirectly as a difference between
the level of actual costs and target
(efficient) costs
- Directly without reference to target
costs using just past performance
 In some regulatory regimes the
X-factor has a dual function:
- Efficiency improvement
- Revenue profiling
A
B
Introduction to Network Regulation
4 November 2013
15
C
D
E
Companies

16. Application of Efficiency Results
Efficiency Convergence Speed
 The X-factor prescribes the rate of change in the company’s prices or revenues, reflecting the
expected transition from the existing price level towards the efficient price level
 The efficiency convergence may be based on an initial one-off cut or gradual adjustment path
during the regulatory period
 Advantage of initial one-off cut, prices can be
brought to more realistic levels at once
Allowed
Revenue
 Large one-off adjustments quickly eliminate
inefficiencies at the beginning, but decrease
incentives for further efficiency improvements
by the company
Initial one-off cut
Initial
Level
Proportional
decrease
1
2
3
4
5
 Incentives for efficiency increase can be
further supported by efficiency carry-over
schemes: companies are allowed to continue
keeping part of the efficiency gains of the
previous period
Regulatory Period
Introduction to Network Regulation
4 November 2013
16

17. Agenda
1. Introduction to Efficiency Analysis
2. Methods for Efficiency Assessment
3. Application of Efficiency Results
4. International Examples
Introduction to Network Regulation
4 November 2013
17

18. International Examples
Country
Benchmarking Methods
Benchmarking Sample
United Kingdom
COLS until 2009; DEA and OLS (OPEX)
14 electricity distribution companies
8 gas network distribution companies
The Netherlands
DEA (total controllable costs)
19 Dutch utilities (electricity)
Germany
DEA, SFA (total controllable costs)
198 electricity distribution companies
188 gas distribution companies
Austria
DEA, MOLS (total controllable costs)
20 electricity distribution companies
20 gas distribution companies
Finland
DEA, SFA (OPEX)
88 electricity distribution companies
Norway
DEA
150 national distribution utilities (electricity)
Sweden
Reference network model until 2007;SFA, DEA
170 electricity distribution companies
Spain
Network reference model
5 large and 320 smaller electricity distribution
companies
Portugal
DEA
11 gas distribution companies
Poland
OLS; COLS & DEA
Introduction to Network Regulation
4 November 2013
18

19. End of Session 2.
Dr. Konstantin Petrov
Service Line Leader Markets & Regulation / Business Line Director Gas Consulting Services
DNV KEMA Energy & Sustainability
KEMA Consulting GmbH
Kurt-Schumacher-Str. 8
53113 Bonn
Tel: +49 228 44690 56
Fax: +49 228 4469099
Mobile: +49 173 515 1946
E-mail: konstantin.petrov@dnvkema.com
Introduction to Network Regulation
4 November 2013
19

20. www.dnvkema.com
Introduction to Network Regulation
4 November 2013
20

21. Appendix: Efficiency Assessment Models
Introduction to Network Regulation
4 November 2013

22. Methods for Efficiency Assessment
Non-parametric Model: Data Envelopment Analysis (DEA)
 Calculates the relative Input-Output efficiency of a regulated company by benchmarking an
individual company in relation to the best-practice (most efficient) companies
 Companies that are able to produce a given output at minimum cost or a maximum output with
a given input define the best-practice frontier that envelops all data points
 Inefficiency is determined by the distance between the observed company and the bestpractice frontier
 Calculation of inefficiency is conducted via a series of linear programming (mathematical
software needed)
 The programs will output a series of efficiency scores, which may be normalised, ranked, and
split according to a number of components (scale, purely technical, allocative etc.)
 Advantages: multi-dimensional method; functional relationships between input and output
factors not required; distinguishes between different types of inefficiency
 Disadvantages: results may be influenced by random errors; no information about statistical
significance of the results; danger of over-specification of model and “made-up” results for
efficiency scores; “extreme” parameters regarded as efficient “by default”
Introduction to Network Regulation
4 November 2013
22

23. Methods for Efficiency Assessment
Non-parametric Model: Data Envelopment Analysis (DEA)
Output Maximisation
Input Minimisation
Output 1
Input 1
G’
Data Envelope
Data Envelope
A
most efficient
companies
G
B
most efficient
companies
A
F’
F
Inefficiency
C
F
Inefficiency
B
D
F’
C
E
D
Output 2
Introduction to Network Regulation
4 November 2013
23
E
Input 2

24. Methods for Efficiency Assessment
Parametric Models
 Regression analysis: Mathematical relationship (functional form) that describes the relationship
between a dependent variable and one or more independent variables
 Use of regression residuals to characterise relative distances between observations in the
sample
 Treats best practice as a “stochastic” process (a mix of true efficiency and “random noise”
effects, SFA)
 Advantages: ability to control for unobserved heterogeneity among companies; less sensitive
to inputs and/or outputs than other parametric models; allows to assess the significance of
each network cost driver; considers stochastic errors explicitly
 Disadvantages: requires assumptions of functional form; requires large data sets in order to
create a robust regression relationship; complex and statistically demanding
Introduction to Network Regulation
4 November 2013
24

25. Methods for Efficiency Assessment
Parametric Models
Input (Costs)
Ordinary Least Square (OLS)
Stochastic Frontier Analysis (SFA)
Corrected OLS (COLS)
Most efficient
observation
Output
Introduction to Network Regulation
4 November 2013
25

26. Methods for Efficiency Assessment
Virtual Network Models
 Construct an efficient (engineering-designed) reference network according to commonly
accepted planning principles and taking into account technical and geographical constraints
 The regulated firm’s relative (in)efficiency is estimated by the firm’s performance in relation to
the virtual network
 Advantages: Virtual network models are not dependent on obtaining and analysing data of
“real” companies; does not require a significant set of comparable companies as benchmarks
 Disadvantages: It might be complicated and difficult to specify; model sensitive to changes in
inputs; reasons for the deviation from reference network might be beyond control of the
company
Introduction to Network Regulation
4 November 2013
26