Review of the EU egovernment benchmark. A comparison with other benchmarks (EU, WEF). Framework and ideas to improve benchmarking egovernment, citizen services.

1. Dr. Alexander Schellong
Berlin, 30/09/2011
EU E-Government Benchmark
Benchmarking Open Government

2. Why benchmarking?
© 2011 Capgemini. All rights reserved. 2
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
• Evaluate
• Control
• Budget
• Motivate
• Promote
• Celebrate
• Learn
• Improve

3. Curent / Past EU eGovernment benchmark
© 2011 Capgemini. All rights reserved. 3
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
Full-On line
Availability
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Aus
tri
a
M
al
taPor
tu
ga
lSlo
ve
ni
a
Uni
te
d
Kin
gd
om
Nor
waySwed
enG
er
m
an
yEst
on
ia
Fr
an
ce
Ita
ly
Spa
inFi
nl
an
dDen
m
ar
k
Net
he
rla
nd
sBel
gi
um
Cze
ch
Rep
ub
lic
Tu
rk
eyHun
ga
ryIc
el
an
dIre
la
nd
Cyp
ru
sG
re
ec
e
Lu
xe
m
bo
ur
gLi
th
ua
ni
aRom
an
iaSlo
va
kia
La
tv
iaPol
an
d
Switz
er
la
ndBul
ga
ria
Full online availability EU27+
Country
Reports
Annex
Service
Results &
Definitions
•More informed /
structured
Landscaping
Management Summary
Pilot /
Elective
Measures:
- Service
Directive
- Life
Events
- PEGS
3
National Portal E-procurement
4
Sophistication User-Centricity
1
2
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Aus
tria
Slo
ve
ni
a
M
al
ta
Por
tu
ga
l
Unite
d
Kin
gd
om
Fra
nc
e
Swed
en
Est
on
ia
Norw
ay
G
er
m
an
y
Spa
in
Neth
er
la
nd
s
Fin
la
nd
Bel
gi
um
Denm
ar
k
Ita
ly
Irel
an
d
Cze
ch
Rep
ubl
ic
Hung
ar
y
Tur
ke
y
G
re
ec
e
Ic
el
and
Lu
xe
m
bo
ur
g
Cyp
ru
s
Li
th
ua
ni
a
Switz
er
la
nd
Rom
ani
a
Slo
va
kia
La
tv
ia
Pol
an
d
Bul
ga
ria
Online sophistication EU27+
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Aus
triaSlo
ve
ni
a
M
al
taPor
tu
ga
l
Unite
d
Kin
gd
om
Fra
nc
eSwed
en
Est
on
iaNorw
ayG
er
m
an
y
Spa
in
Neth
er
la
nd
sFin
la
ndBel
gi
umDenm
ar
k
Ita
ly
Irel
an
d
Cze
ch
Rep
ubl
icHung
ar
yTur
ke
yG
re
ec
eIc
el
and
Lu
xe
m
bo
ur
gCyp
ru
sLi
th
ua
ni
a
Switz
er
la
ndRom
ani
aSlo
va
kia
La
tv
iaPol
an
dBul
ga
ria
Online sophistication EU27+
Basic 20 Services
User Satisfaction5

4. Timetable EU eGovernment benchmark
© 2011 Capgemini. All rights reserved. 4
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
March Kick Off
April Revision of measurements
Expert group peer review of approach
Methodology & indicator definitions accepted
Country list finalized
Start of execution of web surveys
May-June Execution of web surveys
Pilots
July-August Processing of Pilots
Quality Control Findings
August-
September Validation with member states
Meeting to review draft report
Oct Completion of final report
Nov: Presentation of report at e.g. Malmoe
Phase 1: Methodological preparations
Phase 2: Survey Benchmark Preparations
Phase 3: Conduct Benchmark study
Phase 4: Process results and reporting
March April May June July August September October
Methodology &
Indicators
Agreed
Country Data
Obtained
Preliminary
report
Final
report
Phase planning & milestones

5. Basic 20 services: Sophistication / Online Availabiity
Policy Goal: i2010 – Efficiency and Effectiveness, Service
Transformation
© 2011 Capgemini. All rights reserved. 5
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
Commentary
• A shift from a supply-side indicator towards a smart indicator: case-oriented view of e-government
services. Requires revision of:
- List of services
- Definition of services
• Value of International Expansion
• Ongoing review of measurement procedures to ensure maximum burden reduction towards MS.
Current Position
Indicator: 5-stages
maturity model
Stage 1 – Information
Stage 2 - Interaction
Stage 3 - Two-way
interaction
Stage 4 - Transaction
Stage 5 - Proactive
and/or automatic
delivery of service.
Unit of analysis:
List of URLs
2009
Indicator:
Indicator will remain stable in 2009 to allow for
comparability.
- Possible addition to review of 20 services:
Pan-European interoperability,
participation/consultation
Action:
- Approach requires rapid validation by
Member State representatives.
- Piloting in 2009 of alternative methods (e.g.
webcrawlers)
Unit of analysis:.
Action: Validation and Update of list of URLs
is required.
Future Direction
-To be evaluated post
2009 measurement
-Gov 2.0 policy
priorities, global policy
priorities, life/customer
events
- Benchlearning
exercise to measure
connected government
(Lot 2)

6. User Centricity (citizen satisfaction, business climate)
Policy Goal: i2010 – No citizen left behind, Effectiveness of online
services
© 2011 Capgemini. All rights reserved. 6
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
Commentary
• Data fields subindicator failed
• Accessibility testing (logo) was contested by many MS, Webcrawler methodology reviewing agreed
standards will provide much richer information
• Extension of authentication subindicator to businesses, Investigation of relevance of STORCK to
Authentication
• Benchlearning method for additional subindicators (link to Lot 2) beyond measurement Lot 1.
Basis: Capgemini measurements 2007
Current Position
Indicator:
- Authentication / eID
- Compliance with
accessibility
standards
- Number of data entry
fields
- Multi-channel access
Unit of analysis:
- Validated list of URLs
previously used
2009
Indicator: shift to customer satisfaction
Review of subindicators:
- Data Field (AB Measure) will be dropped
- Looking into feasibility of review of
Accessibility of 20 services by Webcrawler
- Identification of relevant maturity levels sub-
transaction
-Addition of user centricity review of national-
level portals
Action: Agree on relevance of reporting on
citizen satisfaction / business climate
Future Direction
To be re-evaluated post
2009 measurement
Strategic Alignment of
methodology with user
satisfaction
measurement

7. eProcurement
© 2011 Capgemini. All rights reserved. 7
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX

8. 20 public services rank distribution (2007) & (2009)
© 2011 Capgemini. All rights reserved. 8
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
Online Public Service # ranks 2009 MS 1st 2009 # ranks 2007 MS 1st 2007
Job search services 2 29 out of 31 3 29 out of 31
VAT 2 30 out of 31 3 28 out of 31
Corporate tax 3 28 out of 31 4 26 out of 31
Customs declaration 3 29 out of 31 3 26 out of 31
Social contributions for employees 3 29 out of 31 4 25 out of 31
Income taxes 2 23 out of 31 3 20 out of 31
Declaration to the police 4 20 out of 31 4 18 out of 31
Public procurement 3 25 out of 31 3 16 out of 31
Submission of data to statistical offices 2 19 out of 31 4 15 out of 31
Registration of a new company 3 19 out of 31 4 15 out of 31
Car registration 5 21 out of 31 5 14 out of 31
Enrolment in higher education 15 15 out of 31 15 11 out of 31
Public libraries 8 14 out of 31 7 11 out of 31
Announcement of moving 9 13 out of 31 11 11 out of 31
Certificates 9 13 out of 31 13 9 out of 31
Environment-related permits 11 8 out of 31 10 5 out of 31
Health related services 11 7 out of 24 10 4 out of 23
Application for building permission 21 8 out of 31 21 4 out of 31
Personal documents 12 8 out of 31 11 2 out of 31
Social security benefits 18 9 out of 31 23 1 out of 31

9. Comparison of major eGovernment benchmarks / ranks
© 2011 Capgemini. All rights reserved. 9
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
EUeGovBe (07/09)
N: EU27+4
Brown/Brookings (2008)
N: 198
UN (08/10)
N: 184
Austria (1/4) Austria (65) Austria (16/24)
Slovenia (2/5) Slovenia (112) Slovenia (26/29)
Malta (2/1) Malta (56) Malta (29/30)
Portugal (3/2) Portugal (18) Portugal (31/39)
United Kingdom (4/8) United Kingdom (35) United Kingdom (10/4)
France (5/11) France (15) France (9/10)
Sweden (6/3) Sweden (72) Sweden (1/12)
Estonia (7/6) Estonia (59) Estonia (13/20)
Norway (8/15) Norway (60) Norway (3/6)
Germany (9/12) Germany (7) Germany (22/15)
Spain (9/13) Spain (14) Spain (20/9)
Netherlands (10/14) Netherlands (33) Netherlands (5/5)
Finland (11/7) Finland (28) Finland (15/19)
Belgium (12/12) Belgium (105) Belgium (24/16)
Denmark (12/10) Denmark (62) Denmark (2/7)
Italy (13/17) Italy (25) Italy (27/38)
Ireland (14/9) Ireland (8) Ireland (19/21)
Czech Republic (15/19) Czech Republic (55) Czech Republic (25/33)

10. Benchmark framework 1/3: Guiding principles
© 2011 Capgemini. All rights reserved. 10
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
Dimensions Details
Policy Which policies should guide the design? e.g. i2011-15
Frameworks / Cause-and-effect
relationships
Which underlying frameworks guide the design? e.g. eGovernment,
citizen satisfaction, government transformation, online sophistication
model, etc.
Priorities/Scope Which priorities? On a macro-level, what should be measured?
Compared with what?
Governance & Responsibility e.g. How does the process of deciding on measures or guiding
Frameworks look like? Which parties are involved (EC, MS,citizens,
academia)?

11. Benchmark framework 2/3: design
© 2011 Capgemini. All rights reserved. 11
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
Dimensions Details
Framework e.g. Online sophistication model
Unit of analysis e.g. 20 public services and national portal in a country
Measures/variables/indicators Natural, proxy, constructed e.g. Service compliance with accessibility
standard
Scoring and metric e.g. How are measures translated into values or ranks?
Frequency How often?
Data collection e.g. timeline, questions to ask, resources required on governemnt side
Data sources e.g. Who provides the data?; sample size
Data analysis e.g. Type qualitative and quantitative methods
Costs e.g. Value for money of the benchmarking activities
Limitations What are the weaknesses and gaps of the benchmark? Which data is
missing and how can it be made available?
Execution e.g. Who conducts data collection and analysis?

12. Benchmark framework 3/3: Learning and reporting
© 2011 Capgemini. All rights reserved. 12
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
Dimensions Details
Results e.g. Which results are presented? How are they presented?
Access e.g. Which data is accessible? Who has access? How can it be
accessed?

13. Overall Scope: Potential design
The future benchmark should have five focus areas
I. Government 2.0 / Open Government
II. European ICT projects
III. eGovernment services
IV. Information society & User opinion
V. Impact
© 2011 Capgemini. All rights reserved. 13
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX

14. Focus area I: Government 2.0 / Open Government
This area tracks the progress and innovations in five domains of Government 2.0 / Open government through
a variety of indicators.
• Open data portals
• eParticipation (in policy; policy-cycle)
• Collaboration (co-production of service / public management support)
• Social media tools
• Transparency
© 2011 Capgemini. All rights reserved. 14
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX

15. Focus area II: European ICT projects
This area tracks the progress of European wide government ICT projects initiated by DGs or EU agencies
through a variety of indicators.
• VIS (DG Home)
• Single Point of Contact 2.0 / Service Directive (DG Internal Market)
• IMI (DG Internal Market)
• INSPIRE (Environment)
• Safesea Net (EMSA)
• National alert platforms (Cyber Security; Digital Agenda)
• REACH (Environment)
• Prüm treaty data exchange
© 2011 Capgemini. All rights reserved. 15
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX

16. Focus area III: eGovernment services
This area introduces new eGovernment services that should be offered on a transactional basis as well as the
implementation of EU Large Scale Pilot (e.g. eID/STORK) / European interoperability elements. Particular
interest is on the life events of studying, working, retiring and receiving healthcare (Digital Agenda).
• eprocurement
• certificates
• building permits
• visa application
• enrolling in university
• application and management fo childcare / kindergarten
© 2011 Capgemini. All rights reserved. 16
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX

17. Focus area IV: Information society & User opinion
© 2011 Capgemini. All rights reserved. 17
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
This area introduces a set of indicators that describe the state of information society and user opinions.
• user satisfaction surveys
• information society indicators (e.g. internet penetration, ICT 4 schools, ICT R&D investment)

18. Focus area V: Impact
© 2011 Capgemini. All rights reserved. 18
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
This area introduces a comprehensive review of projects that are a case in point for X in a annually changing
policy area.
• Significant budget savings
• eparticipation
• project failure

19. EU eGovernment index (Ranking)
© 2011 Capgemini. All rights reserved. 19
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
NEW
eGovernment index
Government 2.0
EU ICT Projects
eGov Services
INFSO / User
One eGovernment index with four subindexes that will compose the key eGovernment index.
5 sub indexes
8-10 sub indexes
5-10 sub indexes
5-10 sub indicators

20. Gov 2.0 pilot scope
© 2011 Capgemini. All rights reserved. 20
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
Reference to previous
measurement framework
Pillar in new measurement
framework
Correspondence with
eGovernment Action Plan
2012-2015 Priority
Group 1 of the Participation
framework: Policy strategy and
Monitoring
PARTICIPATION INDICATOR
Involvement of citizens and
businesses in policy-making
processes
Group 3 of the Participation
framework: Participation in policy
making -- Openness of policy
process
Group 5 of the Participation
framework: Presence on social
media
COLLABORATION INDICATOR
Collaborative production of
services
Group 1 of the Transparency
framework: Transparency of
Service delivery
TRANSPARENCY INDICATOR Improvement of Transparency
Group 2 of the Participation
framework: Passive participation --
Organizational transparency and
accountability
Group 3 of the Transparency
framework: Personal data

21. Gov 2.0 pilot: collaboration indicators 1/3
© 2011 Capgemini. All rights reserved. 21
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
Question Answer options Classification of indicator Data collection
method
Does this administration involve
citizens in the co-production of
services?
Yes
No
Binary benchmark indicator
0-100% availability score
Web survey of
Institutional
websites
If yes, for which services and how? Yes
No
Binary benchmark indicator
0-100% availability score (if
a portfolio of services can
be determined for the
assessment)
Web survey of
Institutional
websites
Is it possible for citizens to track their
co-production inputs?
Yes
No
Binary benchmark indicator
0-100% availability score
Web survey of
Institutional
websites
Does the website provide evidence on
how the citizens’ input was used?
Yes
No
Binary benchmark indicator
0-100% availability score
Web survey of
Institutional
websites
Are the data and tools needed for co
production up to date?
Yes
No
Binary benchmark indicator
0-100% availability score
Web survey of
Institutional
websites

22. Gov 2.0 pilot: collaboration indicators 2/3
© 2011 Capgemini. All rights reserved. 22
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
Does the website include an an explicit social
media policy by the administration?
Yes
No
Binary benchmark
indicator 0-100%
availability score
Web survey of
Institutional
websites
In which of the following channels is the
administration active?
• Social networking sites (like Facebook)
• Media sharing sites (like Youtube)
• Other (such as Second life)
Yes
No
Binary benchmark,
answers for each option
0-100% availability
Composite score
(weighted by type of
option)
Web survey of
Institutional
websites
Does the administration provide information and
communication through the following tools?
• Tweeting
• Blogging
• Wikis
• social bookmarking, tagging, canvasssing
• polling / voting
• petitioning
• games
• data visualization and/or analytics tools
• other (please specify)
Yes
No
Binary benchmark
answers for each option
0-100% availability
Composite score
(weighted by type of
option)
Web survey of
Institutional
websites

23. Gov 2.0 pilot: collaboration indicators 3/3
© 2011 Capgemini. All rights reserved. 23
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
Does the administration provide open
data for mashing up new content,
services, apps, etc.
Yes
No
Binary benchmark
answers 0-100%
availability
Web survey of
Institutional
websites
Is the use of social media framed
around:
• The institution
• Specific topics/issues suggested
by the government
• Specific topics/ issues suggested
by citizens or businesses
• other (please specify)
Yes
No
Binary benchmark
answers for each option
0-100% availability
Composite score
(weighted by type of
option)
Web survey of
Institutional
websites

24. Open data portals measures
© 2011 Capgemini. All rights reserved. 24
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
Question Variable Data provider
Is there a national OGD portal Yes / No Contractor
Is there a satistics portal Yes / no
Type of formats Formats (e.g. rtf, xls, csv)
SLF criteria: Completness MS
SLF criteria: primary contractor
SLF criteria: timeliness Not measurable
SLF criteria: Is there a registration contractor
SLF criteria: Is the registration fee of charge contractor
SLF criteria: Can disabled access the data contractor
SLF criteria: Tags and other navigation contractor
SLF criteria: machinereadable / API MS
Connected to social media platform Contractor
Statistics & metadata contractor

25. Open data portals measures
© 2011 Capgemini. All rights reserved. 25
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
Question Variable Data provider
RSS feeds Yes / no contractor
Is there a forum? Yes / no contractor
Are users allowed to post apps? Yes / No Contractor
Are users allowed to rate user apps? Yes / no Contractor
Type of formats Formats (e.g. rtf, xls, csv) Contractor
Does the government have an open data policy Yes / no contractor

26. How should we measure?
© 2011 Capgemini. All rights reserved. 26
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
• Open data
• eParticipation (policy)
• collaboration (service)
• Transparency
• Impact

27. © 2011 Capgemini. All rights reserved. 27
EU E-GOVERNMENT BENCHMARK_BARCAMP.PPTX
Thank you!
alexander.schellong@capgemini.com
Blog: http://www.iq.harvard.edu/blog/netgov
Twitter: /schellong

28. www.capgemini.com
The information contained in this presentation is proprietary. ©2011 Capgemini. All rights reserved