The proposed methodology allows a comprehensive assessment of the various types of value generated by a PSI e-infrastructure for each stakeholder group, and also the interconnections among them.

1. University of the Aegean – Department of Information and Communication Systems Engineering
A methodology for Evaluating PSI e-Infrastructures
based on Multiple Value Models
Charalampos Alexopoulos, cPhD
Euripides Loukis, Associate Professor

2. INTRODUCTION
e-Science: cross border research collaboration and use of
huge high-capacity computing resources – tools for
collection, storage, analysis and modeling of data
large amount of data is very useful for conducting scientific
research in many areas
socio-economic benefits
understanding of social and economic problems, and also of the
effectiveness of various policies government agencies
implement for addressing them
opening up this resource could amount to about € 40 billion a
year in the EU – a new e-market governments start to invest on
5/10/20122 PCI2012 - University of the Aegean

3. PURPOSE
a systematic evaluation of these PSI e-Infrastructures, aiming
at a better understanding and assessment of value they
generate
a structured and comprehensive evaluation methodology is
missing
“ The proposed methodology includes initially the definition
of one value model for each stakeholder group, which
consists of the main dimensions of value the PSI e-
infrastructure generates for it, and also the connections
among them”
5/10/20123 PCI2012 - University of the Aegean

4. BACKGROUND
Scoping eInfrastructures
Stakeholders Data Acquisition Data Provision
Communicatio
n
Literature Review
IS Evaluation TAM IS Success Models E-Services
5/10/20124 PCI2012 - University of the Aegean

5. Research Streams Insights
IS Evaluation
IS’s offer various types of benefits, both financial and non-
financial, and also tangible and intangible ones, which differ
among the different types of IS
it is not possible to formulate one generic IS evaluation
method, which is applicable to all IS
a comprehensive methodology for evaluating a particular
type of IS should include evaluation of both its efficiency and
its effectiveness, taking into account its particular
characteristics, capabilities and objectives
5/10/20125 PCI2012 - University of the Aegean

6. Research Streams Insights
TAM
identify the characteristics and factors affecting the attitude towards
using an IS, the intention to use it and finally the extent of its actual
usage
perceived usefulness and perceived ease of use determine an
individual's intention to use a system with intention to use serving as a
mediator of actual system use
IS Success Models
IS evaluation should adopt a layered approach based on the above
interrelated IS success measures (information quality, system quality,
service quality, user satisfaction, actual use, perceived usefulness,
individual impact and organizational impact) and on the relations
among them
5/10/20126 PCI2012 - University of the Aegean

7. Research Streams Insights
e-Services Evaluation
frameworks that assess the quality of the capabilities that the
e-service provides to its users
frameworks that assess the support it provides to users for
performing various tasks and achieving various objectives, or
users’ overall satisfaction
the above frameworks do not include advanced ways of
processing the evaluation data collected from the users, in
order to maximize the extraction of value-related knowledge
from them
5/10/20127 PCI2012 - University of the Aegean

8. Our Approach based on Value
Models
Ease of Use Experience
Performance
Data Search &
Download Capabilities
Data Provision
Capabilities
Accompl. Users
Objectives
Use
Future
Behaviour
Users’ Data Analysis
Capabilities
Efficiency Level
Effectiveness
Level
Fut. Behavior
Level
Data Users
5/10/20128 PCI2012 - University of the Aegean

9. Our Approach based on Value
Models
Efficiency Level
Effectiveness
Level
Fut. Behavior
Level
5/10/20129 PCI2012 - University of the Aegean
Data Providers
Ease of Use Experience
Performance
Providers’ Data Analysis
Capabilities
Data Upload
Capabilities
Accompl. Providers
Objectives
Use
Future
Behaviour

10. Value Measures
The total of 51 value measures were defined
15 common value measures
22 value measures for users
14 value measures for providers
These value measures was then converted to a
question to be included in questionnaires to be
distributed to stakeholders
A five point Likert scale is used to measure
agreement or disagreement
2 Questionnaires have been formulated
5/10/201210 PCI2012 - University of the Aegean

11. Value Model Estimation Algorithm
1. For each value dimension a composite variable is calculated as the average of
its individual measure variables.
2. Average ratings are calculated for all value dimensions (using the composite
variables calculated in step 1
3. For each value dimension of the first level we calculate its correlations with
all value dimensions of the second and the third levels (using again the
composite variables calculated in step 1).
4. Combination of 2 classes of analytics calculated in steps 2 and 3 for the
construction of a high-level value model of the PSI e-Infrastructure
5. First Layer Value Dimensions Classification into four groups:
 low rating – high impact
 low rating – low impact
 high rating – high impact
 high rating – low impact
1. Finally we repeat stages 2, 3, 4 and 5, but this time for the individual value
measures/variables instead of the value dimensions’ composite variables.
5/10/201211 PCI2012 - University of the Aegean

12. Conclusions
5/10/2012PCI2012 - University of the Aegean12
This paper has presented a methodology for evaluating an emerging class
of IS: the PSI e-Infrastructures.
These IS aim to support the evaluation of
government agencies for opening their data, in order to be used for
scientific, commercial and political purposes
various groups of users interested in them (e.g. scientists for conducting
research, active citizens and journalists for drawing conclusions on previous
government activity)
The proposed methodology assesses a wide range of types of value
generated by PSI e-Infrastructures for these two stakeholders’ groups
An algorithm for advanced processing of stakeholders’ evaluation data,
which results in the estimation of value models for these two groups and
the identification of improvement priorities

13. References
5/10/2012PCI2012 - University of the Aegean13
 Commission of the European Communities (2009),
“Communication from the Commission to the European
Parliament, the Council, the European Economic and
Social Committee and the Committee of the Regions – ICT
Infrastructures for e-Science”, COM (2009) 108 Final,
Brussels.
 Commission of the European Communities (2011),
“Communication from the Commission to the European
Parliament, the Council, the European Economic and
Social Committee and the Committee of the Regions –
Open data: An engine for innovation, growth and
transparent governance”, COM (2011) 882 Final, Brussels.
 Loukis, E. Pazalos, K. Salagara, A. (2012), “Transforming
e-services evaluation data into business analytics using value
models”, Elsevier, Electronic Commerce Research and
Applications 11 (2012), 129–141.
 Makx Dekkers, Femke Polman, Robbin te Velde and Mark
de Vries, MEPSIR – Measuring European Public Sector
Information Resources: Final report of study on
exploitation of public sector information – benchmarking
of EU framework Conditions, report for the European
Commission, June 2006
 Kim, Y. and Crowston, K. (2011), Technology adoption
and use theory review for studying scientists' continued use
of cyber-infrastructure. Proc. Am. Soc. Info. Sci. Tech.,
48: 1–10. doi: 10.1002/meet.2011.14504801197
 Stockdale, R., and Standing, C. (2006), “An interpretive
approach for interpreting information systems: a content,
context, process framework”. European Journal of
Operational Research, 173, , 1090–1102.
 Irani, Z., and Love, P., (2008), “Information systems
evaluation – a crisis of understanding” In Z. Irani and P.
Love (eds.), “Evaluating Information Systems – Public and
Private Sector”, Butterworth-Heinemann, UK.
 Schepers, J. and Wetzels, M. (2007). A meta-analysis of
the technology acceptance model: Investigating subjective
norm and moderation effects. Information & Management,
44, pp. 90-103.
 Fishbein, M., & Ajzen, I. (1975). Belief, Attitude,
Intention, and Behavior. Reading, MA: Addison-Wesley.
 Lu, J., and Zhang, G. (2003), “Cost benefit factor analysis
in e-services”, International Journal of Industry Service
Management, 14, 5, 570–595.
 Fassnacht, M., and Koese, I. (2006), “Quality of electronic
servic
 DeLone, D.H., McLean, E. R. (2003), “The DeLone and
McLean model of information systems success: a ten-year
update”, Journal of Management Information Systems, 19,
4, , 9–30

14. Thank you for your attention
5/10/2012PCI2012 - University of the Aegean14