A rapid overview of a methodology to measure the qualitative and financial value of ICT projects in hospitals. This methodology was designed in a collaborative way by PMO of 4 swiss french speaking hospitals by adaptation of existing methods

1. How to measure the value of the ICT projects:
the HUG methodology
B. Debande, MD, MHA
CIO
University hospital of Geneva
Benoit.Debande@hcuge.ch

2. Agenda
 Reason for project value measurement
 Collaborative design of a methodology
 Overview of the tools
 Is it used in « real life » ?
 Future developments
 Conclusions

3. Reason for project value measurement
 IT support nearly all hospital activities
 Critical for operations : without IT just close the hospital doors !
 Mandatory for evolution and growth : each service needs IT to
support/optimize their working processes, better billing,…
 There are many « bosses » in an hospital
 CEO
 CMO
 CNO
 and each head of medical service
 IT resources are not following the demand
 Budgets nearly flat
 C* view : IT is expensive but dont give us a financial return to fund it

4. Reason for project value measurement
Multiple and contradictory constraints on the CIO’s head…
To survive, we need to prioritize projects

5. Reason for project value measurement
 What is mandatory to prioritize projects
 Governance : many things to say about that….. next time !
 Decision support tools
 Which tools
 Portfolio : mandatory but not easy to set up : also many thing to say
about that….. next time !
 Indicators to support the governance decision process : a tool to
help not a tool to choose in place of governance !
 An alternative to ROI
 In healthcare, ROI = financial = the devil !
 We need to measure the return both on quality AND on investment

6. Collaborative design of a methodology
There are more ideas into two heads than one
Don’t reinvent the wheel
Keep it simple to be sure the tool is really used
 There are more ideas into two heads than one
 In 2011, we started a collaborative work with PMO of 3 other swiss
french speaking public hospital ( 2 general hospitals and 2
university hospitals), leaded by HUG PMO
 Don’t reinvent the wheel
 Adapt 2 existing methods in the hospital context
 Keep it simple
 First time we ask our project managers to evaluate their projects
 All projects managers are NOT professionnal PM
 We must have only ONE methodology adopted by EVERYONE

7. Collaborative design of a methodology
 Utilitas
 Used for evaluation of swiss e-government projects
 Easy to understand, graphical view of results
 Needs to be adapted to hospital context
 Avalis
 Designed for the evaluation of healthcare projects by ANAP
(Agence Nationale d'Appui à la Performance des établissements de santé et médico-
sociaux)
 Too rich and too complex (115 multiple choice questions), won’t be
adopted by our project managers
 Suitable for our needs but must be simplified
 The group of PMO’s worked nearly 6 month to merge, simplify and test
on the field the new methodology

8. Overview of the tools
 An excel file with 38 multiple choice questions, a tool to calculate ROI
and some macro to produce graphical results
 All multiple choice questions inspired by ANAP questionnaire
 Needs (5 questions)
• Strategic alignment
• Consequences if we don’t start the project
 Benefits (21 questions)
• Impact on quality of care
• Impact on working processes, efficiency
 Risks (12 questions)
• Do we have all condition of success ?
• Are there already potential blocking points ?
 Cost effectiveness – ROI (8 questions)
• ROI calculated on 4 years, including all costs and revenues

9. Overview of the tools
 Utility score : calculation and graphical representation coming form Utilitas
 qualitative utility (needs + benefits) vs cost effectiveness

10. Overview of the tools

11. Overview of the tools

12. Overview of the tools
 Graphical view of the results, with a special focus on « benefits »
Quality of
service
Efficiency (ops
and economic)
Working
condition
Impact
Benefits for
extra-hosp
Acquired skills
0
1
2
3
4
Needs
Benefits
Risks
Cost
effectiveness
UTILITAS Result A2

13. Is it used in « real life » ?
 HUG
 First try of the method on 20 running projects
• Positive return of projects managers (IT and non IT)
• More difficulties on economic evaluation, even with the tool
 Since 2013, each new project must evaluate his utility
• Graphic view and Utilitas score are mandatory for decision
process
 But, till now, there is an evidence that governance doesn’t use the
utility to prioritize the project
• Lack of maturity of our governance concil (less than 1 year in
this configuration)
• Politic reasons vs real needs in the prioritization process….
 Other hospitals
 Method and tool validated
 But not really used because lack of real governance process

14. Future developments
 Continuous improvement
 Less multiple choice questions : from 38 to 29
• Easier to fill but risk of being non discriminant (to be evaluated
and improved if necessary)
 Include real estate assets and medical equipment in questions
about benefits
 Risk assessment not only at the beginning of the projet but
evaluated during all the project cycle
 Extension of utility measurement
 By consistency with the global project methodology (Hermes)
 Use the utility measurement for all HUG, project IT and non IT

15. Conclusions
 Our utility measurement methodology is the result of a collaborative
work of PMO from four swiss french speaking hospitals
 Inspired by Utilitas and Avalis methodologies and adapted for
healthcare, the usability of the tool has been validated by professionnal
and non professional hospital project managers
 Altough further work is needed to refine and extend the scope of
methodology, it is already used since 2 years in HUG
 Next challenge is to convince governance to take in account these
objective data to prioritize project portfolio !
Interested to learn more about methodology ?
Interested to participate in the further development ?
Join us : David.Bandon@hcuge.ch