* Linking research to the real world * Adding real-world value and impact

1. Developing a World Leading Technology
Enabled Health Programme of Research
Linking research to the real world
Adding real-world value and impact
Maged N. Kamel Boulos, MBBCh, PhD, SMIEEE
Professor and Chair of Digital Health
The Alexander Graham Bell Centre for Digital Health
University of the Highlands and Islands
Scotland, UK

2. The Vision: P5 Health
• P5-Health-compliant (e-)interventions are: Precise, Predictive, Preventive,
Personalised and Participatory (person-centred, flexible and user
negotiable).
• Also: Sustainable and scalable in non-technical environments (the real world).
• ‘Digital’ is obviously a key enabler of P5 Health, BUT…

3. The Problem (1): The ‘mHealth App Glut’
• Many new digital health start-
ups are failing to thrive, and the
market is suffering from what
has been described as the
‘mHealth app glut’: too many
(mismatched) apps and online
services (supply) vs. lesser or
declining user interest/
acceptance (demand).
• This is not because of a real
market saturation, but because
of a deep supply-demand
mismatch. The Digital Health Bubble

4. The Problem (2): Declining User Interest
• Tossed in a drawer: About a third of owners of ‘smart wearables’,
including the likes of Fitbit, Jawbone and health tracking
smartwatches, ditch their devices within six months of purchasing
them. (Endeavour Partners, 2014)

5. The Problem (3): The Supply-Demand Mismatch
• We have many apps and services on the market that we don’t need or
which are not yet ready or fully optimised for mainstream adoption due
to issues related to (among others):
• Cost and affordability;
• Consumers’ digital and health literacy needs,
and other users’ characteristics (have implica-
tions on design); and
• Interface and integration (seamless without medical or technical
errors): with clinicians’ workflows and organisational practices, and
with other health/care apps and digital services.
• Symptoms (manifest as declining interest in current offerings = declining
demand): Poor user acceptance/uptake and compliance/adherence (be
it lay users/patients or healthcare professionals/clinicians).
Stock photo

6. The Problem (4): Reasons Behind Mismatch
• False and misguided assumptions made by digital health researchers,
designers/developers and engineers, businesses, managers/policy
makers and politicians.
• Doing what they think (in their own mind) would be best or what they
personally like most: ‘designer’s blindness’ has been described in the
literature.
• Ignoring or not properly collecting or interpreting evidence.
• Late involvement of target users (late engagement with the real
world and stakeholders), just to discover failure and lack of
acceptance, which can be very costly at this point.
Just a couple of news headline examples reporting recent research

7. The Problem (5): Learning from Past Failures
• Wisdom learned from the history of digital health failures in the UK and
elsewhere tells us this ‘blindness to the real world’ was often the main
reason for failure, despite good intentions originally; for example, the very
famous NHS National Programme for IT—NPfIT failure, in which a top-
down (‘political’) approach was dictated and adopted, and stakeholder
consultations were largely inadequate or ignored.
• NPfIT cost ~£12 billion and was dismantled in
2011.

8. The Solution (1): Approach and Methods
• Bring stakeholders together (continuous stakeholder
consultation and engagement):
• Lay public, patients, carers, community leaders (various age,
demographic, socio-economic and clinical groups,
as appropriate in each case);
• Clinicians, health/care professionals, health psychologists;
• Health policy makers and managers, health organisations/
healthcare providers, such as the NHS in the UK;
• The business sector/industry/SMEs and SMBs (small and
medium-sized enterprises and businesses), funding
agencies; and
• Technology experts: designers, developers, engineers
and researchers (academic and industry R&D); legal, ethics,
data protection and privacy experts; etc.
• Broker between-stakeholder/between-stakeholder-
groups knowledge transfer.
Photo: AGBC for Digital Health, Elgin

9. Conceived as a
Or catalyst, bringing together
• Academia/Research
• Healthcare providers
(NHS), social care
• Digital health
industry/SMEs and start-
ups (incubators)
• The public, patients,
carers, etc. Photo by Maged N. Kamel Boulos

10. The Solution (2): Approach and Methods
Within the aforementioned stakeholder partnership:
• Establish effective mechanisms to promptly address any arising
IPR issues.
• Establish effective mechanisms to address all applic-
able regulatory compliance and certification issues:*
• Many much needed clinical-grade digital solutions are never developed by the
industry (particularly SMEs) and other less needed and/or ill-founded health and
fitness apps are produced instead – an ‘avoidance tactic’ that attempts to make or
maximise profit while escaping the tedious and costly regulatory compliance and
certification issues that apply to ‘apps or software as a medical device’.
• ‘Patient-centred design’ and appropriate involvement of end users in app and
device design are now mandatory requirements, explicitly stated in the US FDA
regulation 21 CFR 820.30.
* Kamel Boulos MN, Brewer AC, Karimkhani C, Buller DB, Dellavalle RP. Mobile medical and health apps: state of the art, concerns, regulatory control
and certification. Online Journal of Public Health Informatics. 2014;5(3):229. doi:10.5210/ojphi.v5i3.4814.
UK Medicines & Healthcare Products Regulatory Agency (MHRA)

11. The Solution (3): Approach and Methods
Within the stakeholder partnership:
• Adopt a Google-style (or inspired) iterative agile design, development
and continuous evaluation approach - fast, lean/efficient and cost-
effective, while trying not to compromise quality or clinical safety.
Google Design Sprint: https://developers.google.com/design-sprint/
• Early and continuous involvement of target users
(not just in a dedicated evaluation study at the end)
using fast prototyping and mockup tools.
https://popapp.in/
Co-creation and co-
production

12. The Solution (4): Approach and Methods
• For digital solutions involving ‘big data’ (e.g., remote health monitoring), remember to
pair those data with traditional forms of information collection, or what some
researchers call ‘small data’ for best results.* In each case, find and go for the ‘right-
sized data’ rather than the ‘big(gest) data’.
• Avoid copycat drug-trial-style evaluations:** drugs and digital solutions are different
beasts, requiring different evaluation and impact assessment instruments (a lesson
learned from the UK DH Whole Systems Demonstrator, the world’s largest
telehealthcare RCT, 2008-2010).
Digital interventions are best served by a dynamic, lean and pragmatic approach to
evaluation – cf. Living Systematic Reviews.*** When conventional approaches are
used, by the time we get some evaluation results published, a new, supposedly better
technology or version has often already replaced the evaluated solution on the market.
* Lazer D, Kennedy R, King G, Vespignani A, Lazer D1, Kennedy R, King G, Vespignani A. The parable of google flu: traps in big data analysis. Science.
2014;343(6176):1203–1205. doi: 10.1126/science.1248506. doi:10.1126/science.1248506
** Kamel Boulos MN, Maramba I. Pitfalls in 3-D Virtual Worlds Health Project Evaluations: The Trap of Drug-trial-style Media Comparative Studies. Journal For Virtual
Worlds Research. 2009;2(2). https://journals.tdl.org/jvwr/index.php/jvwr/article/view/669
*** Elliott JH, Turner T, Clavisi O, Thomas J, Higgins JP, Mavergames C, Gruen RL. Living systematic reviews: an emerging opportunity to narrow the evidence-practice
gap. PLoS Med. 2014 Feb 18;11(2):e1001603. doi: 10.1371/journal.pmed.1001603. eCollection 2014.

13. The Solution (5): Approach and Methods
Traps in comparative evaluation studies:
• Digital solutions can often co-exist in a synergistic, complementary fashion
rather than being mutually exclusive.
• Digital solutions vs. conventional healthcare: Optimised face-to-face and conventional healthcare is
rather excellent today in many places, but not very sustainable due to healthcare professional staff
shortages vs. increasing demands (e.g., ageing populations). e-interventions are not necessarily
less costly (they can indeed be more costly), but might require fewer healthcare professionals to
run (maximising staff efficiencies), and could thus be more sustainable as a mainstream model for
future healthcare. We still need to evaluate their clinical safety/reliability and effectiveness, both
of which need not be better than optimised traditional healthcare, but of course any
improvements over conventional care would always be very welcome. We also need to adapt
workflows to suit the new e-interventions (and vice-versa) plus a few other key ingredients, to
ensure user (clinician and patient) acceptance.
So it is not a question of which is clinically better or less costly, but one related to long term
sustainability (while ensuring safety and effectiveness). e-interventions require and create other
types of (non-clinician) jobs and this should also be considered.
Stock photo

14. Developing a World Leading Technology
Enabled Health Programme of Research
Linking research to the real world
Adding real-world value and impact
Focus on, and fully engage, good
representative samples of all users and
stakeholders throughout service or product
lifecycle from conception to retirement.

15. Thank you!
mnkboulos@ieee.org -