3. 738 i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771
effectiveness of telemedicine with particular reference to 3.1.3. Comparisons
both outcomes and methodologies for evaluation. This paper Reviews of studies comparing telemedicine to standard care or
focuses mainly on the evidence about effectiveness, and to another type of care, as well as reviews of studies comparing
assesses the range of conclusions drawn by reviewers about different e-health solutions were included.
the effectiveness of telemedicine and the gaps in the evidence
base. A companion paper focuses on the methodological
issues and recommendations [13]. 3.1.4. Outcomes
Only reviews reporting relevant outcomes were included,
specified as health related outcomes (morbidity, mortality,
quality of life, patient’ satisfaction), process outcomes (qual-
2. Objectives
ity of care, professional practice, adherence to recommended
practice, professional satisfaction) and costs or resource use.
The objective of the work was to conduct a review of reviews Systematic reviews reporting emerging issues, such as an
on the impacts and costs of telemedicine services and con- unexpected finding or important new insights were also
sider qualitative and quantitative results, with the purpose included.
of synthesizing evidence to date on the effectiveness of
telemedicine. The key questions addressed were firstly, how
are telemedicine services defined and described in terms of 3.1.5. Languages
participants, interventions, comparisons and outcome mea- No articles were excluded based on language, although the
sures; secondly, what are the reported effects of telemedicine: main focus of the project was telemedicine in Europe.
thirdly which methodologies were used to produce knowl-
edge about telemedicine in studies included; fourthly, what
3.2. Exclusion criteria
are the strengths and weaknesses of these methodologies,
including HTA methodologies; and finally what are the knowl-
3.2.1. Design
edge gaps and what methodologies can be recommended for
Reviews considered not systematic, including commentaries
future research? The present paper addresses the first two of
and editorials, were excluded. Systematic reviews with major
these questions, and identifies assessments of the evidence
limitations (low quality reviews) according to a revised check-
base provided within the reviews and knowledge gaps in terms
list for systematic reviews from EPOC (Cochrane Effective
of outcomes.
Practice and Organisation of Care Group) were excluded.
If the same authors had produced several publications of
the same review, the most updated and/or the full report of
3. Methods the review was selected, and other versions excluded. Disser-
tations, symposium proceedings, and irretrievable documents
An initial search identified systematic reviews of telemedicine
were excluded.
published from 1998. A systematic review was defined as an
overview with an explicit question and a method section with
a clear description of the search strategy and the methods 3.2.2. Participants
used to produce the systematic review. The review should Studies with participants considered not relevant for the
also report and analyse empirical data. In addition, reviews review, for instance studies on use of ICT on people outside
which described or summarised methods used in assessing health care were excluded. Animal studies were excluded.
telemedicine were included. Because of the large number of
reviews retrieved, a decision was taken to include only reviews
published from 2005 and onwards in the final review. 3.2.3. Interventions considered not relevant for the review
Other exclusions were studies on interventions considered not
relevant for the review, such as studies on Internet and other
3.1. Inclusion criteria ICT media used for information seeking; quality of informa-
tion on the Internet; Internet based education of students and
3.1.1. Population/participants health professionals, including use of games; medical tech-
Systematic reviews on patients and consumers, health pro- nology in clinical practice in general, i.e. medical and surgical
fessionals and family caregivers, regardless of diagnoses or examinations and treatments based on computer technolo-
conditions, were included in the searches for systematic gies, except when used as remote diagnosis and treatment
reviews. (telehealth); ordinary use of electronic patient records; use of
telephone (including cell phones) only; e-health as only a very
limited part of an intervention; use of Internet for surveys and
3.1.2. Interventions research; online prescriptions; mass media interventions and
All e-health interventions, information and communication veterinary medicine.
technologies (ICT) for communication in health care, Internet
based interventions for diagnosis and treatments, and social
care if an important part of health care and in collaboration 3.2.4. Outcomes
with health care for patients with chronic conditions were Articles without relevant outcomes, i.e. not on the list of out-
considered relevant. comes specified above under inclusion criteria, were excluded.
4. i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771 739
3.3. Information sources to which the systematic reviewers had assessed risk of bias in
individual studies.
Literature searches of the following databases: ACM Digital Systematic reviews with major limitations were excluded.
Library (ACM – The Association for Computing Machin- We assessed the methodological quality of studies in the
ery), British Nursing Index, Cochrane library (including field of telemedicine based on the review authors’ assess-
Cochrane database of systematic reviews (CDSR), Database ments of risk of bias in the primary studies they had
of reviews of effects (DARE), Health Technology Assess- included.
ment Database (HTA), CSA, Ovid Medline, Embase, Health
Services/Technology Assessment Text (HSTAT), Interna- 3.9. Summary measures and synthesis of results
tional Network of Agencies for Health Technology Assess-
ment (INAHTA), PsycInfo, Pubmed, Telemedicine Information The authors analysed the data collected by the members of
Exchange (TIE), Web of Science. the expert team. Due to the expected heterogeneity of stud-
The main search was performed in February 2009, and an ies, regarding participants, interventions, outcomes and study
updated search was performed in July 2009. designs, a quantitative summary measure of the results was
not planned. We did a qualitative and narrative summary
3.4. Search of the results of the systematic reviews. The results of the
literature review were presented and discussed in two work-
The search strategies are available on the website: (to be shops intending to validate results. In the first workshop
inserted). different user groups took part and in the second workshop
methodology experts participated. The analysis was inspired
3.5. Study selection by principles of realist review [14], considered appropriate for
complex interventions.
Based on the criteria for inclusion and exclusion, AGE and SF
independently screened the lists of titles/abstracts identified
4. Results
through searches for systematic reviews. Any discrepancies
were solved by discussion with the third member of the team,
We identified 1593 records through the searches and excluded
AB. The potentially relevant systematic reviews were retrieved
1419 following screening. We retrieved 174 potentially rel-
in full text.
evant articles in full text. We excluded 94 of these based
on the pre-specified inclusion and exclusion criteria. The
3.6. Data collection process
qualitative synthesis below relate to 73 of the 80 included
articles.
Data collection was carried out online using a data extrac-
The results of the 80 systematic reviews included are sum-
tion form. Each potentially relevant systematic review was
marised in seven tables in Appendix 1. Tables one through
assessed in full text by one member of an expert panel of
six list populations, interventions, outcomes, results and con-
reviewers. A revised check list from EPOC (Cochrane Effective
clusions for the reviews cited in this paper, according to the
Practice and Organisation of Care Group) was used to assess
headlines presented in the discussion below. Table 7 list the
the quality of the systematic reviews. The quality domains
seven included reviews not cited in this paper.
assessed according to this checklist were methods used to
identify, include and critically appraise the studies in the
review, methods used to analyse the findings and an overall 5. Telemedicine is effective
assessment of the quality of the review. The review team (AGE,
AB and SF) subsequently checked review reports for agreement Twenty reviews (Table 1) concluded that telemedicine works
regarding the inclusion and exclusion criteria. and has positive effects. These include therapeutic effects,
increased efficiencies in the health services, and technical
3.7. Data items usability.
Types of interventions that were found to be therapeuti-
Data on type of participants, interventions and outcomes cally effective include online psychological interventions [15];
included in the reviews were collected. Other data items were: programmes for chronic heart failure that include remote
geographical coverage of review, time frame of included stud- monitoring [16]; home telemonitoring of respiratory con-
ies, range of data collection methods used in studies included ditions [17]; web and computer-based smoking cessation
in the reviews, disciplines/areas covered and methodologi- programmes [18]; telehealth approaches to secondary preven-
cal traditions included in the review. The reviewers were also tion of coronary heart disease [19]; telepsychiatry [20]; virtual
asked to indicate emerging issues identified by the authors of reality exposure therapy (VRET) for anxiety disorders [21];
the reviews. robot-aided therapy of the proximal upper limb [22]; inter-
net and computer-based cognitive behavioural therapy for the
3.8. Quality of systematic reviews and risk of bias in treatment of anxiety [23,24]; home telehealth for diabetes,
individual studies heart disease and chronic obstructive pulmonary disease [25];
and internet based physical activity interventions [26]. A
The members of the expert team assessed the quality of the review comparing telepsychiatry and face-to-face work [27]
systematic reviews, including questions regarding the degree found no differences between the two, and suggested that
5. 740 i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771
telepsychiatry will increase in use, particularly where it is are heterogeneous and interventions complex, making these
more practical. difficult to understand [49].
Interventions that are effective in reducing health service
use include vital signs monitoring at home with telephone
follow-up by nurses [28]; computerised asthma patient educa- 7. Evidence is limited and inconsistent
tion programs [29]; and home monitoring of diabetes patients
[30]. Twenty-two reviews (Table 3) however concluded that the evi-
Technical effectiveness and reliability are reported in dence for the effectiveness of telemedicine is still limited and
respect of remote interpretation of patient data [31]; smart inconsistent, across a wide range of fields.
home technologies [32]; and home monitoring of heart failure In terms of therapeutic effectiveness, there is some lim-
patients [33]. ited evidence regarding telemonitoring for heart failure [50];
One review concluded that home based ICT interventions despite reviewers suggesting that electronic transfer of self-
in general give comprehensive positive outcomes for chronic monitored results has been found to be feasible and acceptable
disease management, despite only identifying a small number in diabetes care, they find only weak evidence for improve-
of heterogeneous studies [34]. ments in HbA1c or other aspects of diabetes management [51];
others found only weak evidence of benefit relating to infor-
matics applications in asthma care [52]; and no evidence of
6. Telemedicine is promising improvement in clinical outcomes following teleconsultation
and video-conferences in diabetes care [53].
Nineteen reviews (Table 2) were less confident about the effec- Frequently, these reviewers call for further research,
tiveness of telemedicine, suggesting that it is promising, or has notably in the form of RCTs. Examples include calls relating
potential, but that more research is required before it is pos- to web-based alcohol cessation interventions [54]; and vir-
sible to draw firm conclusions. In some cases, in which the tual reality in stroke therapy, despite this being found [37] to
same conditions and interventions are discussed, these more be ‘potentially exciting and safe’. More work on telemonitor-
tentative conclusions must temper those of authors who find ing in heart failure is called for [55]; on e-therapy for mental
conclusive evidence. health problems [56]; on smart home technologies [57]; and
One review [35] for example found internet-delivered CBT on technological support for carers of people with demen-
to be a ‘promising’ and ‘complementary’ development, but did tia [58]. Others [28] underlined that lack of evidence does not
not provide the endorsements that others [23,24] did for CBT imply lack of effectiveness, and that in many cases interven-
for the more specific conditions of anxiety and depression. tions are simply ‘unproven’. Caution is also urged by reviewers
Similarly psychotherapy using remote communication tech- [59] who identified small numbers of heterogeneous studies
nologies was seen as promising [36], but still requiring more in relation to chronic disease management. One review [60]
evidence. found it impossible to draw any significant conclusions about
Areas in which review authors agreed that telemedicine the impact of interventions to promote ICT use by health care
shows therapeutic promise, but still requires further research, personnel.
include virtual reality in stroke rehabilitation [37,38]; improv- Several reviewers found that research has been somewhat
ing symptoms and behaviour associated with and knowledge narrowly focused and suggested further research which takes
about specific mental disorders and related conditions [39]; a broader perspective or a different one. They suggested that
diabetes [40,83]; weight loss intervention and possibly weight telemedicine researchers have not yet asked all the impor-
loss maintenance [41]; and alcohol abuse [88]. tant questions, or conducted research in appropriate ways.
Other authors found promise in terms of health service For example, in the cases of dermatology, wound care and
utilisation. One review [42] for example suggested that asyn- ophthalmology, it was argued that evaluation has explored
chronous telehealth developments could result in shorter ICT-based asynchronous services for efficacy, but outcomes
waiting times, fewer unnecessary referrals, high levels of or access issues have not been considered [61]. In a simi-
patient and provider satisfaction, and equivalent (or better) lar vein, although most of the studies of smart homes found
diagnostic accuracy. Another [43] found that home telehealth technical feasibility, there remain certain topics that require
has a positive impact on the use of many health services as further research, notably, ‘technical, ethical, legal, clinical,
well as glycaemic control of patients with diabetes. economical and organisational implications and challenges’
Positive patient experiences were highlighted as promising [32]. Others [44], whilst seeing significant potential for teleon-
in relation to home telemonitoring for respiratory conditions cology, especially in rural areas, suggested that local studies
[17]. There is potential for using Internet/web-based services may be needed to confirm this. A further contribution to the
for cancer patients in rural areas [44], and telemonitoring can debates about CBT (see above), found that whilst it appears to
empower patients with chronic conditions [45]. be effective for panic disorders, social phobia and depression,
Promising impacts on service delivery were identified its effects on obsessive–compulsive disorder and anxiety and
[46,47] in use of electronic decision support systems and depression combined remain insufficiently clear [62]. Causal
telemedicine consultations promise to support improved pathways in HbA1c decline in diabetes care remain unclear,
delivery of tPA in patients with stroke (a treatment which and this conclusion can be linked with the variations in pro-
requires to be administered within 3 h) [48]. Computer gramme designs [63]. Whilst smoking cessation programmes
reminders to professionals at the point of care show ‘small to appear to be effective across a range of studies, nevertheless
modest improvements’ in professional behaviour, but studies the mechanisms of action are not well understood [64].
6. i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771 741
Telemedicine is a dynamic field, and new studies and One review found that health service users with ICT
new systematic reviews are rapidly being published. As used in support, education and virtual consultation feel
telemedicine extends into new clinical areas, it is unsurpris- more confident and empowered, with better knowledge and
ing that reviewers give renewed accounts of limited evidence. improved health outcomes, as well as experiencing better
Some examples of new areas from our review include little nurse-patient relationships [73]. The reviewers call for more
research on health promotion provided through the Internet research on the mechanisms for these changes. Generally
[65]; a Cochrane review that found no studies of smart homes there is evidence of high patient satisfaction ratings for telere-
that met their inclusion criteria [57]; a review of studies on habilitation, but reviewers argue that more process research,
spiritual care that found little systematic research in this area case studies and qualitative studies are needed to improve
[66]; and a review concluding that formative evaluation is our understanding of these outcomes [74]. Interactive health
needed for remote monitoring in hypertension [90]. communication applications (IHCAs) for people with chronic
disease appear to give benefit in terms of improved sup-
port, better knowledge and improved health outcomes, but
8. Economic analysis the authors asked for more larger studies to be conducted
[75].
An important emerging issue from our review is the lack of Others found no consistent results regarding user expe-
knowledge and understanding of the costs of telemedicine riences, though suggested that access can be improved [69].
(Table 4). Alongside development of technologies which aim to ben-
Several reviewers suggested that telemedicine seemed to efit patients and citizens as well as professionals, we need
be cost-effective, but few draw firm conclusions. One review research on the impacts of technologies for these groups [76].
found that 91% of the studies showed telehomecare to be cost- An example is that information websites relating to dementia
effective, in that it reduced use of hospitals, improved patient are geared more to carers than to people with dementia them-
compliance, satisfaction and quality of life [67]. This was the selves, and that the websites do not usually offer personalised
clearest conclusion, with others being much more cautious: information [77].
telemedicine was found to be cost-effective for chronic disease
management, but the authors cautioned that studies were few
and heterogeneous [34]. A comparison of the costs of telemon- 10. Asking new questions
itoring and usual care for heart failure patients found that
telemonitoring could reduce travel time and hospital admis- We have already noted the emergence of new topic areas in
sions, whilst noting that benefits are likely to be realised in the this dynamic and complex field. The focus on patient bene-
long term [68]. Others found home telehealth for chronic con- fits however indicates a more basic development, namely that
ditions to be cost saving, though underlining that studies were reviewers are starting to explore new questions beyond those
generally of low quality [25]. One review found remote inter- of clinical and cost-effectiveness. Our review produced two
pretation in medical encounters to be more expensive than its key examples (Table 6). Firstly, a review that identified gender
alternatives [31]. differences in computer-mediated communications relating
Other reviewers did not find good evidence about cost- to online support groups for people with cancer cautioned that
effectiveness; the cost-effectiveness of home telecare for studies are limited and heterogeneous [78]. Nevertheless, the
older people and people with chronic conditions is uncer- authors suggested that this issue needs to be considered by
tain [28]; there is a lack of consistent results regarding costs those designing interventions of this kind. This implies a con-
of synchronous telehealth in primary care [69]; there is lit- sideration that telemedicine is an ongoing intervention where
tle evidence for the economic viability of home respiratory users influence its development and hence that effectiveness
monitoring [17]; the cost-effectiveness of IT in diabetes care of outcome is a complex collaborative achievement. Secondly,
is undetermined [40]; one review was able to identify only one a review focusing on stroke thrombolysis service configura-
study of the costs of CBT, with significant weaknesses [70], tions, their potential impact and ways of recording data to
with another finding little evidence in the same area [62]. inform which configuration could be most suitable for a partic-
A particular limitation identified in terms of costs concerns ular situation, highlighted the need to consider a wider range
the wider social and organisational costs of telemedicine. One of service delivery issues [79]. Similarly, it was argued that in
review found that a societal perspective on costs has not yet post-stroke patients, the consideration of caregivers’ mental
been developed for home telehealth [71] and another high- health and high levels of patient satisfaction should be an
lighted the need to consider not only costs to health services integral element of studies [80].
of interventions, but also costs to service users and their social Furthermore, some of the papers included in the review
networks [72]. explored issues which can inform the future development
of telemedicine, that is, they provide formative assessments.
Examples include a review of 104 definitions of telemedicine
9. Is telemedicine good for patients? [81] which, in identifying four broad types of definitions,
suggested how stakeholder interests can alter perceptions
A second emerging issue concerns patient satisfaction with of priorities in telemedicine interventions, such that some
telemedicine, and indications that telemedicine may alter may focus on delivering healthcare over a distance and
the relationships between patients and health professionals others on the potential of technology per se; and work argu-
(Table 5). ing that clinical and technical guidelines can inform the
7. 742 i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771
future development of telemedicine and facilitate evaluation
[20,82]. Summary points
“What was already known on this topic”
11. Reflections on the methodology of our • Evidence regarding the effectiveness of telemedicine
study is patchy and incomplete.
• The quality of much of the research conducted is poor.
Our study is a review of systematic reviews. There are some
inherent weaknesses in this approach. In general we have to “What this study added to our knowledge”
rely on the information in the included reviews. The quality
of the reviews may vary; the reviews may have done a poor • The evidence base is accumulating robust knowledge
job in specifying their inclusion and exclusion criteria, the about the effectiveness of telemedicine.
searches may not be comprehensive, the review authors may • As the field is rapidly evolving however, new knowl-
not have assessed or extracted data from the primary studies edge is constantly needed.
adequately, nor analysed and synthesised the findings across • Continuing areas of weakness but also of great interest
the studies properly. But even using high quality reviews, we include economic analyses, understandings of patient
necessarily lose information and details that we can only find perspectives, of effectiveness and outcomes as com-
if we go back to the primary studies. plex and ongoing collaborative achievements, and
Although we did a thorough job in developing the search formative assessments.
strategy and identified a vast amount of reviews on the effects
of telemedicine, we might have missed relevant systematic
reviews.
Some of the included reviews are probably outdated. Stud-
12. Conclusions
ies that are published after the search date in the reviews are
not included. Ideally we could have supplemented the review
Despite large number of studies and systematic reviews
with more recent primary studies not included in the reviews,
on the effects of telemedicine, high quality evidence to
but we did not have the resources to do this.
inform policy decisions on how best to use telemedicine
We did not check whether reviews included the same ref-
in health care is still lacking. Large studies with rigorous
erences. Several reviews have studied similar or overlapping
designs are needed to get better evidence on the effects of
topics, and have at least partially included the same studies.
telemedicine interventions on health, satisfaction with care
It may therefore be that evidence is counted twice, or that
and costs. As the field is rapidly evolving, different kinds
different interpretations of effectiveness are given by review
of knowledge are also in demand, e.g. a stronger focus on
authors. We have not analysed the degree to which there are
economic analyses of telemedicine, on patients’ perspec-
discrepancies in the analyses of similar studies, nor the rea-
tives and on the understanding of telemedicine as complex
sons for different interpretations of the same findings, for
development processes, and effectiveness and outcome as
instance did we not analyse the heterogeneity of the results
ongoing collaborative achievements. Hence formative assess-
among the reviews based on the quality of the reviews.
ments are also pointed out as an area of weakness and
The data collection and assessment of each included
interest.
review was accomplished by one external expert, while two
is considered to be optimal in order to reduce risk of bias. Acknowledgements
We did not train the data extractors, and we did not pilot
the data extraction form. The experts were not completely The study was funded by the EU under SMART 2008/0064
consistent in their judgments. This limitation was partly due and was conducted as part of the MethoTelemed project. We
to the resources and organisation of the project, in that two acknowledge the support of our MethoTelemed colleagues, the
workshops were held, intending to validate results. In addi- group of external review experts, the workshop participants,
tion, the review team made a quality check of the reviews by the project officers at the Norwegian centre for integrated care
comparing the reported data with information in the full text and telemedicine, and Ingrid Harboe at the Norwegian Knowl-
papers. Any unclear themes were discussed in the team to edge Centre for the Health Services, who did the literature
reach consensus. searches.
We have limited information regarding effect sizes and the
strength of evidence for the outcomes that we have studied.
We have however demonstrated that it is possible to make
Appendix 1.
such a large overview in quite a short time, involving both
methodology and content experts. We have used systematic In Tables 1–7, columns listing results and conclusions quote
methods in the literature searches and the assessment of the from the authors’ work. Where a review appears in more than
reviews, and we have excluded reviews of low methodological one table, this reflects the range of evidence produced. Full
quality. access to a searchable database of abstracts of items included
In combining rigorous and systematic methods with a in the review will be available on the MethoTelemed web-
pragmatic approach we have produced a relevant and rich site, which also includes guidance for evaluating telemedicine.
overview of the field. www.telemed.no/MethoTelemed.
8. Table 1 – Systematic reviews reporting that telemedicine is effective.
Reference Conditions Geographic Service/ Outcome Authors’ summary of results Authors’ conclusions
included area intervention
Barak et al. [15] Mental health Not stated Internet based Behavioural, Sixty-four studies included covering 94 Internet based intervention is as
psychotherapy Health, Percep- services. The overall mean weighted effect effective as face-to-face
tion/satisfaction, size was 0.53, similar to the average effect intervention.
Social size of traditional, face-to-face therapy.
Comparison between face-to-face and
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771
Internet intervention across 14 studies
showed no differences in effectiveness.
Clark et al. [16] Cardio- All countries Remote Behavioural, Fourteen studies (RCTs) included. Four Programmes for chronic heart
vascular monitoring, Cost/economic, evaluated telemonitoring, nine structured failure that include remote
(CHF) telephone Health telephone support, and one both. Remote monitoring have a positive effect
support monitoring programmes reduced the rates of on clinical outcomes in
admission to hospital for chronic heart community dwelling patients with
failure by 21% and all cause mortality by 20%. chronic heart failure.
Three studies reported quality of life
improvements and four, reduced cost, one
found no gain in cost-effectiveness.
Jaana et al. [17] Respiratory USA, Europe, Remote Behavioural, Twenty-three studies included. Good levels of Home telemonitoring of
conditions Israel, Taiwan monitoring Cost/economic, data validity and reliability were reported. respiratory conditions results in
Feasibility/pilot, However, little quantitative evidence exists early identification of
Health, Percep- about the effect of remote monitoring on deteriorations in patient condition
tion/satisfaction patient medical condition and utilization of and symptom control. Positive
health services. Positive effects on patient patient attitude and receptiveness
behaviour were consistently reported. Only of this approach are promising.
two studies performed a detailed cost However, evidence on the
analysis. magnitude of clinical and
structural effects remains
preliminary, with variations in
study approaches and an absence
of robust study designs and formal
evaluations.
Myung et al. Smoking Worldwide Web and Behavioural Twenty-two studies included (RCTs). In a The meta-analysis of RCTs
[18] cessation computer-based random-effects meta-analysis of all 22 trials, indicates that there is sufficient
programmes the intervention had a significant effect on clinical evidence to support the
smoking cessation. Similar findings were use of Web- and computer-based
observed in nine trials using a Web-based smoking cessation programs for
intervention,(and in 13 trials using a adult smokers.
computer-based intervention Subgroup
analyses revealed similar findings for
different levels of methodological rigor,
stand-alone versus supplemental
interventions, type of abstinence rates
employed, and duration of follow-up period,
but not for adolescent populations.
743
9. 744
Table 1 (Continued)
Reference Conditions Geographic Service/ Outcome Authors’ summary of results Authors’ conclusions
included area intervention
Neubeck et al. Cardio- USA (3 Communication Behavioural, Health, Eleven studies included (RCTs). Telehealth interventions provide
[19] vascular studies), using ICT, psychosocial state, Telehealth interventions were associated effective risk factor reduction and
(CHD) Norway (1), patient- quality of life with non-significant lower all-cause secondary prevention. Provision of
Canada (3), professional mortality than controls. These telehealth models could help
Australia (3), interventions showed a significantly increase uptake of a formal
Germany (1) lower weighted mean difference at secondary prevention by those
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771
medium long-term follow-up than who do not access cardiac
controls for total cholesterol, systolic rehabilitation and narrow the
blood pressure, and fewer smokers. current evidence-practice gap.
Significant favourable changes at
follow-up were also found in high-density
lipoprotein and low-density lipoprotein.
Pineau et al. Psychiatric Focus on Telepsychiatry Cost/economic, ‘About 60’ studies included. The authors The review concludes that
[20] conditions Canada and Ethical issues, Legal, argue that definition of clinical guidelines telepsychiatry should be
(adult and USA Organizational, and technological standards aimed at implemented in Québec and
paediatric) Technology related, standardising telepsychiatric practice will provides detailed clinical and
clinical guidelines promote its large scale implementation. technical guidelines for
and technical implementation. They add that
standards taking into account human and
organizational aspects plays a part
in ensuring the success of this
type of activity; that legal and
ethical aspects must also be
considered; and that a detailed
economic analysis should be
carried out prior to any large
investment in telepsychiatry.
Finally, implementation of
psychiatry should be subjected to
rigorous downstream assessment
in order to improve management
and performance.
Powers and Anxiety Not stated Virtual reality Behavioural, Percep- Thirteen studies included. VRET (Virtual Given the advantages and the
Emmelkamp (especially exposure tion/satisfaction, reality exposure therapy) is highly efficacy of VRET supported by this
[21] phobias) therapy Psychophysiology, effective in treating phobias and more so meta-analysis a broader
perceived control than inactive control conditions. VRET is application in clinical practice
over phobias slightly, but significantly more effective seems justified.
than exposure in vivo, the gold standard
in the field. Advantages of VRET: can be
conducted in the therapist’s office, rather
than in vivo situations, the possibility of
generating more gradual assignments
and of creating idiosyncratic exposure.
VRET is cost-effective.
10. Prange et al. Stroke USA Rehabilitation Health Eleven studies included. Robot-aided This systematic review indicates
[22] (robots) therapy of the proximal upper limb that robot-aided therapy of the
improves short and long-term motor proximal upper limb can improve
control of the paretic shoulder and elbow: short and long-term motor control
however, there is no consistent influence of the paretic shoulder and elbow.
on functional abilities. Robot-aided therapy appears to
improve motor control more than
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771
conventional therapy.
Reger and Mental health Not stated Internet/computer- Behavioural, Health Ninteen studies included (RCTs). The results of this meta-analysis
Gahm [23] problems based Meta-analysis showed that ICT was provide preliminary support for
(anxiety) treatment superior to waitlist and placebo the use of Internet and
assignment across outcome measures computer-based CBT for the
The effects of ICT were equal to treatment of anxiety.
therapist-delivered treatment across
anxiety disorders. Conclusions were
limited by small sample sizes, the rare
use of placebo controls, and other
methodological problems. The number of
available studies limited the opportunity
to conduct analyses by diagnostic group.
Spek et al. [24] Mental health Global CBT via internet Health Twelve studies included (RCTs). Authors Despite study limitations, eCBT
(depression concluded that eCBT was effective, but seemed to be effective.
and anxiety) noted that there was only a small number
of studies and significant heterogeneity.
Tran et al. [25] Diabetes, heart Canada Home telehealth Cost/economic, Seventy-nine studies included. Of the Conclusions relate to the potential
failure, COPD focused, but Health, Percep- included studies, 26 pertained to for home telehealth in Canada
and other international tion/satisfaction diabetes, 35 to CHF, nine to COPD, and which is seen as positive. However,
chronic publications eight to mixed chronic diseases. The more research, such as multicentre
diseases included comparator “no care” was not identified RCTs, is warranted to accurately
in any of the included studies, so usual measure the clinical and economic
care was used as the comparator impact of home telehealth for
throughout the clinical review. Home chronic disease management to
telehealth appeared generally clinically support Canadian policy makers in
effective and no patient adverse effects making informed decisions.
were reported. Evidence on health service
utilization was more limited, but
promising The economic review
suggested cost-effectiveness, but the
quality of studies was low.
van den Berg et Internet based Not Physical activity Behavioural, Health Ten studies included. The analysis There is indicative evidence that
al. [26] physical mentioned focused on the methodological quality of internet based physical activity
activity other than the studies, which showed variation in interventions are more effective
interventions language study populations and interventions than a waiting list strategy.
limitations making generalization difficult.
745
11. 746
Table 1 (Continued)
Reference Conditions Geographic Service/ Outcome Authors’ summary of results Authors’ conclusions
included area intervention
Hyler et al. [27] Mental health France, Telepsychiatry Feasibility/pilot, Per- Fourteen studies included. Telepsychiatry Only a handful studies have
Australia, ception/satisfaction, was found to be similar to In person for attempted to compare
Canada, Quality of different studies using objective assessments. telepsychiatry with in-person
Japan, UK instruments used Bandwidth was a moderator. psychiatry (IP) directly, using
and US for consultations Heterogeneous effect sizes for different standardised assessment
moderators (bandwidth) High bandwidth instruments to permit meaningful
was slightly superior for assessments comparison. According to the
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771
requiring detailed observation of patients. meta-analysis, there was no
difference in accuracy or
satisfaction between the two
modalities. Telepsychiatry is
expected to replace IP in certain
research and clinical situations.
Barlow et al. Elderly people, Worldwide Home telecare Behavioural, Health, Sixty-eight RCTs and 30 observational Having identified where there is
[28] chronic Organisational, studies with 80 or more participants evidence of effectiveness, and
diseases Safety included. Results show that the most where it is lacking, the authors
effective telecare interventions appear to conclude that insufficient
be automated vital signs monitoring (for evidence does not amount to lack
reducing health service use) and of effectiveness: more research is
telephone follow-up by nurses (for needed.
improving clinical indicators and
reducing health service use). Evidence on
cost-effectiveness is less clear, and on
safety and security alert systems
insufficient.
Bussey-Smith Asthma USA, Hawaii, Computer-based Behavioural, Nine studies included. One study each Although interactive CAPEPs may
and Rossen Sweden patient Cost/economic, showed reduced hospitalizations, acute improve patient asthma
[29] education Health, Percep- care visits, or rescue inhaler use. Two knowledge and symptoms, their
programmes tion/satisfaction, reported lung function improvements. effect on objective clinical
(CAPEPs) Social Four showed improved asthma outcomes is less consistent
knowledge, and five showed
improvements in symptoms.
Jaana et al.[30] Diabetes North Remote Behavioural, Health, Seventeen studies included. Most studies Positive effects are reported, but
America, monitoring Technology related, reported overall positive results in there is variation in patient
Europe and Structural Diabetes mellitus type 2, and found that characteristics (background,
Asia IT based interventions improved health ability, medical condition) sample
care utilisation, behaviour attitudes and selection and approach for
skills. treatment of control groups.
Azarmina and All All countries Remote Cost/economic, Nine studies included. Results showed The review suggests that remote
Wallace [31] interpretation in Feasibility/pilot, that time between encounters was interpretation is an acceptable and
medical Health, reduced, but evidence on consultation accurate alternative to traditional
encounters Organisational, Per- length was not consistent. Good client methods, despite the higher
ception/satisfaction, and doctor satisfaction was shown, but associated costs.
Safety those interpreting data preferred to do so
face to face. Costs of these interventions
are high, but efficiency gains are possible.
12. Demiris and Older people, Europe, USA, Smart home Behavioural, Health, Twenty-one projects included (drawing Most of the studies demonstrated
Hensel [32] people with Asia Safety, Social, on 114 publications). A table is presented the feasibility of the technological
disabilities Physiological and with their technologies, target audience, solution. Technical, ethical, legal,
i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771
functional technologies and different outcome. A clinical, economical and
lack of evidence on clinical outcomes is organisational implications and
identified. challenges need to be studied
in-depth for the field to grow
further.
Martinez et al. Heart failure All countries Remote Behavioural, Forty-two studies included. (1) Remote Evaluating the articles showed
[33] monitoring Cost/economic, monitoring for cardiac heart failure that home monitoring in patients
(home) Feasibility/pilot, appears to be technically effective for with heart failure is viable.
Health, Legal, following the patient remotely; (2) it
Organizational, Per- appears to be easy to use, and it is widely
ception/satisfaction, accepted by patients and health
Safety, Social, professionals; and (3) it appears to be
Technology related economically viable.
Gaikwad and Chronic Not stated Home based ICT Behavioural, Twenty-seven studies included. These Telecare, telehealth etc. have
Warren [34] disease interventions Cost/economic, systems can improve functional and positive clinical and cost outcomes
Health, Percep- cognitive patient outcomes in chronic – although studies are few in
tion/satisfaction disease and reduce costs. However, the number and heterogeneous. Better
research is not yet sufficiently robust. evidence-based outcome measures
are needed, especially regarding
costs and physician perspectives.
747
![i n t e r n a t i o n a l j o u r n a l o f m e d i c a l i n f o r m a t i c s 7 9 ( 2 0 1 0 ) 736–771 737
3. Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738
3.1. Inclusion criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738
3.1.1. Population/participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738
3.1.2. Interventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738
3.1.3. Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738
3.1.4. Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738
3.1.5. Languages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738
3.2. Exclusion criteria . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738
3.2.1. Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738
3.2.2. Participants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738
3.2.3. Interventions considered not relevant for the review . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738
3.2.4. Outcomes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 738
3.3. Information sources . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739
3.4. Search. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739
3.5. Study selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739
3.6. Data collection process . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739
3.7. Data items . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739
3.8. Quality of systematic reviews and risk of bias in individual studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739
3.9. Summary measures and synthesis of results. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739
4. Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739
5. Telemedicine is effective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 739
6. Telemedicine is promising . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 740
7. Evidence is limited and inconsistent . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 740
8. Economic analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 741
9. Is telemedicine good for patients? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 741
10. Asking new questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 741
11. Reflections on the methodology of our study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 742
12. Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 742
Acknowledgements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 742
Appendix 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 742
Appendix B. Supplementary data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 769
References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 769
study focused on cost-effectiveness interventions concluded
1. Introduction that there is no good evidence that telemedicine is or is not a
cost-effective means for delivering healthcare [5].
Previous reviews of telemedicine have concluded that
The quality of studies is a recurrent concern in these
irrefutable evidence regarding the positive impact of
reviews [1,2,4–6]. There is also a debate about appropriate
telemedicine on clinical outcomes still eludes us. One
research methodologies. For example, economic analysis of
review [1] of more than 150 articles concluded that poten-
telemedicine has not yet met accepted standards [5]; there is
tial effectiveness could only be attributed to teleradiology,
a relative lack of exploration of the socio-economic impact
telepsychiatry, transmission of echocardiographic images
of telemedicine [7]; evidence on factors promoting uptake of
and consultations between primary and secondary health
telemedicine is lacking [8]; there is relatively undeveloped use,
providers. Another systematic review [2] that assessed more
at the time, of qualitative methods [9]; many studies have not
than 1300 papers making claims about telemedicine out-
been well-designed [4,10]; and, considering perceived difficul-
comes found only 46 publications that actually studied at
ties of building a robust evidence base for recent innovations,
least some clinical outcomes. A review that analyzed the
researchers have argued that simulation modelling needs fur-
suitability of telemedicine as an alternative to face-to-face
ther development [11].
care [3] concluded that establishing systems for patient care
The lack of consensus raises questions about the quality
using telecommunications technologies is feasible; however,
of research evidence in terms not only of the data collected
the studies provided inconclusive results regarding clinical
and analysed, but also in terms of the approaches to evalua-
benefits and outcomes. A report on peer-reviewed litera-
tion, that is, the underlying methodologies used, which may
ture for telemedicine services that substituted face-to-face
not be capable of addressing the questions to which different
services with ICT-based services at home and in offices or
stakeholders seek answers. Others have noted that evaluation
hospitals [4] identified 97 articles that met the inclusion
traditions do not sufficiently collaborate to cross borders and
criteria for analysis. The authors concluded that telemedicine
that a common language for evaluation is missing [12].
is being used even if the use is not supported by high quality
This paper reports on research funded under EU SMART
evidence. Reviews on cost outcomes have fared similarly. A
2008/0064, which sought to review the evidence on the](data:image/gif;base64,R0lGODlhAQABAIAAAAAAAP///yH5BAEAAAAALAAAAAABAAEAAAIBRAA7)