The issue of open-source models in the cost-effectiveness and disease-level (collaborative) models has been brewing for many years. There has been a marked growth in open science, and funding bodies and publishers increasingly require that research data be made available. As mentioned in our previous Issue Panel, “cost-effectiveness models synthesise a wide range of evidence to facilitate extrapolation over time and from intermediate to final decision endpoints. These models are often statistically sophisticated and require assumptions that are not directly testable. This can lead to decision-makers “discounting” the results of cost-effectiveness analyses, particularly if the developer is seen as partial.” Open-source models, then, would encourage greater transparency in pharmacoeconomic modeling and the reuse and updating of the best/most useful models; they are essential if cost-effectiveness analyses are to be widely accepted to reduce bias, increase transparency, improve model access, and allow for faster access to critical knowledge. The ISPOR-SMDM guidelines and the EUnetHTA joint action projects, are supportive of these views on collaboration, transparency, confidentiality, processes and consistency offered by the availability of open-source models to improve decision-making around health care and reimbursement. With openness and sharing, however, come issues of copyright and access and a need to define how model sharing can be achieved in a fair and equitable manner. There is, therefore, a need to develop an ongoing dialog on openness, especially where the research may be considered precompetitive and not worthy of IP investment. The pros and cons of open source models and the proposed mission of the Open Source Model SIG to curate an ongoing dialog regarding issues around creating, disseminating, sharing, evaluating, and updating open source cost-effectiveness and comparative effectiveness models will be debated amongst SIG members. Author(s) and affiliation(s): Nancy Risebrough, Senior Principal, ICON plc, Toronto, Canada Jeroen P Jansen; Innovation & Value Initiative; Precision Medicine Group; and Stanford University Lotte Steuten, Vice President & Head of Consulting, Office of Health Economics, UK Renée JG Arnold, PharmD, RPh, ICON plc, New York, NY and Icahn School of Medicine at Mount Sinai, New York, NY, USA Event: ISPOR 2019 Annual Meeting Date: 20/05/2019

1. Lies, Damned Lies and Cost-
Effectiveness: Open-Source
Models, the SIG
Monday, May 20, 2019 12:30PM – 1:45PM, Rooms 391-392

2. Antitrust Compliance Statement
• ISPOR has a policy of strict compliance with both United States, and other
applicable international antitrust laws and regulations.
• Antitrust laws prohibit competitors from engaging in actions that could result
in an unreasonable restraint of trade.
• ISPOR members must avoid discussing certain topics when they are
together, including, prices, fees, rates, profit margins, or other terms or
conditions of sale.
• Members have an obligation to terminate any discussion, seek legal
counsel’s advice, or, if necessary, terminate any meeting if the discussion
might be construed to raise antitrust risks.
• The Antitrust policy is available on the ISPOR website, under “Policies &
Legal.”

3. Agenda
• SIG Mission Statement, Background, Goals, Rationale and Priority (10 minutes)
• Presentations ( 12 minutes each, 10 minutes discussion at end)
– Renée JG Arnold, PharmD, RPh, ICON plc, New York, NY and Icahn School of
Medicine at Mount Sinai, New York, NY, USA
– Lotte Steuten, PhD, MSc, The Office of Health Economics, London, United
Kingdom
– Jeroen P Jansen, PhD, Innovation & Value Initiative, Los Angeles, CA, USA
– Nancy Risebrough, BA, ICON plc, Toronto, ON, Canada
– Discussion and questions
• SIG key and member engagement projects (5 minutes)
– Interest in taking on immediate Book of Terms update
– Overall discussion and questions from Members

4. 10
Open Source Model SIG Working Mission Statement
• The mission of the Open Source Model SIG will be to curate an ongoing dialog
regarding issues around
• Creating
• Disseminating
• Sharing
• Evaluating
• Updating
• opensource cost-effectivenessand comparativeeffectivenessmodels.

5. Background
1

6. 6
Marked growth in open science requires that research
data be made available
• Private and public entities are already undertaking development
of platforms to build and/or display models for examination, but
this is being done in a vacuum that may be better served being
led or at least examined by the preeminent organization
dedicated to advancing HEOR science--ISPOR

7. Why Open Source Models?
• “Cost-effectiveness models
synthesise a wide range of evidence
to facilitate extrapolation over time
and from intermediate to final
decision endpoints. These models
are often statistically sophisticated
and require assumptions that are not
directly testable. This can lead to
decision-makers “discounting” the
results of cost-effectiveness
analyses, particularly if the developer
is seen as partial.”
Follow on to an Issue Panel, presented at ISPOR 2018, Baltimore, MD,
chaired by Neil Hawkins, MD, with Renée Arnold, PharmD and Jaime
Caro, MD as panelists, entitled Lies, damned lies and cost-
effectiveness: Open-source models are essential if cost-
effectiveness analyses are to be widely accepted

8. Open Model SIG Goals,
Rationale, and Priority
2

9. Goals
• Definethe issuessurroundingopensourcemodels
• Developpossiblesolutionsto these issues
• Have an ongoingdialog to discussthese issuesand advance thescience.
• Should ISPOR govern/curate a repository or other means of model representation to
ensure its longevity?
• If yes, should ISPOR or others assess its success?
9

10. Rationale and Priority: Impact of the SIG on HEOR
• Important to have accurate,vetted,timely models on which to base reimbursement
for new technologies.
• The SIG’smissionwill be to improvemodel transparency, updating,idea exchange,
currencyand policydevelopment.
10

11. Rationale and Priority: Why should this topic be a priority
for ISPOR?
• Topic of discussion in the peer-reviewed literature for at least 10 years
• Open-source models are essential if cost-effectiveness analyses are to be
widely accepted to reduce bias, increase transparency, improve model
access, allow for faster access to critical knowledge and to “promote
trustworthy, reproducible, validated, comparative and flexible health
economic models” [Dunlop et al and many others].
• ISPOR-SMDM guidelines and EUnetHTA joint action projects supportive of
these views
• ISPOR, as the leading HEOR organization, has developed numerous
guidances in the field and is well-positioned to explore these ideas via the
proposed SIG
11

12. Presentations
3

13. Summary of Presentations
• Lotte Steuten will provide a general introduction and overview from the UK,
Europe and the US.
• Jeroen P Jansen will discuss the rationale and logical consequences of open
source models, using his NSCLC model as an example
• Nancy Risebrough will present the pros and cons of having developed a COPD
open source model from various stakeholder perspectives
10

14. What are we talking about?
Transparency
Validity
Efficiency
Vemer P et al., Pharmacoeconomics. 2016;34(4):349-61.
Husereau D et al., Value Health. 2013 Mar-Apr;16(2):e1-5.
Eddy DM et al., ISPOR−SMDM Modeling Good Research Practices Task Force. Value Health. 2012;15(6):843-50.
AdViSHE Tool
ISPOR-SMDM
Modeling Good Research Practices
CHEERS Statement

15. 30,000 feet view: models for rheumatoid arthritis
Decision TreesMarkov Models
Discrete Event
Simulation
Individual Sample
Methods
n = 58 publications, 1996 to 2012
Decision tree, n=13
Markov, n=25
Individual sampling, n=13
DES, n=7
Bold letters represent the original
model within a tree of references
- Scholz S and Mittendorf T, Health Econ Rev, 2014

16. A deeper dive:
breast cancer models
40-fold
difference
ICERs
Frederix GW et al.
Value Health. 2013;16(6):1100-2

17. View from the UK

18. Organisational structures
Processes
Accountability
Methodology
Legislation
30 countries
80+ organisations

19. ICER Pilot
Transparency
Project

20. Description of Transparency Pilot Project
• Contracting/Fees
– Manufacturers contract & pay fee to ICER’s academic collaborators to cover time needed to
produce sharable executable files and all supporting documentation
• Ownership/Use requirements
– To protect IP of researchers and institutions, models shared under confidentiality or licensing
agreements
– Unable to keep or alter model for their own purposes
• Security/NDA
– Signed non-disclosure/confidentiality agreements
– Secured access (e.g., Dropbox)
• Access/Support infrastructure
– Access for limited timeframe, with no saving or retention of materials
– Confined to team of individuals responsible for review
– Support via initial orientation, follow-up email/phone contacts (limited), in-person workshops
• Expectation is that reviews will inform written public comments on our draft reports
Chapman RH, Kumar VJ. Manag Care Spec Pharm. 2019 May;25(5):521-524.

21. Transparency Pilot: Results/Lessons Learned
• Feedback received from manufacturers and other stakeholders will help
determine ICER’s future approach to sharing economic models associated
with other evidence reviews
• Bridges
– Model access provided to manufacturers/consultants
– Technical support processes developed
– Contracting, sharing infrastructure developing models
• Walls
– Timing for contracting, review, etc.
– Level of support expected (e.g., level & number of interactions), model changes
over time
– Financial barriers to access for non-manufacturer stakeholders
– Confidential/redacted data
Chapman RH, Kumar VJ. Manag Care Spec Pharm. 2019 May;25(5):521-524.

22. Final considerations
• Need to get more precise when we talk about ‘transparency’
– Of what, for whom?
• Nurture existing incentive structures to facilitate model sharing and stimulate
building collaborative networks that
– save time and money to industry and HTA-bodies
– enable all stakeholders to have a more complete view of relevant models
– create safe space for decision modellers to share the content of their models
• Better understand the disincentives as well as trade-offs for different
stakeholders including patients and society
– Consider the wider societal costs and benefits of sharing vs. not sharing
Thanks to Chris Sampson, The Office of Health Economics, for sharing his thoughts and insights.

23. Key Project/Member
Engagement Projects
4

24. WorkingGroups
• KeyProject
• Type: Identification of Issues
Surrounding and Possible Solutions
for Making Open Source Models
Available
• Issues to address: model creation,
access, trust, intellectual property
rights, model ownership and oversite,
payment, storage, maintenance,
updating and incentives, resources
and legal remedies available to
develop and maintain models
• Deliverables:
• Value in Health manuscript
• Part of a short course on issues in
model building24
• MemberEngagement
• Types/deliverables:
• At least annual webinar to the SIG
and/or open to the ISPOR population
around cost-effectiveness, comparative
effectiveness and disease models
• SIG group discussion, during at least one
major ISPOR conference per year, on one
or more forums/webinars
• Possible very short-term project to
comment on terms related to our SIG in
the ISPOR Book of Terms, including
defining the terms, providing references
and related terms

25. Immediate Next Steps of the SIG
• Determination of SIG Key Project and Member Engagement Co-Chairs
• Members solicited to work on webinar, SIG group discussion and manuscript
• Develop and refine webinar content outline and possibly present webinar;
develop pre-webinar questions for ISPOR audience
• Develop and refine detailed outline of ViH manuscript
• Develop and refine SIG group discussion outline for ISPOR US, May 2020
• Book of Terms?

26. ISPOR Special Interest
Groups (SIGs)
5

27. Special Interest Groups
• Mission: To provide an opportunity for ISPOR members to identify current
and trending topics and initiate platforms that focus on such topics.
• Composition: Any ISPOR member interested in a specific topic. A SIG must
be multi-stakeholder and geographically diverse.
• Goal: To engage ISPOR members within the topic area to advance health
economic and outcomes research and the use of this research in health care
decisions.
• Objectives: To develop area topics into educational and/or scientific work
products. To monitor and share information relating to area topics

28. Special Interest Group
New Structure
SIG Members
Special Interest
Group
Leadership
Member
Engagement
Working Group
Key Project
Working Group
Special Interest Group Leadership
• Chair-elect
• Chair
• Past chair
Each position is a 1-year term, resulting in a 3
year commitment

29. 29
Special Interest Group
Responsibilities of Chair-elect, Chair, Past Chair
• Provide overall direction and
leadership
• Identify the topics SIG members
would like to address
• Categorizing topics into long,
short term projects
• Identify the appropriate platforms
for delivery
• Work with the Working Group co-
chairs to ensure the timeliness of
the key project
• Address any issues within the
group
• Provide quarterly updates to the
SIG membership
• Current activities, new
developments in field
• Report the progress of the SIG
projects
• Via the yearly business and
project plan to HSPC
• Recruit new SIG members

30. 30
Special Interest Group
Each SIG will have at least 2 Working Groups
Key Project Working Group
• Manuscript in Value in Health or
Value Outcomes & Spotlight
• Short Courses
• Tools for Website
Member Engagement Working
Group
• Webinars
• Journal Club
• Presentation at ISPOR meetings
or other organization meetings
• Encouraged communication

31. 31
Working Group Co-Chairs
Responsibilities
Key Project
• Serve until the project is
complete (average 24 months)
• Lead the project
• Ensuring adherence to ISPOR
processes and working group
deadlines
• Address any conflicts or issues
Member Engagement Projects
• Serves for at least the year
• Work with SIG chairs to identify
member engagement topics
• Identifying speakers, articles,
topics for member engagement
activities
• Ensuring that all member
engagement activities occur
Those interested in leadership positions, please contact apavlock@ispor.org or
opensourcesig@ispor.org.

32. 32
Special Interest Group
Next Steps
Identify:
• A list of Potential Topics to be
developed as:
– Key Project
– Member Engagement Project
• Members interested in
developing those topics into
projects (Leadership group)
• Leadership group to complete
Business plan
– Reviewed by Scientific and
Health Policy Team
– Submitted to Health Science
Policy Council

34. Divider Slide
1
SECTION

37. ISPOR, the professional society for health economics and
outcomes research (HEOR), is an international, multistakeholder,
nonprofit dedicated to advancing HEOR excellence to improve
decision making for health globally. The Society is the leading
source for scientific conferences,
peer-reviewed and MEDLINE-indexed publications, good practices
guidance, education, collaboration, and tools/resources in the
field.
ISPOR’s community of more than 20,000 individual and chapter
members from 120+ countries includes a wide variety of
healthcare stakeholders, including researchers, academicians,
regulators and assessors, public and private payers, healthcare
providers, industry, and patient representatives. The Society’s
leadership has served as an unbiased resource and catalyst for
innovation in the field for more than 20 years.