Software Sustainability in e-Research: Dying for a Change

Software Sustainability Institute
www.software.ac.uk
Software Sustainability
in e-Research:
Dying for a Change?
Neil Chue Hong
N.ChueHong@software.ac.uk
e-Research Australasia
11 November 2010

www.software.ac.uk

www.software.ac.uk
UK e-Science Programme:
Preparing the Ground
• “e-Science is about global
collaboration in key areas
of science, and the next
generation of
infrastructure that will
enable it”
 John Taylor, D-G RCUK
• e-Science Centres
• e-Science Pilot Projects

www.software.ac.uk
OMII: Sowing the first seeds
11 initial projects funded
by Managed Programme
Many projects
flourished
But some wilted and
decayed
OMII setup to harvest and maintain software
output of UK e-Science Core Programme

www.software.ac.uk
OMII-UK: Cultivating and
Nurturing
• Emphasis on helping
existing software grow
 Extra gardeners brought in
(Edinburgh and
Manchester) with their
own plant stock
 Making the garden public
through initiatives like
Google Summer of Code
and ENGAGE
 Inviting specialists through
the PALs scheme
Cultivate and sustain community
software important to research

www.software.ac.uk
Software Sustainability
Institute: pruning, staking,
grafting
• Working with research software
users and developers
 Helping review and refactor
 Providing support and skills
 Identifying areas of convergence
• Producing strong, capable
software able to live long
and be successfully built on

www.software.ac.uk
SSI Pilot Projects
Pilot collaborators:
• Fusion
Energy
• Climate
Policy
• Geospatial
Linked Data
• Crystal
Structure
• Brain
Imaging
• Scholarly
Journals

www.software.ac.uk
No one startsNo one starts
writing softwarewriting software
intending tointending to
abandon itabandon it
completelycompletely

www.software.ac.uk
… and yet …
• “Sustainability is not an issue for researchers,
we want others to take the software over
subject to IP issues”
• “Sustainability is a big issue, we are producing
complex tools we want to continue to use but
it’s not clear how they’ll be sustained”
 Quotes taken from ENGAGE interviews

www.software.ac.uk
Just the Nature of the problem?
Maintenance is not fun
Hacking is fun
Statistics courtesy of Greg Wilson, Software Carpentry, from Nature article
Published online 13 October 2010 | Nature 467, 775-777 (2010)
doi:10.1038/467775a

www.software.ac.uk
A quick comparison
• Basic training for
kitchen chef: 3-4 years
• Head chef: 10 years
• Basic training for s/w
engineer: 3-4 years
• Architect: 10 years
PhotobyZagatBuzz
• Training in S/W Dev in UG Physics: 140 hours
• Training in S/W Dev in UG Geography: 0 hours

www.software.ac.uk
Software development
comes in stages
• Bridging criteria: strength of team; strength of
market; proximity of software to market
Idea Prototype Research
Idea Prototype
Idea
Idea Prototype Research Supported Product
Idea Prototype Research Supported
An idea to solve a problem
Scaling to work for others
Understand the functionality
Allow others
to participate

www.software.ac.uk
e-Research is multidisciplinary,
timescales don’t synchronise
14
Gap in Interest?
Cutting Edge Research
Applied Research
Timescales vary:
-ARIES (1989 – 1994)
-Giant Magnetoresistance (1988 – 1999)
-Frequency Hopping (1903/1942 – 1976)
-Bayesian statistics (1763 – 1996)

www.software.ac.uk
Software is noSoftware is no
longer easy tolonger easy to
define, let alonedefine, let alone
sustainsustain

www.software.ac.uk
What do we sustain:
- Workflow?
- Software that runs workflow?
- Software referenced by workflow?

www.software.ac.uk
Novel reuse of public sector data
http://www.mysociety.org
What do we sustain:
- Map?
- Software that creates map?

www.software.ac.uk
Sustaining Function or Form
19
What do we sustain:
- Function?
- Form?

www.software.ac.uk
Repositories areRepositories are
not the solution…not the solution…
… only a mechanism… only a mechanism

www.software.ac.uk
re pos i tor y⋅ ⋅ ⋅ ⋅
/noun/ [ri-poz-i-tawr-ee]
• 1. a receptacle or place where things
are deposited, stored, or offered for
sale.
• 2. a burial place; sepulchre.

www.software.ac.uk
Preserving something …

www.software.ac.uk
… is not the
same as sustaining it

www.software.ac.uk
The Zombie Effect
• Software not always fully alive
when you reanimate it!
• Complex set of dependencies
 Significant Properties of Software
 Purposes and benefits of
software preservation
http://www.jisc.ac.uk/media/documents/
programmes/preservation/significantpro
pertiesofsoftware-final.doc
http://softwarepreservation.jiscinvolve.org/wp/

www.software.ac.uk
Open Source isOpen Source is
not the solution…not the solution…
… only a mechanism… only a mechanism

www.software.ac.uk
Open Source Software is Free...

www.software.ac.uk
Free as in Puppy...
• Long term costs
• Needs love and
attention
• May lose charm
after growing up
• Occasional clean-
ups required
• Many left
abandoned by their
owners
Inspired by Scott McNealy
Photos of Great Pyrenees from Jen Schopf

www.software.ac.uk
Open Source
Process is
rarely completed
Committer
Policies
Contributor
Agreements
Project
Infrastructure
Licence
Resolution
Open Source is not just a licence

www.software.ac.uk
Exploiting software for
sustainability
Models
• Grant Mosaic
• Institutional support
• Fully Costed Service
• External Enterprise /
Consultancy
• Royalties and Fees
• Donations
• Advertising
• T-shirt (spinoff
merchandising)
Vehicles
 University based
 Spin out company
 Consultancy and
Customisation
 Industrial knowledge transfer
 Contracts
 Licensing
 Certification
 Support services / training
 Software as a Service
 Software Foundation

www.software.ac.uk
Software sustainability
is part of the process
• Comparable to risk management
 No one right “solution” but many examples of best
practice and process
• Plan from before the start if possible
 But must be reviewed regularly
• No longer considering timescales bounded by a
project, but considering the product

www.software.ac.uk
Sustainability in Context
Product
Management
Funding/
Effort
Community
Engagement
Software
Engineering
Software
Sustainability
Support /
Contributions
Market
Development

www.software.ac.uk
Communities areCommunities are
important…important…
youyou are the solution!are the solution!

www.software.ac.uk
Sustainable Communities
• Cohesion and Identity: Creating
a community
• Tolerance and Diversity: Smart
growth through collaboration
• Efficient use of resources:
Leveraging infrastructure
• Adaptability to change:
Governing sustainably

www.software.ac.uk
A: Analyse Customers
Slide from Jim
Austin

www.software.ac.uk
B: Build Community
Building intellectual access ramps to support incremental
engagement – building capacity and capability
•Individual
•Group
•Consortium
•W/ industry
•Community
•Active
Teams change as
project matures
35http://www.castep.org/

www.software.ac.uk
C: Cultivate Contributors
• Basics: Website, mailing list, code repository, issue resolution
• Remove barriers to participation, increase efficiency
• 1993: First public release; 2 devs
• 1995: Code open sourced; 3 devs
• 1996: r-testers list set up
• 1997: lists split: r-announce, r-help,
r-devel; public CVS; 11 devs
• 2000: CRAN split and mirror
• 2001: BioConductor
• 2003: Namespaces
• 2005: I8n, L8n
• 2007: R-Forge
• Today: BioConductor (33 core devs),
R-Forge (532 projects, 1562 devs),
CRAN (1400+ packages)
36
http://cran.r-project.org/doc/html/interface98-paper/paper_2.html

www.software.ac.uk
Reuse,Reuse,
not reinventnot reinvent
Innovate,Innovate,
not replicatenot replicate

www.software.ac.uk
The Software Maturity Curve
Softwareproliferation
Time
CustomisationInnovation Consolidation
Portals
Quantum
chemistry
Cloud
Computing
RDBMS
Social
Simulation
Workflows
Spatio-
Temporal
viz
Molecular
Dynamics
Geospatial
viz
Digitised
Doc Analysis
Digital
repositories

www.software.ac.uk
Enabling Innovation
• Supporting emergent disciplines
 Needs recognition of innovative software development
as part of funding
• Breaking down barriers
 We cannot assume that the way people interact with
resources will conform to expectations
 e.g. researchers will use/store files outside of universities
 Researchers will do whatever they can to get an edge –
they will not always conform
 We need to understand the relationship between e-
Research and e-learning frameworks (VREs/VLEs?)

www.software.ac.uk
Supporting Consolidation
• “e-Science is an organic, emergent process requiring ongoing, coordinated
investment from multiple funders and coordinated action by multiple
research and infrastructure communities. It is both an enabler of research
and an object of research” – RCUK Review of e-Science
• Bridging the expectation gaps between participants
 Maintenance vs. research
 Different timescales for “exciting” work
• Well supported open platforms are the key in the age of
the research mashup
 Platforms to enable bottom-up innovation
 Platforms to enable citizen participation
 Competition/innovation built on top c.f. industry

www.software.ac.uk
Sustaining Customisation
• “The time constants for real transformative impact and
significant competitive advantage is decades” – RCUK
Review of e-Science
• Sustain software infrastructure in the long term
 Differing models: through centres; within institutions;
distributed
• Need to change perceptions so that software is seen as
valuable! (and not just invaluable)
• Lower barriers to community growth and participation
• Increase value of providing services
• Virtually merge + map small amounts of effort / funding

www.software.ac.uk
Case Study: NeISS
• Evaluate impact of traffic control
measures over next 5/10/15 years
• Access baseline demographic data
about the city
• Execute simulation of traffic
system and population
• Visualise simulation outputs
• Augment with new forms of data
• Run dynamic models to assess
future patterns (congestion,
health, social inequality)

www.software.ac.uk
Case Study: NeISS
44
Photo: NASA/GSFC,

www.software.ac.uk
Case Study: NeISS
45

www.software.ac.uk
One last questionOne last question
How do we getHow do we get
credit for sustainingcredit for sustaining
software?software?

www.software.ac.uk
This is where YOU come in
• Research credit is based on publication output
• Data citations and credit for reuse are still not
commonplace
• Software credit is the next stage
 Otherwise how can we persuade people to
contribute back?
• YOU are the researchers, developers, providers
and funders… YOU can make it happen!

www.software.ac.uk
In conclusion
• Software Sustainability is not easy, but it is
possible
• Build it into the process and build the right teams
• Three key things to remember:
 ANALYSE CUSTOMERS
 BUILD COMMUNITY
 CULTIVATE CONTRIBUTORS
• We need to give credit so people contribute back

www.software.ac.uk
A National Facility
for Research Software
Become our next collaborators!
Email: info@software.ac.uk
Blog: http://www.software.ac.uk/blog
Twitter: twitter.com/SoftwareSaved
SlideShare: slideshare.net/SoftwareSaved
YouTube: youtube.com/user/SoftwareSaved
Telephone: +44 (0) 131 650 5030
Pilot collaborators:
• Fusion
Energy
• Climate
Policy
• Geospatial
Linked Data
• Crystal
Structure
• Brain
Imaging
• Scholarly
Journals

Software Sustainability in e-Research: Dying for a Change

Recommended

Recommended

More Related Content

Viewers also liked

Viewers also liked (20)

Similar to Software Sustainability in e-Research: Dying for a Change

Similar to Software Sustainability in e-Research: Dying for a Change (20)

More from Neil Chue Hong

More from Neil Chue Hong (6)

Recently uploaded

Recently uploaded (20)

Software Sustainability in e-Research: Dying for a Change

Editor's Notes