This presentation was provided by Daniel Calto of Elsevier during the NISO virtual conference, Research Information Systems: The Connections Enabling Collaboration, held on August 16, 2017.

1. | 1| 1| 1
16 August 2017
Commercial RIS Systems: Benefits,
Costs, and Key Considerations of Use
Daniel Calto
Director of Solution Services
Research Intelligence

2. | 2| 2| 2
Summary of recommendations (August 2010)

3. | 3| 3| 3
Metrics Scope: Inputs, Process, Outputs (August 2010)

4. | 4| 4| 4
The Research Lifecycle—Inputs

5. | 5| 5| 5
The Research Lifecycle—Outputs and Impact

6. | 6| 6| 6
Presentation Structure
1. Determination of key needs
2. Buy or build considerations
3. The importance of data quality and completeness
4. Vendor selection process
5. Importance of data model--configuration vs. customization
6. Ease, speed, and support level of implementation
7. Importance of total cost of ownership analysis
8. Change management considerations
9. Need for ongoing internal marketing and financial support post-
implementation

7. | 7| 7| 7
Determination of Key Needs
• Determination of critical information and system needs has to come first in the
process of determining the best solution for your institution.
- Best practice initially is to keep this discussion internal to the university—if you invite vendors or
preselect solutions too early you’re likely to shape the solution around the product rather than the
reverse
- Ideally you can map the current and desired business process as part of the determination of key
needs. Such visual maps can be incredibly useful when designing a solution, and offer a way to
talk consistently across the institution about what is being proposed.
- Avoid laundry lists of features/functions or too much detail at this stage—emphasis should be on
what the most acute information problems are, and what could be done broadly to solve these
problems.
- Consensus is key, but absolute agreement isn’t possible. Make clear to stakeholders, even
important ones, that there will be no hijacking or special pleading on what the system should do.
No system is a tabula rasa.
- Hire an experienced project manager if you don’t have one already—someone from outside is
often more objective.
- Outlining an ideal solution is a useful exercise, but be pragmatic and willing to settle for an 80%
solution—rainbows and unicorns don’t really exist in the software world.
- Bang for buck considerations are important. Measure a system’s effectiveness and value vs. how
system currently functions.
- Don’t underestimate the difficulty of changing longstanding business processes and how people
at the university will react. The human factor in implementations is the most volatile.

8. | 8| 8| 8
Buy or Build Considerations
• Buy or build considerations are among the most challenging issues facing
institutions early in the decision process.
- Building or enhancing an existing system internally can be the best choice, but it is essential to take into account
implementation timelines, the total cost of ownership, the potential for cannibalizing or taking away resources
from other internal IT projects, and the increased possibility that an internal project will become a political football.
- Be aware that university IT departments have their own budgetary and personnel agendas. “No, we can build
that ourselves” should not be determinative absent some very strong evidence that the department has done so
successfully in the past.
- Commercial software systems are generally developed over many years by dedicated teams of computer
scientists and software designers. This narrow focus on optimizing and bringing to maturation a specific product
is probably one of the primary reasons OTS software solutions can be an attractive choice for institutions, as it is
unlikely that most institutions can commit so many resources for an open-ended project.
- Increasingly institutional CIO’s are looking to commercial OTS (off-the shelf) solutions to implement some
institutional IT priorities. This is not always because the capability of developing something is absent, but
because of the need to manage the long list of projects that the IT department is likely responsible for. For
example, if you are currently upgrading a legacy finance system internally, it may be smart to commission an RIS
system from a commercial vendor.
- Making a bad choice has very high opportunity costs, and can burn up years of an institution’s time, energy, and
money. Regardless of the ultimate decision, it should be driven by an analytical look at budgets, time, system
quality, etc. Effective use of project managers, good governance structures, and objective considerations of
options should be the driving force behind decision-making.
- At this stage, it can be very useful for decisionmakers and advisors to start talking with other customers who
have purchased the products being considered. Try to get a sufficiently large sample, and to avoid “steering” by
the vendors. Most commercial systems under consideration will have a number of institutional customers similar
to you, and IT/information considerations across universities are often quite similar.

9. | 9| 9| 9
The importance of data quality and completeness
• Data quality and completeness is fundamental to the success
of any IT implementation. With the emergence of modular XML-
based systems, it is far easier than ever before to link systems
together. Data is like water—it can flow from one place to
another, but it is essential that the water is clean.
• Any analysis of what system would be ideal and implementable
at your university needs to include a careful analaysis of data
quality and availability,
• The other primary consideration about data should be its
completeness—does the system include a comprehensive data
source and/or a means to obtain such data? If so, what it the
expected effort to compile such data? There should be an
assessment of what time and effort are required to complete
such data gathering. Failure to consider this can make projects
far more expensive and time-consuming to complete and
increase the chance of implementation failure,

10. | 10| 10| 10
Vendor Selection Process
• A number of institutions have a well-established means to review available
vendors and to assess which may offer the most appropriate solution to the
institution. Price, system capabilities, and overall workload to implement should
all be considered. Money and time are both scarce commodities, and it may be
preferable to spend more money if it saves significant time and effort or vice
versa.
• After determining your key needs, mapping your current and desired business
process, talking to other institutions, and if possible hiring or assigning an
experienced project manager, you’ll likely want to gather a shortlist of possible
vendors.
• Some institutions may prefer or be required to go to an RFP process, others
may run a more informal process, but it is important that you maintain control
and structure over the process as a whole. I think it’s essential at some point to
have the top vendors on the shortlist present their systems to you. It’s also
useful to have a dedicated “hit man” with technical knowledge to ask probing
questions to understand both the strengths and weaknesses of any system.
• Deciding which choice is right for you may be difficult. It’s important to be
comfortable with the people you’ll be working with. Implementations can be
intense, are likely to have a few unanticipated bumps--and it’s important to
trust the company and the people you’ll be working with to deliver what is
promised. At the same time, you need to complete a more analytical
technology and product capability analysis to decide which system is best
suited for your organization.

11. | 11| 11| 11
Importance of Data Model--Customization vs. Configuration
• Another key consideration when looking at an RIS system is the
underlying data model that the platform uses. What are the data types or
data entities that make up the data model? Is the model sufficiently
articulated to represent a wide range of scholarly research outputs, not
only research papers and books, but interviews, film screenings, art
exhibitions, software authorship etc.? Does the model include teaching
and service activities? Does it conform with CERIF or other data
standardization and modelling initiatives?
• Universities are often attracted to vendors who can promise customization
of certain features of capabilities. Customizations can be beneficial, but
also have potential downsides. It isn’t possible to change a data model
easily. A vendor that makes multiple customizations may fork their code
base and have significant difficulties in updating the software because of
multiple versions. Having configurable standard elements may be a viable
alternative to customization, although these may offer less overall
flexibility.
• Normally, most software developers will try to retain a standard code base
and incorporate improvements to the entire code base and make these
enhancements available to all of their customers. This is also far more
cost effective in product development terms.

12. | 12| 12| 12
Ease, Speed, and Support Level of Implementation
• As mentioned, money and budget are key considerations in
making a buy vs. build decision. If you decide to go with a
commercial vendor, the ease of implementation, the related
timelines, and the overall support level from the vendor both
during the implementation and after launch should be carefiuilly
considered. These equate to money and effort—the longer an
implementation takes, the higher the overall rate of failure;
going over budget and missing deadlines can erode support for
a new system very quickly. Previous track record and prior
experience with similar implementations can be important
indicators. No implementation is completely without issues or
probelms. Another important signal is a vendor’s ability to
produce a detailed implementation plan, either shortly before
signature or at a kickoff event to start the implementation.

13. | 13| 13| 13
Importance of Total Cost of Ownership Analysis
• A good vendor selection process should have a financial analysis
done that looks carefully at the total cost of ownership of a system
over time. Typically commercial solutions will require more outlay
of funds initially that building a solution in-house, but the overall
costs of in-house systems often exceed the total cost of
commercial systems in the medium to long term. If an in-house
solution requires 4 IT people working full-time vs. 1 for a
commercial solution, this can cost upwards of $360,000 per year
in salary and benefits, which may exceed the entire cost of
purchase. This is why an IT department saying is that free
software is like a free puppy. Often such an analysis will show
roughly equal costs, in which case ease and speed of
implementation may become critical considerations. TCO analysis
can also be useful in gauging the relative benefits of a more
expensive but comprehensive system vs. a lighter but more limited
solution. System lifetime (5 years, 10 years?) should also be
considered.

14. | 14| 14| 14
Change Management Considerations
• As mentioned, change management and the reaction to new systems can
be one of the most unpredictable and potentially divisive aspects of
implementing IT systems. People are generally resistant to changing their
work habits and workflows without some significant personal benefit for
doing so, such as saving time, reducing drudgery, or saving money.
Similarly, individual faculty may be underwhelmed by arguments that its
good for university administrators, but happy to change if a system offers
them a better way to build CV’s, manage their grant information, or gives
them relevant reports in a painless way.
• In general, because commercial software is usually more mature and built-
for-purpose, it is easier to implement with a small group and then roll it out
to the broader university when the time is right to do so. In my experience,
building systems internally can result in having 50 unwanted “advisors”
telling you what the system should do—if the build takes too long, the risk
of failure is significantly higher, and funding of the project is more
imperiled.
• Initial impression and early rollout of the system should be carefully
planned and considered, and these can have a major effect on overall
acceptance and success of the system.

15. | 15| 15| 15
Need for ongoing internal marketing and financial support
post-implementation
• Most implementations require substantial initial effort and commitment of
resources. Once a system is up and running, a normal reaction is often to
relax and congratulate your team on a job well done.
• A key best practice once a system is up and running, however, is to make
certain that there are continued dedicated resources to properly market
and expand the use of the system among faculty, staff, students and
others. This is essential to ensure that your university gets maximum value
out of your investment, and it’s frequently an area that universities are not
well-equipped for.
• A second important consideration, even after an implementation has been
a success and its usage has built substantially over time, is to ensure that
the system has ongoing financial resources allocated to keep it healthy in
the long term. If continued funding is dependent on a single champion and
or senior leader’s support, if he or she leaves or takes a different role,
even highly successful systems may have to be discontinued because of
fudning issues. The best way to guard against this risk is to have a
governance board for the system that is responsible for allocating
resource, hiring project managers, and solving problems. These boards
can be small in the number of members, but should draw from a cross-
section of stakeholders in the system,

16. | 16| 16| 16
www.elsevier.com/research-intelligence
Thank You
Q&A