The Texas Digital Library moved all of its infrastructure and services to Amazon Web Services in 2011. This allowed TDL to deploy services more easily and elastically without being constrained by physical hardware limitations. While the transition was successful overall, TDL continues to evaluate AWS costs and considers options for certain services like digital preservation that may be better suited to on-campus infrastructure. The outage experienced by many AWS users in late 2012 highlighted the benefits of AWS's communication during such incidents but also reinforced the importance of TDL's ongoing evaluation of cloud strategy and risks.

2. Our Bits Sit on AWS
Texas Digital Library and Amazon
Ryan Steans
Senior Program Coordinator, Texas Digital Library
info@tdl.org

3. Agenda
• Who we are
• Fall 2010 – where we were
• Why we looked at Amazon
• Spring 2011
• Today and Looking Forward
TDL DLF Presentation

4. What is Texas Digital Library?
• 14 Member Consortium of Libraries at
academic/ research institutions across
Texas
• TDL provides centralized software support
and development
• Cost share
• Provide scholarly communication
options to schools all at the same level
TDL DLF Presentation

5. Texas Digital Library Services
• DSpace internet repositories (DuraSpace)
• Vireo ETD Management System (TDL)
• Open Journal System (PKP)
• Open Conference System (PKP)
• WordPress for research group websites and
blogs
• MediaWiki for research group data sharing and
management
• Training, consultation, conference, community
TDL DLF Presentation

6. TDL Activities – in the works
• Digital Preservation Network
• Fedora/ Hydra
• Data management planning
• Shared research repositories/ communities
• Vireo users community (thesis and dissertation
community)
TDL DLF Presentation

7. Fall 2010
• TDL located on UT Austin Campus
• TDL hardware at UT Data Center
• Data Center move scheduled for December
2010 for all of UT campus
• TDL hardware was very close to expiration
of service contracts, etc…
• TDL began looking at AWS for a backup
option during the move
TDL DLF Presentation

8. Texas Digital Library Inventory - Austin
Make Model QuantityUsage
Database / Application Server / DNS / LDAP / Handle /
Servers Sun SunFire 490 2 Shib
Dspace / WWW / Blogs / Wikis / Faculty Pages / Journals
Sun Sunfire T5220 2 / Mail / Services / Support / SVN
Sun Sunfire 245 2 Dspace Labs / Development
Sun Sunfire T5120 1 Preservation (iRods/SAM-QFS)
Old content management servers (positioned to help w/
Sun Sunfire X4150 2 web applications )
NetworkCisco Cataylst 3750 2 Network connectivity
Citrix Netscaler 2 Load Balancers
BrocadeSilkworm 300E 2 SAN Connectivity
AvocentCycladed ACS 2 Serial Console Connectivity
Dspace Assets / Web Application Data / Home
Storage NetApp Filer 3020 2 Directories / Backup
Preservation (iRods / SAM-QFS) / Virtual Machines /
Sun StorageTek 614 1 Databases
Sun SL500 1 Preservation (iRods / SAM-QFS)
Texas Digital Library Inventory - College Station
Servers Sun T 5220 2 Dspace
Sun T 5120 1 Preservation
Storage
NetApp Filer 3020 1 Dspace Assets / Backup
TDL DLF Presentation

9. 2010 ‐ Our existing hardware
• Very locked down with space
• Had to over‐buy in order to guess need
• Limited staff was worrying about physical
hardware
• All eggs in one basket
• Tied to university security protocols that were
difficult with multiple members
• IT infrastructure had led to bad user experience –
shared hardware, machines, etc…
TDL DLF Presentation

10. Looking at Amazon Web Services
• ELASTIC
• Could increase size of a machine
• Could increase number of machines
• Removed guess work and bad estimates
from the equation
• Elastic cost
• Service Contracts were highly reasonable
• Road tested. AWS launched in 2006.
TDL DLF Presentation

11. Spring 2011
• We moved to AWS for all services
TDL DLF Presentation

12. Why?
• Allowed us to separate service instances,
bring each up on a VM
• Created far greater user customization/
access/ control
• Rapid deployment of services from master
image
• Accounting
TDL DLF Presentation

13. How?
• Set up a duplicate system in AWS
• Used load balancer as the fulcrum
• Caused a minimum of downtime
TDL DLF Presentation

14. Today
• Running 100 separate AWS instances as of
11/19/2012
• Deploying services easily
• Literally nothing is on UT’s hardware nor at
any TDL member school
• Depend on standard security and AWS
• Cost
TDL DLF Presentation

15. TDL DLF Presentation

16. It’s actually working just fine
• A bit of “beg forgiveness instead of asking
permission”
• Costs are down
• Members don’t notice
• Our staff is handling it well (2 longtime
university IT people as sys admins)
TDL DLF Presentation

17. An overly simple idea of service
provisioning
TDL DLF Presentation

18. But…
• Network cost could become burdensome
• Cost savings per unit no longer as obvious
• UT and our own team more comfortable
with VMs and could move to purchased
hardware
• AWS billing not easy to manage month‐to‐
month
TDL DLF Presentation

19. And…
• As we look at the Digital Preservation
Network ‐ we will rebuild on campus
• Security concerns
• Goal of group is to do this in academia
• It may also affect how we think of
implementing Fedora/ Hydra
TDL DLF Presentation

20. Week of October 22nd…
• TDL learned of outage at 1:15 PM Central
• Services were visible, but not accessible
• Contacted members
• Watched logs, news reports, etc…
• 7:30 PM we were back up and running
with no data loss
• Received 7 communications through
helpdesk
TDL DLF Presentation

21. It’s bad, but…
• AWS was in communication with users
• We feel confident in them, and they got it
back up
• Checking our systems was fairly easy
• Took some pressure off our staff
• Outages are not limited to AWS
TDL DLF Presentation

22. Looking forward
• We will always rethink our strategy on
AWS
• AWS is providing us with a sales person
and a consultant before end of year
• Cost has to remain reasonable for
uploading, downloading
• Will be moving preservation efforts to
campus
TDL DLF Presentation

23. Ryan Steans, Program Coordinator
Kristi Park, Program Coordinator
info@tdl.org

24. Tales from the Cloud:
Experiences and Challenges
Andrew Woods
http://duracloud.org
04 Nov 2012

25. Why might I even be
interested in “the cloud”
?
2

26. What are some risks
?
3

27. How is it different from
local infrastructure
?
4

28. What about this
“DuraCloud”
?
5