1.
Big Data Initiatives for
Agroecosystems
Cynthia Parr
Knowledge Services Division
National Agricultural Library
Ecological Society of America, 2015

2.
Outline
• Data management at the
National Agricultural Library
• Four examples
1. Insects 5K – i5K Workspace
2. Life Cycle Assessment
3. Long-Term Agroecosystem
Research
4. Ag Data Commons
• General principles
8.1 million items,
Agricola, PubAg

3.
3
http://blog.thingarage.com/
raw data
citable
publication

4.
4
raw data
collection
cleaning, enrichment, analysis
registration, preservation
temporary
data
referable
data
citable
data
citable
publication
Modified from Peter Wittenberg, Research Data Alliance
https://rd-alliance.org/group/data-fabric-ig.html

5.
i5k.nal.usda.gov
5

6.
Genome project hosting at the
i5k Workspace
• 27 pilot genomes hosted; 45 total
– Storage and dissemination of a
genome assembly and anything
mapped to it.
– BLAST, JBrowse Genome Browser
• Manual Curation: Web Apollo
• Post-curation maintenance
– Quality Control
– Official Gene Set generation
• Research plan
• Generate material
• Sequencing
• Assembly
• Automated
annotation
• Manual Curation
• Official gene set
generation
• Genome project
maintenance
• Biological
insights/Publicatio
n
GenomeProjectTrajectory

7.
Life Cycle Assessment Commons
7
www.lcacommons.gov

8.
Unformatted,
non-standard
LCA Commons Concept
LCA Community
Open LCA Framework
Common computing environment, application,
data standards, and development
NAL
LCADC
NREL
USLCI
XYZ
LCI DB
ABC
LCI DB
Distributed computing
environment & application
Common data standards
Distributed computing
environment
DEF
LCI
DB
Common application
& data standards
Interoperability Tools
Ag Data
Commons
Catalog and
Repository

9.
Long Term Agro-ecosystem Research
(LTAR)

10.
LTAR Data
Common Observatory
– Meteorology
– Hydrology
– Eddy flux CO2
– Non-CO2 gasses
– Soil
– Biological
10
Common Experiment
Approach
– Business as usual
– Aspirational
Will include data about
– Management practices
– Results

11.
LTAR Data Loss
N=194 of ~500 citations in 2011 LTAR site proposals
Bad links
to data
No data
available
80% of papers provide
no way to obtain data
Data are
accessible
Refers to
general data
source

12.
LTAR information management
• Support for download of files, web services
• Metadata in FGDC CSDGM, ISO 19115, EML,
Project Open Data
• Catalog of instrument specs using SensorML 2
• Data dictionaries in ISO 19110
• Weather data to be converted to other formats
• Field names could be converted to match different
conventions (AgMIP, etc.)

13.
Ag Data Commons
13

14.
data.nal.usda.gov
Enhanced
DKAN

15.
Distributed
repositories
Search &
Knowledge
Discovery
Thesaurus &
Indexing
Ag Data
Commons
Repository
Organization
& Curation
Grant
management
systems
INGESTION DISSEMINATION
PubAg
Dataset
Submission
Analytics
& Tools
Data.gov
Forest Service
NCBI
Ag Data
Commons
Catalog
Color Legend:
Building
Adapt/Re-use
Existing
LCA Commons

16.
Guiding principle 1:
a distributed network ….
Geospatial
Catalog
Geospatial
Repository
STEWARDS
Ag Data
Commons
(catalog)
Ag Data
Commons
(repository)
USDA
Enterprise
Inventory
National
Weather
Service
Data.gov
Ecosystems
.data.gov
of Networks…

17.
Public access to open, machine
readable data enables larger
scale, integrative and innovative
data science
The long tail
Guiding principle 2:
big data AND long tail

18.
Guiding principle 3:
curation adds value
• Data dictionaries
• Standards & templates
• Linkages
• Semantics
• Preservation

19.
Thanks!
National Agricultural Library
Knowledge Services Division: Susan McCarthy
LTAR Jeffrey Campbell, Charles Lockwood
i5K Monica Poelchau, Chris Childers
LCA Commons Peter Arbuckle, Ezra Kahn
Ag Data Commons Ursula Pieper, Jocelyn McNamara,
Qing Qu, Erin Antognoli, Melissa
Lowrey, Jaylen Nathwani, NuCivic
… and collaborators and testers