It is the first in a series of talks to explain various aspects of software development in the context of BExIS 2.The idea of this series of talks is to gather the knowledge needed to develop the BEXIS 2 data management platform. So the talks are geared towards software engineers and developers.
The first talk is focused on the general functional and non-functional requirements of a scientific data management system.
1. BEXIS Tech Talk Series
#1: The Big Picture
Javad Chamanara
December 2015
2. BExIS++ Project
2
BEXIS 2 SOFTWARE
DEVELOPMENT
SUSTAINABILITY
OUTREACH
SUPPORT
TRAINING
BEXIS Tech Talk #1: The Big Picture
3. Software Development
Data Management Platform
• (large) research projects
• tabular data, but not limited to
• data integration and re-use
4BEXIS Tech Talk #1: The Big Picture
4. Community
5
GRK 1666
GRK 1086
BEXIS 2
BExIS 1
BExIS 2
Biodiversity
Exploratories
Jena
Experiment
MPI-BGC
Research
Database
Kilimanjaro
EFForTS
GFBio
AquaDiva
BEFmate
iDiv
UFZ Halle
TerraSensE
BEXIS Tech Talk #1: The Big Picture
6. Tech Talk Series
• A Series of talks on
– Various parts of the software
– Software development
• In order to:
– Promote knowledge sharing
– Encourage external contribution
This is the first one!
7BEXIS Tech Talk #1: The Big Picture
7. Requirements
• Data Lifecycle Management
• Generic
• Extensible
• Portable
• Scalable
8BEXIS Tech Talk #1: The Big Picture
8. Requirements -> DLM
• Flexible Data Structures
• Data Submission
• Validation
• Preserving
• Metadata Management
• Versioning
9BEXIS Tech Talk #1: The Big Picture
10. Requirements -> Extensible
• Built-in Modules
– DLM related modules
• Third Party Modules
– BExIS 1 modules
– AquaDiva modules
– iDiv modules
– Open modules
• Interaction with external systems
– Geographic Information Systems
– Persistent Identifier Providers
– Authentication Providers
– Annotation Providers
11BEXIS Tech Talk #1: The Big Picture
11. Requirements -> Portable
• Multi DBMS support
– IBM DB2
– PostgreSQL
– MS SQL Server
• Various Installation Scenarios
– Dedicated hosting
– Shared hosting
– Portable hosting
– Multi-tier Deployment
– Secure Access
12BEXIS Tech Talk #1: The Big Picture
12. Requirements -> Scalable
• To be used in large projects with
– Multitude of Users
– Big Datasets
– Frequent Access
13BEXIS Tech Talk #1: The Big Picture
13. Out of Scope
• Sophisticated Data Analysis Tool
• Long Term Public Repository
14BEXIS Tech Talk #1: The Big Picture
17. Conceptual->Data Structure
• Data Structure
• Data Attribute
– Unit of measurement
– Constraints
– Value Validation
• Reuse promotion
18BEXIS Tech Talk #1: The Big Picture
22. Design Overview -> Data
• Relational
• XML
• Files
– CSV, Excel, Images, etc.
• Configuration
• Seed
• Geographical
23BEXIS Tech Talk #1: The Big Picture
23. Design Overview -> Functionality
• Core Functionality
– Data Lifecycle Management
• Dataset/ Data Structure
• Metadata/ Structure
• Versioning
– AAA
• Authentication
• Authorization
• Auditing
24BEXIS Tech Talk #1: The Big Picture
24. Design Overview -> Architecture
Components
• DLM
• AAA
• Transformation
– XML <-> XML
– CSV <-> Excel
– …
25BEXIS Tech Talk #1: The Big Picture
25. Design Overview -> Architecture
Modules
• A set of user oriented/facing functions
• Crossing multiple layers, including the UI
• Examples:
– DCM: Data Collection Module
– DDM: Data Discovery Module
– DIM: Data Integration Module
26BEXIS Tech Talk #1: The Big Picture
26. Design Overview -> Architecture
External Dependencies
• Geo Server
• Services
– Terminology
– PID
– Authentication
27BEXIS Tech Talk #1: The Big Picture
27. Design Overview -> Architecture
Public Services (APIs)
• Public REST Services
– Download a dataset
– Metadata Harvesting
• R Interface
28BEXIS Tech Talk #1: The Big Picture
29. Development Environment
Tools
• Visual Studio
• Team Foundation Server (task & source
management)
• Github (Public Releases)
30BEXIS Tech Talk #1: The Big Picture
31. Development Environment
DBMS
• IBM DB2 version 9.x and upper
• PostgreSQL version 8.x and upper
• MS SQL Server 2008 and upper
32BEXIS Tech Talk #1: The Big Picture
32. Deployment
• MS IIS
• Dedicated Hosting
• Virtual Machine Hosting
• Portable Hosting
• Multi-tier Deployment
33BEXIS Tech Talk #1: The Big Picture
33. Outlook
Whats next in the talk series?
• The Conceptual Model in more detail
• The database design
BEXIS Tech Talk #1: The Big Picture 34
