Secure your environment with UiPath and CyberArk technologies - Session 1
Scientific Data and Knowledge Management in Aerospace Engineering
1. Scientific Data and Knowledge Management in
Aerospace Engineering
Korea e-Science AHM 2008 (September 8th 2008, Daejeon)
Andreas Schreiber <Andreas.Schreiber@dlr.de>
German Aerospace Center (DLR), Cologne
http://www.dlr.de/sc
Folie 1
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
2. Outline
German Aerospace Center (DLR)
Introduction
Excurse: DLR CFD Codes
Grid Computing: D-Grid and AeroGrid
Software Tools: DataFinder, RCE, and XPS4CFD
Conclusion
Folie 2
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
3. The DLR
German Aerospace Research Center
Space Agency of the Federal Republic of Germany
Folie 3
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
4. Sites and employees
5,700 employees working Hamburg
in 28 research institutes and Bremen- Neustrelitz
facilities Trauen
Berlin-
Braunschweig
at 13 sites.
Göttingen
Offices in Brussels,
Paris and Washington. Köln
Bonn
Lampoldshausen
Stuttgart
Oberpfaffenhofen
Weilheim
Folie 4
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
5. Introduction
Simulation in Aerospace and Avionics
Designing new space and aerospace vehicles require high-resolution
numerical simulation steps conducted in complex workflows
The complete simulation of all flow phenomena throughout the
entire flight envelope including the multidisciplinary simulation of
all involved disciplines
The multidisciplinary optimization of the overall aircraft design
as well as the design of major parts, such as the turbine engines
Involved disciplines:
Aerodynamics – Structure – Heat – Flight mechanics – Radar & Infrared
signature – Materials (physics/chemistry) – Combustion …
Folie 5
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
6. Software Technology for Aerospace Simulations
Numerical Codes and Supporting Tools
Highly sophisticated and optimized numerical simulation codes
For example, high-fidelity CFD codes
Many codes available (free, commercial, proprietary)
To major CFD codes of DLR
TAU
TRACE
Simulation infrastructure and supporting tools
Grid environments
Data and workflows management tools Integrated
Integrated
Knowledge management ( documentation!) environments
environments
Preprocessing, post processing, visualization
Folie 6
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
7. DLR CFD Codes
Folie 7
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
8. DLR TAU Code
Finite Volume CFD Solver
Reynolds-averaged Navier-Stokes
(RANS) code
Steady and unsteady flow, hybrid grids
State-of-the-Art turbulence models
Adaptation module for local refinement
Chimera method
Completely parallelized (MPI)
Used in European industry (Airbus,
EADS)
Contact:
DLR Institute of Aerodynamics and Flow
Technology
http://www.dlr.de/as/case
Folie 8
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
9. DLR TRACE Code
Turbo machinery CFD
TRACE: Turbo machinery Research
Aerodynamics Computational Environment
unsteady Reynolds-averaged Navier-Stokes
(URANS) code
Steady and unsteady flow (finite volumes)
Hybrid multi block method
Completely parallelized (MPI)
Used in European industry (Siemens, MTU
Aero Engines, Rolls Royce)
Contact:
DLR Institute of Propulsion Technology
http://www.dlr.de/at
Folie 9
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
10. Simulation Infrastructure
Grid Computing in Germany – D-Grid
Folie 10
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
11. German D-Grid Initiative
Grid Infrastructure for e-Science
Objectives
Building a Grid Infrastructure for science and business in Germany
Combine existing German grid activities
Integration of middleware components developed in Community Grids
Three components of the infrastructure
Networks and associated tools
Software layer for connecting tools and services
Scientific information and knowledge
Important aspects
Continuing sustainable production grid infrastructure after the end of the
funding period
Integration of additional grid communities (2./3. generation)
Business models for grid services
Folie 11
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
12. D-Grid Projects
Community Grid Projects
User Services (Portals)
PartnerGrid
BauVOGrid
Astro-Grid
HEP-Grid
MediGrid
AeroGrid
Business Services, SLAs, SOA Integration, Virtualization
TextGrid
WISENT
C3-Grid
In-Grid
●●●
Knowledge Management
Grid Middleware und Grid Services
Integration Project
Folie 13
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
13. General D-Grid Architecture
User/Application
Application
development GAT API GridSphere
and Plug-In
user access
Nutzer
Scheduling
Workflow Management
higher
Grid Monitoring
Functions
Datenmanagement LCG/gLite UNICORE
Accounting
Billing
Base User/VO-Mngt
services Globus
Security
Network-
distributed distributed
Data/ infrastructure
data archives compute
Software
Resources
Folie 14
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
14. Specifics of the D-Grid Architecture
D-Grid supports three middleware and two data access protocols.
Combines requirements of all communities.
Goal: Support of all middleware systems by all resource providers
Folie 15
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
15. D-Grid Infrastructure (2007)
Folie 16
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
16. AeroGrid Project
Folie 17
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
17. AeroGrid
Project Data
Grid-based cooperation between industry,
research centres, and universities in
aerospace engineering
Runtime: April 1, 2007 – March 30, 2010
Website: http://www.aero-grid.de
Folie 18
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
18. AeroGrid
Use Cases and Project Goals
Usage scenarios
Use of computing resources via the AeroGrid
Collaboration in designing engine components
Co-operative further development of TRACE code
Project goals
Allow cooperation in research and development projects
Use of up-to-date program versions, data, and compute resources
across all locations
Detailed documentation of history of a computational process that leads
to a certain result (“Provenance”)
Folie 21
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
19. Service-Provider A
Service-Provider A
WebDAV/
WebDAV/ Resources
Resources
AFS/SRB/
AFS/SRB/
Cooperation in AeroGrid GridFTP/…
GridFTP/…
UNICORE6
UNICORE6
Data/ Tools
User
User GridSphere
GridSphere Meta data
Data
Data Server
Server
Code
Code Management
Management
Developer
Developer Tool
Client
Client
Workflow
Workflow
Service*
Service*
CPU Resources
Grid Interfaces
Web
Web Service-Provider B
Service-Provider B
Portal
Portal
Simulation
Simulation Resources
WebDAV/
WebDAV/ Resources
User
User AFS/SRB/
AFS/SRB/
GridFTP/…
GridFTP/…
UNICORE6
UNICORE6
Data/ Tools
Workflow
Workflow GridSphere
GridSphere Meta data
Client*
Client* Server
Server
Workflow
Workflow
Service*
Service*
* Workflow service and client are not part of the project. CPU Resources
They will be added for later user communities.
...
Folie 22
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
20. Provenance
Origin and Authenticity of Results
Provenance of computational processes
Recorded documentation of processes as they take place
With this documentation, we can determine:
the origin of electronic data and
the compliance of the process that led to the data
Definition of provenance
The Provenance of some information is the history of its creation
Related Terms
“Data Lineage” is another word for Provenance
Provenance is NOT “Logging”
Folie 23
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
21. Provenance
Example Queries
Given some data item, what was the simulation case?
Given some parameter, in what simulation(s) has it been used?
What data has been recorded in a simulation with a specific parameter?
What simulations have been run using a given model (aircraft design)?
Given two/more simulations with the same setup, what is the result and
the difference in provenance?
What have been the initial conditions for this simulation?
What were the termination criteria?
How often has the workflow been executed?
What specific algorithm/version has been used in computations?
What were the hosts participating in the simulation run?
Folie 24
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
22. Provenance
Benefits for Aerospace Applications
Clean documentation of distributed simulation workflows
Ability to analyze and reason over all conducted computations
Understanding of computations and their results
Tool based support for analyses
Allows checking for requirements
Compliance to legal and business regulations
Proof and reproducibility of results
Folie 25
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
23. Example
Provenance Information in Simulations
Process d2
control File-Server
m1
i0 c-4
c4
c1 c-1 d1
c2 c-2 c3 c-3
Pre- i1 Parameter i2 Simulation i4
Visualization
Processing variation
i3 Relationen:
Interactions:
- r0: i0 causes i1
Configuration State - r1: i1 causes i2
of the Actors - r2: i2 causes i3
Process flow - r3: i2 causes i4
Monitoring - r4: i3 causes i2
- r5: i2 causes m1
Data management - r6: i2 causes d1
- r7: i0 causes d2
Folie 27
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
24. AeroGrid User Interfaces
Portal
Web-basaed access
Devlopment based on GridSphere
Client applications
Automation of recurring tasks
Integration in existing working
environments
Folie 28
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
25. Data Management Client Software
DataFinder
Folie 29
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
26. DataFinder
Short Overview
DataFinder
Efficient management of scientific and technical data
Focus on huge data sets
Development of the DataFinder by DLR
Available as Open-Source-Software
Primary functionality
Structuring of data through assignment of meta information and self-
defined data models
Complex search mechanism to find data
Flexible usage of heterogeneous storage resources
Integration in the working environment
Folie 30
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
27. DataFinder
Uses
Large-scale simulations
aerodynamics
material science
climate
…
Measured data
wind-tunnel experiments
earth observations
traffic data
…
Folie 31
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
28. DataFinder Overview
Basic Concept
Client-Server solution
Based on open and stable standards, such as XML and WebDAV
Extensive use of standard software components (open source /
commercial), limited own development at client side
Folie 32
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
29. DataFinder
Mass Data Storage using “Data Stores“
Storage
Logical View User Client Locations
“Data Stores”
External Medias
Folie 34
(CD, DVD,…)
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
30. Data Structure
Mapping of Organizational Data Structures
Object
Object (file)
(collection)
Relation File 1 Project Mega
Code Ultra
User Eddie
Simulation I
Key Value
Project Mega
Code Ultra
User Eddie
Key Value
File 2 Project Mega
Code Ultra
User Eddie
Project A Simulation II
Key Value
Project Mega Project Mega
Code Ultra Code Ultra
User Eddie User Eddie
Key Value Key Value
Experiment
User Project Mega
Project B
Project Mega
Project
Code
User
Key
Mega
Ultra
Eddie
Value
Code Ultra Code Ultra
User Eddie User Eddie
Key Value Key Value
Attributes
Project C
Project
Code
User
Key
Mega
Ultra
Eddie
Value
(meta data)
Folie 36
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
31. DataFinder in AeroGrid
Turbine Simulation
Folie 38
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
32. Turbine Simulation
Data Model
Simulation steps (example):
1. splitCGNS
Preparing data for TRACE
2. TRACE (CFD solver)
Main computation
3. fillCGNS
Conflating results
4. Post Processing
Data reduction and
visualization
Automation with customized
DataFinder
Folie 39
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
33. Turbine Simulation: Graphical User Interface
Folie 40
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
34. Turbine Simulation: Customized GUI Extensions
1
2
1. Create new simulation
2. Start a simulation
3. Query status
3
4. Cancel simulation
5. Project overview
4
5
Customization based
on user requirements!
Folie 41
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
35. Turbine Simulation
Starting External Applications
1. CGNS Infos / ADFview / CGNS Plot
2. TRACE GUI
3. Gnuplot
1
2
Integration with
3 other tools!
Folie 42
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
36. Integration Platform and Workflow Management
Reconfigurable Computing Environment (RCE)
Folie 43
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
37. Beyond DataFinder…
Use cases for “lightweight” tools like DataFinder
Simple recurring tasks
Easy and fast customization (but “static” workflows!)
Instant access to distributed storage resources
Easy installation
Additional complexity requires more sophisticated systems
Dynamic (large) workflows
Complex configurable user interface
Use of existing services
Folie 44
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
38. Reconfigurable Computing Environment (RCE)
The RCE general purpose integration platform
Used to manage collaborative engineering processes
Provision of commonly used software components
Data management, graphical user interface, distributed computing,
workflow component
Concentration on the application specific software components
Developed as a service oriented software framework
Using OSGi as service platform (for Java)
Eclipse for graphical user interface
Folie 45
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
39. RCE Architecture
Macro
Notification
Config.
Ext. Code
Application Server
SOAP
Communication
RMI
Application
Privilege
Wrapper
Broker
Data
Application
GUI
… …
… Reconfigurable Computing Environment
OSGi/Eclipse Equinox
Java Virtual Machine
Folie 46
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
40. RCE
DB Server
Distributed System
Communication
SOAP
Applic. Server
RMI
Privilege
Broker
Data
Application
…
RCE
GUI OSGi/Eclipse Equinox
Java Virtual Machine
Communication SOAP
Applic. Server
RMI
Privilege
Broker
Data
GUI
…
Ext. Code
RCE
Communication
SOAP
Applic. Server
Simulation
OSGi/Eclipse Equinox RMI
Privilege
Wrapper
Broker
Data
Java Virtual Machine
…
RCE
OSGi/Eclipse Equinox
Java Virtual Machine Simulation
Desktop
Folie 51
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
41. GUI Example
Shipbuilding Industry
Folie 52
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
42. Use Cases at DLR
RCE as integration platform
IMENS (Multi disciplinary design of space re-entry vehicles)
Simulation of hot structures
RCE-based environment for coupled,
distributed simulation in Grids
CEF (Concurrent Engineering Facility)
Collaborative early design of space vehicles
Mission planning
Experts for all disciplines in one room
RCE-based software infrastructure
Reuse of existing Excel-spreadsheets
XPS4CFD (Expert System for CFD)
Folie 54
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
43. Integration with Microsoft Excel
If you cannot avoid Excel:
Solution 1: Use Excel, integrate interfaces to computational resources
into Excel
Solution 2: Integrate Excel into execution environment and
workflow system
RCE-Excel-Integration
Excel user interface integrated
Excel is nice!
into RCE/Eclipse It's ugly to use, but nice.
Excel files stored in RCE data And in the end all data
management
Sharing data with other RCE ends in Excel.
Hans Rosling
components
Folie 55
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
44. Folie 56
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
45. Folie 57
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
46. Expert System for Aerospace Engineers
XPS4CFD
Folie 58
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
47. XPS4CFD (1)
Expert System for Aerospace Engineers
Assistance of users of DLRs CFD-software TAU
Provide best-practices and guidelines, depending on specific problems
and facts
Rule-based system using JBoss Drools as the rule engine
Eclipse Rich Client Platform application, based on RCE
Folie 60
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
48. XPS4CFD (2)
Expert System for Aerospace Engineers
Combined with a Lucene search engine for finding relevant documents
Easily maintainable and extensible by experts, not programmers
Forms and graphical editors for adding, changing and deleting
rules, workflows and documents
Generators for automatic creation of the fact model and UI
elements like interview-components
Folie 61
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
49. Choosing and Planning a Scenario
Folie 62
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
50. Guiding Users Through a TAU Workflow (1)
Folie 63
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
51. Guiding Users Through a TAU Workflow (2)
Folie 64
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
52. Guiding Users Through a TAU Workflow (3)
Folie 65
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
53. Creating Rules
Folie 66
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
54. Creating Forms
Folie 67
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
55. Creating Workflows
Folie 68
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
56. Expert System
Additional Use Cases (related to space/aerospace)
Health monitoring
Monitoring health status of
astronauts and pilots
Managing knowledge about
irregular health parameters
Send alarms
Astronaut assistance
Assist astronauts in standard
procedures and problem situations
Configurable for all subsystems of
space crafts
Folie 69
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
57. Conclusion
Putting All Together…
Expert
Expert feedback analysis
User System
System
Shares Knowledge
Uses Knowledge generate workflow
Works with data
Data &
Data &
Search meta data Workflow
trace Provenance
Workflow user
Select resources Management
Management action Store
execute workflow
Software tools
trace
Generate workflow description workflow
execution
Executes workflows in Grids
Record Provenance info
Folie 70
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008
58. Questions?
Folie 71
Korea e-Science AHM 2008 > Andreas Schreiber > Data and Knowledge Management in Aerospace Engineering > 08.09.2008