2. Job Submission: Software Layers
Client
https “web service firewall”,
message authentication
and forwarding
Gateway
Web Services
https
Service: coherent chunk
of functionality exposed
(WSRF)
through a web-service
interface
Security
Atomic Additional
Services Services
Service Container
Execution Management (XNJS)
Target System Interface (TSI)
non WS
(batch) execution
systems, file systems,
databases, ...
Target systems
07/07/2009 Slide 2
3. Deployment Scenario: Workflow Services
Client
lookup Gateway
Global
Registry
lookup,
create TSS, Service
submit job, Container
transfer file
Gateway Gateway
UAS Local UAS Local UAS Local
Registry Registry Registry
Service Service Service
Container Container Container
lookup
XUUDB user XUUDB
07/07/2009 Slide 3
4. WSRF
Web Services Resource Framework
WS Resource
■ Stateful web service
■ Represented by an XML document
■ Resource properties
■ Standard methods: getter, setter, queries
■ Lifetime
Service Group
■ List of WS addresses
■ Used for Registry
WS-BaseFaults
07/07/2009 Slide 4
5. Configuration
Service Container
■ Web Services to be deployed
■ Address of the shared Registry
■ XUUDB address, “Grid Component ID“
■ Gateway address
Gateway
Everybody:
■ Connection list Security settings
(Keystore, certificate, ...)
Registry
■ Lifetime for entries
Client
■ Registry Address
07/07/2009 Slide 5
6. UNICORE Atomic Services (UAS)
Gateway
map grid
users
Target System Service Container to local
Factory (TSF) users
UNICORE Site
Target System Target System
Service (TSS) Service (TSS)
XUUDB
Security
Job Mgmnt Job Mgmnt Storage Mgmnt
Service (JMS) Service (JMS) Service (SMS)
Target System Interface (TSI) Key:
Storage Mgmnt Storage Mgmnt File Transfer reference
Service (SMS) Service (SMS) Service (FTS) file transfer
lookup
07/07/2009 Slide 6
7. UAS: Target System Factory Service
Target
1. createTSS System
Factory
3. return TSS
address
2. create
Target
Client 4. use TSS System
Service
07/07/2009 Slide 7
8. UAS: Target System Service
Abstract web service interface to target system
■ List of applications
■ Links to jobs and storages (e.g. user home)
Security
■ User authentication through XUUDB
■ Authorization: Users' target system instances and jobs are
protected by configurable XACML policy
■ Secure job submission through message signing
Extensibility
■ Virtualization
■ Exclusive resource reservation
07/07/2009 Slide 8
9. UAS: Job Management Service
Abstract web service interface to submitted jobs
■ Jobs can be accessed and controlled from anywhere
Job status (queued, running, finished, failed, ...)
Link to storage that represents the working directory (uspace)
■ Used to securely access output files
Detailed execution log, exit code of the application
Applications are abstracted: path of executable invisible
Provide a copy of the job description
■ Can be used for resubmission
Have a lifetime (like all WS-Resources)
■ Used for automatic clean-up
07/07/2009 Slide 9
10. UAS: Job Management and Storage Services
Client Target
1. submit System
Service
1.1.1 return job
address
1.1 create
3. start
Job
2. import data 2. stage-in data
Local
Filespace 4. export data USpace 4. stage-out data Remote
Storage
Spaces
07/07/2009 Slide 10
11. UAS: Storage and File Transfer Services
Storage
Management
1. importFile() /exportFile() Service
3. return FTS address
2. create
File
Transfer
Client 4. write/read data, Service
monitor
07/07/2009 Slide 11
12. UAS: File Transfer Protocols
Pluggable mechanisms
■ Both for client-server and server-server transfers
Default mechanism: Simple OGSA ByteIO
■ Sends data as SOAP messages through the full stack
■ Needs no additional ports
■ No installation effort (pure Java)
■ Performance of ~400kB/sec
Plain http: ~ 3MB/sec
GridFTP: Speed depends on line & number of parallel TCP ports
■ Drawbacks: Lots of open ports, installation effort
UDT: ~ 100MB/sec on 1Gbit/sec line, C++ Implementation
07/07/2009 Slide 12
13. Deployment Scenario: Workflow Services
trace Client
lookup
workflow Global
submit Registry
workflow Service
Container
Workflow Location
Tracer Engine Mapper
Service Container publish
Service
Container
store submit jobs
messages callback
Service Orchestrator query Information
Service Container Service
submit jobs, Service
check job status Container
UAS UAS UAS
Service Service Service collect
Container Container Container data
07/07/2009 Slide 13
15. Configurable Security Handlers
User U
Security handler chain
SSL
U = SSL partner?
Did U sign R1?
Request R1 login, group,
User: U & role of U?
Service: S Is U allowed
XUUDB to use S?
U
XACML
Policy File read
Service S
07/07/2009 Slide 15
16. Trust Delegation
User U Request R2
Consignor: W SSL Security handler chain
SSL Service: S2 W = SSL partner?
Request R1 Did W sign R2?
Request R1 Workflow
User: U User: U
Trusts: W
Engine W Trusts: W Does U trust W?
Service: S1 (offers S1) => SAML
U
U W Is U allowed
to use S2?
XUUDB
read
XACML Service S2
Policy File
07/07/2009 Slide 16
17. UNICORE as a Web Service Hosting Environment
Security
Platform independence
Lightweight and performing: Jetty, XFire
High level programming APIs => Minimal effort
Hot deployment of web services
Transparent persistence layer using relational databases
07/07/2009 Slide 17
18. Ongoing Development (Incomplete List!)
European Projects
■ Smart LM: License management
■ Phosphorus: Meta-scheduling, network reservation
■ Etics: Tool for distributed builds on different platforms
German Projects
■ D-Mon: Monitoring in the D-Grid
■ BIS-Grid: Business workflows using BPEL
■ WisNetGrid: Data Management
Other Activities at the JSC
■ Information service (GLUE 2.0)
■ Purely Java based UDT implementation
07/07/2009 ■ Improved MPI support Slide 18
