Workflow Management for Multiple Workspaces

3. Workflow Management What are we talking about? Using the power of FME Workbench to design workflows  Moving and Copying Files  Data Transformation  Quality Control  Database Updates  Notification (Emails, Mobile Alerts)  Etc.

4. Workflow Management Why do we care?  Because many of you are doing this already!  Users have noticed that FME Workbench is a powerful process control tool  Many of you running complex workflows with multiple workspace already  FME Server Survey 2012 #1 Enhancement Request was “Automated Workflows”

5. Workflow Management: 2 Case Studies 1. “Call Before you Dig” Client (FME Desktop based) 2. Norwegian Environment Agency (thanks to VAR Norkart) Please note: Demos are very loosely based on these real projects

6. Workflow Management - Key Tools  Directory and File Pathnames Reader  File Copy Writer  WorkspaceRunner  FMEServerJobSubmitter  FMEServerNotifier  Notification Service

7. Case Study #1 “Call Before You Dig” Customers (ie Utilities) submit DGN Files – 1000s of them! Need to:  Find out which territory each file belongs to  Update database with files names and territory  Copy file into directory by territory

8. Case Study #1 “Call Before You Dig”  Reading a directory – Directory Reader  Processing files with WorkspaceRunner  Copying Files with File Copy Writer

9. Case Study #2 Norwegian EA Goal: automatically add valid data from the SOSI-format to the ArcSDE operational database • geometry must be validated and inside the database before running any other database updates • split between geometryhandling, attributehandling, databasehandling • easily add other procedures. Currently working new one to take attributes from the features and stores these in another database. View the Application here: http://innsyn.naturbase.no/

10. Case Study #2 Norwegian EA SOS I-file ArcSDE Operational DB MSSQL Logging etc SOSI- file, PDF- report, text- report 1 2 3 4 Verfiy Data and Database updates Generate paths/published parameters based on attributes in SOSI-file To Do all checks: - Geometry - SOSI - Attributename/values - Exisiting Ids Create logfiles (in SOSI, txt, 2D / 3D PDF) and add geometry to ArcSDE All data verified and all logs updated To Do: - Wrap-up files from previous FMW- files into one package and send by e-mail to supplier

11. Case Study #2 Norwegian EA  Controller Workspace Running two consecutive tasks 1. Quality Control Workspace 2. Database Update Workspace  Notification at all stages (Success and Failure)

12. Thank You!  Questions?  For more information:  info@safe.com  www.safe.com

Notes de l'éditeur

Processing tasks in sequence
- Obviously data transformation is huge and most of what FME does but there are so many other things FME can do
There was a packed presentation on this in FME World Tour 2012 Stockholm, tons of people trying to do this stuffNot just power usersPerhaps admit we’ve been behind what are user are doing here
Demos are dumbed down versions of real workflowCustomer workflows are often much more complex that what we do but or goal is to show the potential
Grouped by Readers/WritersVery powerful to read all of the file names in a directoryFile copy writer lets you move things around based on attributes or anything reallyTransformersFMEServerJobSubmitter has had a very important enhancement, if you set it to WAIT it does not need another engineThis means you can for example , check data…then load data after the checking is completeNotification ServiceTell people that a process had completed, that data passed, that new data arrived, that workspace failed etc
Processing lots of files can be painful and FME can automate thisDavid Eagle once used FME to copy 100 000’s of files because it was faster that the Windows Explorer or DOS
WorkflowDirectory reader to read all file namesWorkspace runner to actually read each file and figure out where it isCopy the files into the right place for each territorDatabase record of all of this was key – need to track all files
Thank SigbjørnTillerliHerstad of Norkart for this exampleThese points came from himHe is still not happy because would prefer to use the FMEServerWorkspaceRunner in sync mode (wait) without using an engine
The goal is a sequential workflow where the second task cannot begin until the first task is doneThis is now possible with FMEServerJobSubmitterAlso in the output from the with FMEServerJobSubmitter in Controller to the output dataset from a child via the output_datasets{}.path list attributeWe can use FMEServerNotifier if a child failsIf we want to know if the controller failed – when we publish it there is Topics to Notify on Failure for any service

Workflow Management for Multiple Workspaces

Recommandé

Recommandé

Contenu connexe

Plus de Safe Software

Plus de Safe Software (20)

Dernier

Dernier (20)

Workflow Management for Multiple Workspaces

Notes de l'éditeur