The future of Delft-FEWS

1. 29 October 2015
The future of Delft-FEWS
Simone van Schijndel and many others
Deltares
Delft-FEWS2020

2. The world is changing...
29 October 2015
...and it is
going faster
and faster!
1951
Von Neumann built the
first real computer for
meteorological
computations

3. 29 October 2015
The cloud...

4. 29 October 2015
every known hurricane and tropical storm since 1851,
developed by IDVSolutions. Data from NOAA, USA
New data sources and techniques

5. 29 October 2015
Accountability becomes increasingly important

6. 29 October 2015
Prevention and adaptation

7. 29 October 2015
http://www.customerintelligence360.com/5-absolutely-stunning-big-data-visualizations/
Relations change...
and organisations with i

8. >20 years of Delft-FEWS Applications
Flood Forecasting Systems
• Running models
• Integration of (spatial) data
• Ensembles etc
Water Information Systems
• Data validation
• Data analysis
• Long records of data
Model Management platform
• Off-line scenario analysis
• Complex modelling systems
And...not to forget
Water quality
Ground water
Storm surge
Data science

9. Our starting points
Delft-FEWS is for free
Delft-FEWS is open software, but not open source
Delft-FEWS balances flexibility and stability
Delft-FEWS stimulates sharing of knowledge and data
Delft-FEWS connects organisations and stimulates collaboration
The community is essential!
29 October 2015

10. Some considerations…
Why do we have Delft-FEWS?
• Focus on good and innovative projects, but no ‘world domination’
• Sharing new knowledge and technologies
• Access to organisations with good ideas (and financial resources)
Growth is essential! We never say no, unless…
But...
Beware of the Microsoft cyclus (lack of focus)…
• Growth→increasing complexity
• Complexity → dependency of Deltares
• Dependency of Deltares → stops growth and innovation
29 October 2015

11. Delft-FEWS worldwide

12. Anno 2015...
Time for reflection
Time for consolidation for what we have achieved so far
Time to think about the future
Three aspects:
1. Market developments
2. Technical developments
3. Organisation of product management
And...attention for (more) active participation of our users
29 October 2015

13. 29 October 2015
Official establishment of the Delft-FEWS Community Strategy
Board (CSB) per 27th October 2015
• Participate actively in Delft-FEWS2020 process
• Discuss support and maintenance and other services
• Funding mechanisms and business model
• Exchange of experiences
• Representation of the community

14. Community Strategy Board, who
Organisation Represented by
Bureau of Meteorology in Australia
(BoM)
Jeff Perkins
Dutch Waterboards (DWB) Roger de Crook
Environment Agency in England (EA) Deborah Summerskill
Federal Office for the ENvironment in
Switzerland (FOEN)
Therese Buergi
National Weather Service in the USA
(NWS)
Jon Roe
Rijkswaterstaat in the Netherlands
(RWS)
Marc Phillipart
29 October 2015

15. Community Strategy Board, how
IUD
Oct ‘15
NA UD
March ‘16
AU UD
April ‘16
NL UD
June ‘16
IUD
Oct ‘16
29 October 2015
1st formal
CSB meeting
2nd CSB
meeting
January CSB
meeting
Topic: S&M
and other
services
May CSB
meeting
Topic: TBD
Sep CSB
meeting
Topic: TBD

16. Challenges in the organisation of PM
Support & Maintenance
• Effective management of an increasing
number of applications
• ICT is increasingly complex
• Maintain a solid base for Delft-FEWS
Developments
• Prioritise following plans for
Delft-FEWS2020
• Generate sufficient financial funds
to continuously develop
Delft-FEWS
Continuity in the organisation is essential!
29 October 2015

17. Delft-FEWS architecture
Delft-FEWS architecture is mainly developed in 2003; technology has evolved:
So, there is a need for a structured review of the Delft-FEWS architecture
Where do we need to focus on:
• Keep it simple, less software components (Jboss, Tomcat, ..)
• Use of standard components introduces many times more complexity!
• Where possible share ideas and knowledge with other Deltares software
systems
Do not forget:
• Why have we made certain decisions in the past, there is an idea behind!
• Watch out for buzz-words; What is the role of FEWS?
Delft-FEWS: 2015-2020

18. Developments: interesting or hype?
What is our view?
Not everything is a hype!
See the changes!
Mostly project driven!
Delft-FEWS: 2015-2020
Focus for now
1. Big data and Large Datasets
2. Open FEWS Archive
3. Thick and Thin clients
4. Cloud computing and Simplification of FEWS roll-out
5. Webservices

19. Big Data
• Large amounts of unstructured data from many sources (i.e. Twitter)
• FEWS will not use this data directly
• We expect external services/software will use BIG data, structure it (find
correlation) and provide information that can be used in FEWS
• FEWS must be able to deal with large amounts of structured data
• Trends in meteorological forecasts, satellite data, ..
Delft-FEWS: 2015-2020

20. Delft-FEWS and Large Datasets
We need to find a solution how FEWS handles large data sets
• Large number of scalar data in Forecasting System seems no problem.
• Operational meteorological forecasts (ensembles) also no problem.
• Store data for longer period in operational database can be problematic
• There must be a solution for storage of large model data sets that do not fit in
FEWS operational database
• Model states of Swan or Delft-3D, 3Di (store on file system, NoSQL)
• Model results (first process, filter and only store important information)
• It is key we connect to open standards
• In 2014 Development of new Open Archive has started
• Will play a central role in storage of large datasets
Delft-FEWS: 2015-2020

21. Delft-FEWS and Open Archive
• Development of new FEWS Archive is important and essential
• Store validated historical data in the archive
• Store historical grids in archive (i.e. Radar grids)
• Users must have direct access to archive (Seamless integration in OC)
• Fits in Open Data Strategy of many FEWS users
Delft-FEWS: 2015-2020

22. Delft-FEWS: Thick Client and Thin Client
• Discussion on Delft-FEWS Client
• Keep the FEWS Thick client; maybe redesign but keep it for the experts
• For the users the look and feel of the application must not change too
much (large investments in training)
• CITRIX and Remote Desktop solutions are more and more popular
• Will we initiate the development of a Thin client (Web-application) from
Delft-FEWS
• Sounds nice, but is complex to develop and maintain
• Large dependency on popular browsers
• Deltares has developed a Delta Data Viewer; we need to have an open
discussion if this is a general solution
• Current focus on webservices and Open-API for the development of
Web viewers by others (Lizard, HydroNet, Delta Data Viewer, ..)
Delft-FEWS: 2015-2020

23. Cloud Computing
• Remote computing through the internet (Cluster is internal!)
• Make use of free capacity of external servers
• FEWS needs to have an answer on this development!
Examples:
• 3Di as cloud solution (FEWS-3Di, how to run models remotely)
• Bring your model to the data; reduce data traffic over the internet
Solutions:
• Simple: Start a model that runs in the cloud
• Complex: Send a complete FEWS system (FSS) with config to the data and
run on a cloud computer. Minimise the data traffic and maintain a good
performance
• Important in the discussion is performance, security, robustness
Delft-FEWS: 2015-2020

24. Cloud Computing: Pilot 2015
The Deltares ID-lab is used to promote new ideas and applications developed by
Deltares (also: initiate new developments)
One of the new developments is a Proof of Concept (PoC) with Microsoft to use
the Azure cloud for FEWS. The PoC for 2015 includes several steps:
1. Roll-out FEWS “as is” in the cloud (FSS and MC+Database)
2. Investigate with the ‘Experts’ what short term developments are needed to
simplify the roll-out of FEWS in the cloud
3. Investigate if larger developments are needed to make better use of the cloud
One of the Goals of this initiative is to simplify the current FEWS components for
easier rollout of Server and Client components and automated upgrades
Delft-FEWS: 2015-2020

25. Web Services
• A webservice makes it possible to request a service from a server. This can be
a data request, execution of a task or process or send data.
• Difference between provider and consumer webservices
• Current FEWS webservice functionality will be made consistent and extended
• FEWS-PI, UM-Aquo, WaterML, Json, …
• Will support running FEWS form external servers (start workflow, modifier,
etc..).
Delft-FEWS: 2015-2020

26. 29 October 2015
Other items to consider
• Simplify Delft-FEWS configuration (partly done) and installation
• Run new model types under Delft-FEWS
• Model in the Cloud de Cloud (3Di)
• Model as a service
• Use of new data data types
• Ecologial data (already developed for Dutch waterboard Waternet)
• 3D data; new Deltares models (NetCDF Unstruc)
• Use of 2-D spectra
• Simplify error solving in Delft-FEWS
Share additional requests and ideas with us now or
by email to fews-pm@deltares.nl!!

27. 29 October 2015
Next steps
January 2015 – October 2015
• Internal discussions
• Discussions with clients and consultants in NL and abroad
• Discussion in CSB
Before March 2016:
• Design step-wise approach for review current architecture
• Further detail technical plans and prioritize resulting activities
• Explore opportunities for funding our plans
User Days next year present first results

28. En hoe doen we het samen?
Nog in te vullen
29 October 2015
To raise the water level in a fish pond
(Zhang Huan, 1997, Beijing)