Undertaking a digital journey starts with clearly articulating the success factors for the entire digital journey, and our experience from the field has shown it to be an Achilles heel for most CXOs, across Fortune 500 organizations. Our findings were corroborated when a Mckinsey study reported that only 15% of the organizations are able to calculate the ROI of a digital initiative. In this talk we will deliberate on demonstrated examples from multi-billion dollar businesses around proven methodologies to measure the value of a digital enterprise. The panel will share experiences as well as provide actionable advice for immediate next steps around the following: Successful metrics for measuring the value for Digital / IoT / AI/ Machine learning engagements How can 'Digital Traction Metrics' help with actionable insights even before the Financial Metrics have been reported What are the best in-class organizational constructs and futuristic employee engagement methods to facilitate the digital revolution Panelists for this session include: • Christian Bilien - Head of Global Data at Societe Generale • Pierre Alexandre Pautrat – Head of Big Data at BPCE/Nattixis • Ronny Fehling – VP , Airbus • Juergen Urbanski – Silicon Valley Data Science • Abhas Ricky - EMEA Lead, Innovation & Strategy, Hortonworks

1. Core Intel
Krzysztof Adamski
On the bank secret service
Munich 6th April 2017

2. Are security breaches common?
https://www.gov.uk/government/uploads/system/uploads/attachment_data/file/4
32412/bis-15-302-information_security_breaches_survey_2015-full-report.pdf

3. Carbanak
3
https://securelist.com/blog/research/68732/the-great-bank-robbery-the-carbanak-apt/

4. Core Intel is a part of ING Cyber Crime Resilience Programme
to structurally improve the capabilities for the cybercrime
• prevention
• detection and the
• response
CoreIntel
4

5. • Measures against e-banking fraud, DDoS and Advanced Persistent Threats (APTs).
• Threat intelligence allow to respond to, or even prevent, a cybercrime attack
• (This kind of intelligence is available via internal and external parties and includes both
open and closed communities)
• Monitoring, detection and response to “spear phishing”
• Detection/mitigation of infected ING systems’
• Baselining network traffic/anomaly detection
• Response to incidents (knowledge, tools, IT environment)
• Automated feeds, automated analysis and historical data analysis
The reasoning
5

6. What is there on the market nowadays?
6

7. The world is not enough

8. So the challenge is…

9. Market leaders Benelux
Growth markets
Commercial Banking
Challengers
Most of our data is within Europe
9

10. Market leaders Benelux
Growth markets
Commercial Banking
Challengers
but we operate globally
10

11. Expect the unexpected to collect all the data
11

12. • What kind of data do we need?
• Where is our data located?
• How we can potentially capture it?
• What are the legal implications?
So there is a challenge to capture „all” the data
12

13. Core Intel architecture

14. So what you would like to see is…
Photo credit: edgarpierce via Foter.com / CC BY

15. …In fact it is slightly more complicated

16. All has its own purpose. Let’s see in details.
16
Photo credit: https://www.pexels.com/photo/dslr-camera-equipments-147462/

17. Local data collector
17

18. But tell how to capture that data
18
https://observer.viavisolutions.com/includes/popups/taps/tap-vs-span.php

19. Broker settings:
Replication factor >= 3
min.insync.replicas = 2
unclean.leader.election.enable = false
replica.lag.time.max.ms
Producer settings:
acks = all
retries = Integer.MAX_VALUE
max.block.ms = Long.MAX_VALUE
block.on.buffer.full = true
To have data in order
max.in.flight.requests.per.connection = 1
Kafka producer configuration (as we don’t like losing data)
19

20. Central data collector
20

21. Time is crucial here
21
Photo credit: Cargo Cult via Foter.com / CC BY

22. But your business data more, so proceed with caution
22
Photo credit: https://www.pexels.com/photo/white-caution-cone-on-keyboard-211151/

23. • Network bandwidth control
• quota.consumer.default
• quota.producer.default
Kafka mirror maker configuration
23

24. Secure data:
listeners=SSL://host.name:port
ssl.client.auth=required
ssl.keystore.location
ssl.keystore.password
ssl.key.password
ssl.truststore.location
ssl.truststore.password
Kafka mirror maker configuration
24
Secure
data in
transit

25. Streaming data
25

26. spark.yarn.maxAppAttempts
spark.yarn.am.attemptFailuresValidityInterval
spark.yarn.max.executor.failures
spark.yarn.executor.failuresValidityInterval
spark.task.maxFailures
spark.hadoop.fs.hdfs.impl.disable.cache
spark.streaming.backpressure.enabled=true
spark.streaming.kafka.maxRatePerPartition
Spark on yarn streaming configuration
26

27. In memory data grid
27
val rddFromMap = sc.fromHazelcastMap("map-name-to-be-loaded")

28. Let’s find something in these logs
28
Photo credit: https://www.flickr.com/photos/65363769@N08/12726065645/in/pool-555784@N20/

29. Matching
29
Tornado - a Python web framework and asynchronous
networking library - http://www.tornadoweb.org/
MessagePack – binary transport format
http://msgpack.org/

30. • Automatically & continually match network logs <->threat intel
• When new threat intel arrives, against full history network logs
• When new network logs arrive, against full history threat intel
• Alerts are shown in a hit dashboard
• Dashboard is a web-based interfaces that provide flexible charts, querying, aggregation
and browsing
• Quality/relevance of an alert is subject to the quality of IoC feeds and completeness of
internal log data.
Hit, alerts and dashboards
30

31. Be smart with your tooling
31
Photo credit https://www.flickr.com/photos/12749546@N07/

32. and leverage e.g. elasticsearch templates
32

33. Data mapping:
- doc_value
- fielddata
- fields
Cluster settings to check:
gateway.recover_after_nodes
gateway.recover_after_master_nodes
gateway.recover_after_data_nodes
indices.recovery.max_bytes_per_sec
indices.breaker.total.limit
indices.breaker.fielddata.limit
Elasticsearch configuration
33

34. For those who know how to use heavy equipment
34
Photo credit: News Collection & Public Distribution @techpearce2 via Foter.com / CC BY

35. Long data storage - HDFS
35

36. Kafka offset management
36

37. Core Intel allows users to perform advanced analytics on network logs using a set of
powerful tools
• Spark API to write code to process large data sets on a cluster
• perform complex aggregations to collect interesting statistics
• run large scale clustering algorithms with Spark’s MLLib
• run graph analyses on network logs using Spark’s GraphX
• transform and extract data for use in another system (which are better for specific analytics or
visualization purposes)
• Kafka, co you can write own Consumers and Producers to work with your data
• to perform streaming analysis on your data
• to implement your own alerting logic
• Toolset
• Programming languages: Scala, Java, Python
• IDE’s: Eclipse / Scala IDE, IPython Notebook and R Studio
Advanced analytics
37

38. How do we schedule the jobs
38

39. How to keep everything under control
39
Photo credit: https://www.flickr.com/photos/martijn141

40. Monitoring crucial points in your data pipeline
40

41. Something for smart guys
41
Photo credit: https://www.flickr.com/photos/jdhancock/5173498203/

42. Plenty of data to analyze
42

43. Upcoming challenges on the operations side
43

44. OpenShift HA deployment
44

45. Shaken, not stirred?
45

46. 46

47. Follow us to stay a step ahead
ING.com
YouTube.com/ING
SlideShare.net/ING@ING_News LinkedIn.com/company/ING
Flickr.com/INGGroupFacebook.com/ING