BPM is established, tools are stable, many companies use it successfully. However, today's business processes are based on data from relational databases or web services. Humans make decisions due to this information. Companies also use business intelligence and other tools to analyze their data. Though, business processes are executed without access to this important information because technical challenges occur when trying to integrate big masses of data from many different sources into the BPM engine. Additionally, bad data quality due to duplication, incompleteness and inconsistency prevents humans from making good decisions. That is status quo. Companies miss a huge opportunity here! This session explains how to achieve intelligent business processes, which use big data to improve performance and outcomes. A live demo shows how big data can be integrated into business processes easily - just with open source tooling. In the end, the audience will understand why BPM needs big data to achieve intelligent business processes.

1. Next-Generation BPM –
How to create intelligent Business Processes
thanks to Big Data
Talend,
Global
Leader
in
Open
Source
Integra7on
Solu7ons
Kai Wähner
kwaehner@talend.com
@KaiWaehner
Xing / LinkedIn
www.kai-waehner.de

2. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Consulting
Developing
Coaching
Speaking
Writing
Main Tasks
Requirements Engineering
Enterprise Architecture Management
Business Process Management
Architecture and Development of Applications
Service-oriented Architecture
Integration of Legacy Applications
Cloud Computing
Big Data
Contact
Email: kontakt@kai-waehner.de
Blog: www.kai-waehner.de/blog
Twitter: @KaiWaehner
Social Networks: Xing, LinkedIn
Kai Wähner

3. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Key messages
BPM should be used (just) for optimizing business processes!
Intelligent business processes need big data and integration!
Big data will reduce human interactions in BPM further!

4. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
• Big
data
paradigm
shiL
• Use
cases
for
big
data
• Intelligent
business
processes
• Technology
and
product
perspec7ve
• Live
demo
Agenda

5. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
• Big
data
paradigm
shiL
• Use
cases
for
big
data
• Intelligent
business
processes
• Technology
and
product
perspec7ve
• Live
demo
Agenda

6. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
William
Edwards
Deming
(1900
–1993)
American
sta7s7cian,
professor,
author,
lecturer
and
consultant
“If
you
can't
measure
it,
you
can't
manage
it.”
Why should you care about big data?

7. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
è „Silence
the
HiPPOs“
(highest-­‐paid
person‘s
opinion)
è Being
able
to
interpret
unimaginable
large
data
stream,
the
gut
feeling
is
no
longer
jus7fied!
Why should you care about big data?

8. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Why should you care about big data?
“Accessing
data
is
now
[again]
the
cri7cal
path
in
making
good
decisions!”
h`p://www.inforbix.com/friday-­‐data-­‐stories-­‐big-­‐data-­‐driven-­‐decision-­‐making/

9. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Why does big data exist?
Changing Interactions
Changing Scale
Changing Expectations
Cloud
Sensors

10. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
The Vs of big data
Volume
(terabytes,
petabytes)
Variety
(social
networks,
blog
posts,
logs,
sensors,
etc.)
Velocity
(real7me
or
near-­‐
real7me)
Value

11. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Big
Data
Integra3on
– Land
data
in
a
Big
Data
cluster
– Implement
or
generate
parallel
processes
Big
Data
Manipula3on
– Simplify
manipula7on,
such
as
sort
and
filter
– Computa7onal
expensive
func7ons
Big
Data
Quality
&
Governance
– Iden7fy
linkages
and
duplicates,
validate
big
data
– Match
component,
execute
basic
quality
features
Big
Data
Project
Management
– Place
frameworks
around
big
data
projects
– Common
Repository,
scheduling,
monitoring
Big data tasks to solve - before analysis

12. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
• Big
data
paradigm
shiL
• Use
cases
for
big
data
• Intelligent
business
processes
• Technology
and
product
perspec7ve
• Live
demo
Agenda

13. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
h`p://www.slideshare.net/brocknoland/common-­‐and-­‐unique-­‐use-­‐cases-­‐for-­‐apache-­‐hadoop
Replacing ETL jobs: Binary files

14. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
h`p://hkotadia.com/archives/5021
Deduce
Customer
Defec7ons
Forecast: Risk management

15. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
➜ With
revenue
of
almost
USD
30
billion
and
a
network
of
800
loca7ons,
Macy's
is
considered
the
largest
store
operator
in
the
USA
➜ Daily
price
check
analysis
of
its
10,000
ar7cles
in
less
than
two
hours
➜ Whenever
a
neighboring
compe7tor
anywhere
between
New
York
and
Los
Angeles
goes
for
aggressive
price
reduc7ons,
Macy's
follows
its
example
➜ If
there
is
no
market
compe7tor,
the
prices
remain
unchanged
h`p://www.t-­‐systems.com/about-­‐t-­‐systems/examples-­‐of-­‐successes-­‐companies-­‐analyze-­‐big-­‐data-­‐in-­‐record-­‐7me-­‐l-­‐t-­‐systems/1029702
Logistics: Flexible pricing

16. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Content
manipula7on
at
a
traveling
portal
–
Which
customer
reviews
are
trustworthy?
Jean-­‐Paul
Schmetz,
Harvard
Business
Manager,
November
2012
Fraud detection: Fraud mining

17. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
➜ A
lot
of
data
must
be
stored
„forever“
➜ Numbers
increase
exponen7ally
➜ Goal:
As
cheap
as
possible
➜ Problem:
(Fast)
queries
must
s7ll
be
possible
➜ Solu7on:
Commodity
servers
and
„Hadoop
querying“
Global
Parcel
Service
h`p://archive.org/stream/BigDataImPraxiseinsatz-­‐SzenarienBeispieleEffekte/Big_Data_BITKOM-­‐Leisaden_Sept.2012#page/n0/mode/2up
Storage: Compliance

18. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
➜ How
do
you
put
this
big
data
easily
in
the
hands
of
the
people
that
need
it?
➜ Making
the
data
“ac7onable”
is
the
real
challenge.
➜ Seeing
the
informa7on
that
helps
make
a
decision
on
a
composite
dashboard
is
just
the
first
step
and
where
too
many
companies
stop.
➜ A
business
must
be
able
to
fire
off
the
business
process
to
execute
the
decision
made
regarding
the
data.
Great big data use cases, but ...
Kai Wähner
h`p://smartdatacollec7ve.com/ma`-­‐davies/104576/data-­‐driven-­‐bpm-­‐making-­‐big-­‐data-­‐ac7onable

19. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
• Big
data
paradigm
shiL
• Use
cases
for
big
data
• Intelligent
business
processes
• Technology
and
product
perspec7ve
• Live
demo
Agenda

20. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
➜ Humans
have
to
interpret
large
data
to
make
decision.
Using
gut
feeling
is
nothing
but
gambling.
➜ Just
doing
big
data
analy7cs
is
not
enough.
Systema7c
and
monitored
human
interac7ons
are
as
important
to
get
best
outcomes.
➜ An
intelligent
business
process
combines
big
data
and
BPM.
This
enables
humans
to
make
data-­‐driven
decisions
based
on
big
data
analy7cs.
Intelligent business processes

21. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
➜ Process
starts
ac7on
(Big
Data
PULL)
• Manual
or
automated
• Faster
responses
(e.g.
„catching
Osama“)
• Be`er
outcomes
(e.g.
„recommenda7on
engine“)
➜ Data
starts
ac7on
(Big
Data
PUSH)
• (Usually)
automated
• Predic7ve
processes
(e.g.
„preven7ng
flu
epidemic“)
• Handle
before
it
happens
(e.g.
„customer
deduc7on“)
Intelligent business processes

22. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
➜ How
are
they
related?
➜ How
to
combine?
➜ How
to
realize
this
technically?
Combination of big data and BPM

23. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
increase efficiency
better quality
reduce costs
increase flexibility
enable new business models
Why BPM?
Kai Wähner
Business-IT-Alignment

24. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
• long-running stateful workflows
• frequently changing processes
• human interaction
When BPM?
Kai Wähner

25. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
How BPM?
Kai Wähner
Script Task Service Task
Groovy
JavaScript
etc.
SOAP Web Service
Everything
from Cobol
to Ruby...
... or a
„big data
service“

26. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Challenge
Kai Wähner
Separation of Concerns

27. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Separation of Concerns
Kai Wähner
Integration
• ETL
• Connectivity / adaptors to connect to external
systems using a variety of different protocols
• Predefined EIP for message routing
Big Data
• Processing
• Analytics
BPM
• Do queries to make decisions
• Human or machine
C
R
E
A
T
E
U
S
E

28. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
• Big
data
paradigm
shiL
• Use
cases
for
big
data
• Intelligent
business
processes
• Technology
and
product
perspec7ve
• Live
demo
Agenda

29. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Kai Wähner
Integration
• ETL
• Connectivity / adaptors to connect to external
systems using a variety of different protocols
• Predefined EIP for message routing
Big Data
• Processing
• Analytics
BPM
• Do queries to make decisions
• Human or machine
Technical / product perspective

30. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Enterprise Integration Patterns

31. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Enterprise Integration Patterns

32. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
AmazonS3
s3
=
new
AmazonS3Client(new
Proper7esCreden7als(
S3Sample.class.getResourceAsStream("AwsCreden7als.proper7es")));
String
bucketName
=
"my-­‐first-­‐s3-­‐bucket-­‐"
+
UUID.randomUUID();
String
key
=
"MyObjectKey";
try
{
s3.createBucket(bucketName);
s3.putObject(new
PutObjectRequest(bucketName,
key,
createSampleFile()));
S3Object
object
=
s3.getObject(new
GetObjectRequest(bucketName,
key));
ObjectLis7ng
objectLis7ng
=
s3.listObjects(new
ListObjectsRequest()
.withBucketName(bucketName)
.withPrefix("My"));
s3.deleteObject(bucketName,
key);
s3.deleteBucket(bucketName);
}
catch
(AmazonServiceExcep7on
ase)
{
//
error
handling...
}
catch
(AmazonClientExcep7on
ace)
{
//
error
handling...
}
Integration framework

33. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
// Producer
from(“ftp:toS3")
.setHeader(S3Constants.KEY, simple(“order.txt"))
.to("aws-s3://myBucket?accessKey=" + a+ "&secretKey= " + s)
// Consumer
from(„salesforce://orders__c?user=dummy1“)
.filter(„customer == ${dummyCustomer})
.to(“ibm-database:orderData")
Integration framework (e.g. Apache Camel)

34. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Enterprise Service Bus (e.g. Talend ESB)

35. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Connectivity
Routing
Transformation
Complexity
of Integration
Enterprise
Service
Bus
Integra7on
Suite
Low High
Integration
Framework
INTEGRATION
Tooling
Monitoring
Support+
BUSINESS PROCESS MGT.
BIG DATA / MDM
REGISTRY / REPOSITORY
RULES ENGINE
„YOU NAME IT“
Alternatives for integration
+

36. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Kai Wähner
Integration
• ETL
• Connectivity / adaptors to connect to external
systems using a variety of different protocols
• Predefined EIP for message routing
Big Data
• Processing
• Analytics
BPM
• Do queries to make decisions
• Human or machine
Technical / product perspective

37. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Technology perspective
How
to
process
big
data?

38. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Slides:
h`p://www.slideshare.net/pavlobaron/100-­‐big-­‐data-­‐0-­‐hadoop-­‐0-­‐java
Video:
h`p://www.infoq.com/presenta7ons/Big-­‐Data-­‐Hadoop-­‐Java
How to process big data?

39. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
The
defacto
standard
for
big
data
processing
How to process big data?

40. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Even
MicrosoL
(the
.NET
house)
relies
on
Hadoop
since
2011
How to process big data?
“A
big
part
of
[the
company’s
strategy]
includes
wiring
SQL
Server
2012
(formerly
known
by
the
codename
“Denali”)
to
the
Hadoop
distributed
compu7ng
plasorm,
and
bringing
Hadoop
to
Windows
Server
and
Azure”

41. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Apache
Hadoop,
an
open-­‐source
soLware
library,
is
a
framework
that
allows
for
the
distributed
processing
of
large
data
sets
across
clusters
of
commodity
hardware
using
simple
programming
models.
It
is
designed
to
scale
up
from
single
servers
to
thousands
of
machines,
each
offering
local
computa7on
and
storage.
What is Hadoop?

42. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
How to process big data?

43. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
MapReduce
HDFS
Ecosystem
Features
included
Hadoop
Distribu7on
Integra7on
Suite
few many
Apache
Hadoop
Packaging
Deployment-Tooling
Support
+
Tooling / Modeling
Code Generation
Scheduling
Other Tools (ESB, BPM, ...)
+
Hadoop alternatives

44. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Kai Wähner
Integration
• ETL
• Connectivity / adaptors to connect to external
systems using a variety of different protocols
• Predefined EIP for message routing
Big Data
• Processing
• Analytics
BPM
• Do queries to make decisions
• Human or machine
Technical / product perspective

45. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Standards
Kai Wähner
BPM
BPEL BPMN
XPDL
WF-XML
BPEL4People
jPDL
ARIS EPC

46. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
BPMN
Kai Wähner
„Business Process Model and Notation (BPMN) is a graphical
representation for specifying business processes in a business process
model.“
Wikipedia

47. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
• BPMN is a standard notation for designing business processes
(versus: UML is a standard modeling language best suited for designing and
implementing software)
• not just flow charts! sufficient restrictions / constraints => executable!
• standardized XML format
• orchestration and choreography
• extension points => add specific needs without breaking interoperability
• optional mapping of a BPMN subset to BPEL
(restricted to block-structured flows without cycles)
BPMN 2.0
Kai Wähner

48. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Alternatives for BPM (technology perspective)
Kai Wähner
Complexity of
Orchestration
Low High
BPM
Framework
BPM
Suite
Integra7on
Suite
Coding
Service Tasks
Human Interaction
GUI
BPM
Tooling
Monitoring
Support
+
ESB
BIG DATA / MDM
REGISTRY / REPOSITORY
RULES ENGINE
„YOU NAME IT“
+

49. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Kai Wähner
Integration
• ETL
• Connectivity / adaptors to connect to external
systems using a variety of different protocols
• Predefined EIP for message routing
Big Data
• Processing
• Analytics
BPM
• Do queries to make decisions
• Human or machine
Let‘s
realize
it
!!!
Technical / product perspective

50. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Alternatives for creating intelligent BPs
Kai Wähner
Complexity of
Orchestration
Low High
Frameworks
Specific
Tools
Suite
of
Tools
e.g.
Talend Unified Platform
i.e ALL-IN-ONE
(Integration, Big Data, BPM)
e.g.
Camel (Integration)
Hadoop (Big Data)
Activiti (BPM)
e.g.
Mule ESB (Integration)
MapR (Big Data)
Camunda (BPM)

51. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Custom combination of integration, big data and BPM?
Kai Wähner
• A lot of glue code
• Testing
• Bugfixing
• No support
Some other people already had
the problems you would have!

52. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
• Big
data
paradigm
shiL
• Use
cases
for
big
data
• Intelligent
business
processes
• Technology
and
product
perspec7ve
• Live
demo
Agenda

53. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Live demo
Complexity of
Orchestration
Low High
Frameworks
Specific
Tools
Suite
of
Tools
e.g.
Talend Unified Platform
i.e ALL-IN-ONE
(Integration, Big Data, BPM)
e.g.
Camel (Integration)
Hadoop (Big Data)
Activiti (BPM)
e.g.
Mule ESB (Integration)
MapR (Big Data)
Camunda (BPM)

54. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Data
Quality
Data
Integration MDM ESB
Big
Data
Big Data
¾ Open
source
license
¾ Free
of
charge
¾ OpDonal
support
¾ Based
on
open
source
projects
such
as
Eclipse
or
Apache
Camel,
CXF,
Hadoop
¾ Commercial
license
¾ SubscripDon
model
¾ Support
included
Data
Quality
Data
Integration MDM ESB BPM
Talend Unified Platform

55. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
„Talend
Integra7on
Suite“
in
ac7on...
Live demo

56. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Example (Talend): Integration

57. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Example (Talend): Big Data Processing

58. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Example (Talend): Business Process

59. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Example (Talend): Big Data Web Service

60. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Did you get the key message?

61. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Key messages
BPM should be used (just) for optimizing business processes!
Intelligent business processes need big data and integration!
Big data will reduce human interactions in BPM further!

62. ©
Talend
2013
“How
to
create
intelligent
Business
Processes
thanks
to
Big
Data”
by
Kai
Wähner
Did you get the key message?

63. Thank you for your attention.
Questions?
KAI WÄHNER
kwaehner@talend.com
www.kai-waehner.de
LinkedIn / Xing
@KaiWaehner