These are the slides of my talk at Devoxx 2013.
This presentation introduces the lambda expressions, how we can write them, and what they bring to us developers. The second part presents in details the new patterns introduced by the Stream and Collector APIs. These new APIs will change the way we process large collections content, enabling concurrent and parallel access, with a very simple programming model, and amazing patterns. Complex examples will show how powerful those APIs are, and how they can help us write efficient and clean code.
11. A very simple example
public class Person {
A plain old bean…
private String name ;
private int age ;
// constructors
// getters / setters
}
List<Person> list = new ArrayList<>() ;
… and a good old list
#DV13LBDS
@JosePaumard
12. Average of the ages
int sum = 0 ;
// I need a default value in case
// the list is empty
int average = 0 ;
for (Person person : list) {
sum += person.getAge() ;
}
if (!list.isEmpty()) {
average = sum / list.size() ;
}
#DV13LBDS
@JosePaumard
13. Trickier : average of the ages of people older than 20
int sum = 0 ;
int n = 0 ;
int average = 0 ;
for (Person person : list) {
if (person.getAge() > 20) {
n++ ;
sum += person.getAge() ;
}
}
if (n > 0) {
average = sum / n ;
}
#DV13LBDS
@JosePaumard
14. Trickier : average of the ages of people older than 20
int sum = 0 ;
int n = 0 ;
int average = 0 ;
for (Person person : list) {
if (person.getAge() > 20) {
n++ ;
sum += person.getAge() ;
}
}
if (n > 0) {
average = sum / n ;
}
« imperative programming »
#DV13LBDS
@JosePaumard
15. … it does not have to be like that !
select avg(age)
from Person
where age > 20
This is a description of the result
#DV13LBDS
@JosePaumard
18. map
Person
age
age > 20
sum
1st step : mapping
Mapping :
- takes a list of a given type
- gives another list of a different type
- same number of elements
#DV13LBDS
@JosePaumard
20. map
Person
filter
age
age > 20
sum
2nd step : filtering
Filtering :
- takes a list of a given type
- gives another list of a the same type
- less elements
#DV13LBDS
@JosePaumard
24. The JDK 7 way
Create an interface to model the mapper…
public interface Mapper<T, V> {
public V map(T t) ;
}
#DV13LBDS
@JosePaumard
25. The JDK 7 way
… and create an implementation …
public interface Mapper<T, V> {
public V map(T t) ;
}
public class PersonToAgeMapper implements Mapper<Person, Integer> {
public Integer map(Person p) {
return p.getAge() ;
}
}
#DV13LBDS
@JosePaumard
26. The JDK 7 way
… that could be anonymous
public interface Mapper<T, V> {
public V map(T t) ;
}
Mapper<Person, Integer> mapper = new Mapper<Person, Integer>() {
public Integer map(Person p) {
return p.getAge() ;
}
}
#DV13LBDS
@JosePaumard
27. The JDK 7 way
We can do the same for the filtering
public interface Predicate<T> {
public boolean filter(T t) ;
}
#DV13LBDS
@JosePaumard
28. The JDK 7 way
We can do the same for the filtering
public interface Predicate<T> {
public boolean filter(T t) ;
}
public class AgePredicate implements Predicate<Integer> {
public boolean filter(Integer i) {
return i > 20 ;
}
}
#DV13LBDS
@JosePaumard
29. The JDK 7 way
We can do the same for the filtering
public interface Predicate<T> {
public boolean filter(T t) ;
}
AgePredicate predicate = new Predicate<Integer>() {
public boolean filter(Integer i) {
return i > 20 ;
}
}
#DV13LBDS
@JosePaumard
30. The JDK 7 way
And for the reduction
public interface Reducer<T> {
public T reduce(T t1, T t2) ;
}
#DV13LBDS
@JosePaumard
31. The JDK 7 way
And for the reduction
public interface Reducer<T> {
public T reduce(T t1, T t2) ;
}
public class Sum implements Reducer<Integer> {
public Integer reduce(Integer i1, Integer i2) {
return i1 + i2 ;
}
}
#DV13LBDS
@JosePaumard
32. The JDK 7 way
And for the reduction
public interface Reducer<T> {
public T reduce(T t1, T t2) ;
}
Reducer<Integer> reduction = new Reducer<Integer>() {
public Integer reduce(Integer i1, Integer i2) {
return i1 + i2 ;
}
}
#DV13LBDS
@JosePaumard
33. The JDK 7 way
So the whole map / filter / reduce looks like this
1) Create 3 interfaces
public interface Mapper<T, V>
{
public V map(T t) ;
}
public interface Predicate<T> {
public boolean filter(T t) ;
}
public interface Reducer<T> {
public T reduce(T t1, T t2) ;
}
#DV13LBDS
@JosePaumard
34. The JDK 7 way
So the whole map / filter / reduce looks like this
1) Create 3 interfaces
2) Apply the pattern
List<Person> persons = ... ;
int sum =
persons.map(
new Mapper<Person, Integer>() {
public Integer map(Person p) {
return p.getAge() ;
}
})
.filter(
new Filter<Integer>() {
public boolean filter(Integer age) {
return age > 20 ;
}
})
.reduce(0,
new Reducer<Integer>() {
public Integer recude(Integer i1, Integer i2) {
return i1 + i2 ;
}
}
}) ;
#DV13LBDS
@JosePaumard
35. The JDK 7 way
So the whole map / filter / reduce looks like this
1) Create 3 interfaces
2) Apply the pattern
List<Person> persons = ... ;
int sum =
persons.map(
new Mapper<Person, Integer>() {
public Integer map(Person p) {
return p.getAge() ;
}
})
.filter(
new Filter<Integer>() {
public boolean filter(Integer age) {
return age > 20 ;
}
})
.reduce(0,
new Reducer<Integer>() {
public Integer recude(Integer i1, Integer i2) {
return i1 + i2 ;
}
}
}) ;
#DV13LBDS
@JosePaumard
37. The JDK 8 way
Let’s rewrite our mapper
mapper = new Mapper<Person, Integer>() {
public Integer map(Person person) {
return person.getAge() ;
}
}
#DV13LBDS
@JosePaumard
38. The JDK 8 way
Let’s rewrite our mapper
mapper = new Mapper<Person, Integer>() {
public Integer map(Person person) { // 1 method
return person.getAge() ;
}
}
#DV13LBDS
@JosePaumard
39. The JDK 8 way
Let’s rewrite our mapper
mapper = new Mapper<Person, Integer>() {
public Integer map(Person person) { // 1 method
return p.getAge() ;
we take }
a
}
person p
mapper = Person person
#DV13LBDS
;
@JosePaumard
40. The JDK 8 way
Let’s rewrite our mapper
mapper = new Mapper<Person, Integer>() {
public Integer map(Person person) { // 1 method
return person.getAge() ;
}
}
and then …
mapper = Person person ->
#DV13LBDS
;
@JosePaumard
41. The JDK 8 way
Let’s rewrite our mapper
mapper = new Mapper<Person, Integer>() {
public Integer map(Person person) { // 1 method
return person.getAge() ;
}
}
… return the
age of p
mapper = Person person -> person.getAge() ;
#DV13LBDS
@JosePaumard
42. The JDK 8 way
Let’s rewrite our mapper
mapper = new Mapper<Person, Integer>() {
public Integer map(Person person) { // 1 method
return person.getAge() ;
}
}
mapper = Person person -> person.getAge() ;
#DV13LBDS
@JosePaumard
43. The JDK 8 way
Let’s rewrite our mapper
mapper = new Mapper<Person, Integer>() {
public Integer map(Person person) { // 1 method
return person.getAge() ;
}
}
mapper = Person person -> person.getAge() ;
The compiler can recognize this as an implementation of
Mapper
#DV13LBDS
@JosePaumard
44. What if …
… there is more than one statement ?
mapper = Person person -> {
System.out.println("Mapping " + person) ;
return person.getAge() ;
}
The return has to be explicit
#DV13LBDS
@JosePaumard
45. What if …
…I have no return value ?
consumer = Person person -> p.setAge(p.getAge() + 1) ;
#DV13LBDS
@JosePaumard
46. What if …
…I have more than one argument ?
reducer = (int i1, int i2) -> {
return i1 + i2 ;
}
Or :
reducer = (int i1, int i2) -> i1 + i2 ;
#DV13LBDS
@JosePaumard
47. The JDK 8 way
How can the compiler recognize the implementation of
map() ?
mapper = Person person -> person.getAge() ;
#DV13LBDS
@JosePaumard
48. The JDK 8 way
How can the compiler recognize the implementation of
map() ?
mapper = Person person -> person.getAge() ;
1) There’s only one method in Mapper
#DV13LBDS
@JosePaumard
49. The JDK 8 way
How can the compiler recognize the implementation of
map() ?
mapper = Person person -> person.getAge() ;
1) There’s only one method in Mapper
2) Both the parameters and the return types are
compatible
#DV13LBDS
@JosePaumard
50. The JDK 8 way
How can the compiler recognize the implementation of
map() ?
mapper = Person person -> person.getAge() ;
1) There’s only one method in Mapper
2) Both the parameters and the return types are
compatible
3) Thrown exceptions (if any) are compatible
#DV13LBDS
@JosePaumard
51. More lambdas
Writing more lambdas becomes natural :
mapper = Person person -> person.getAge() ; // mapper
filter = int age -> age > 20 ; // filter
reducer = (int i1, int i2) -> i1 + i2 ; // reducer
#DV13LBDS
@JosePaumard
52. More lambdas
And most of the time, the compiler understands this :
mapper = person -> person.getAge() ; // mapper
filter = age -> age > 20 ; // filter
reducer = (i1, i2) -> i1 + i2 ; // reducer
The « parameter types » can be omitted
#DV13LBDS
@JosePaumard
53. Just a remark on the reduction
How does it really work ?
#DV13LBDS
@JosePaumard
64. Reduction
2 examples :
Reducer r1 = (i1, i2) -> i1 + i2 ; // Ok
Reducer r2 = (i1, i2) -> i1*i1 + i2*i2 ; // Oooops
Caveat :
the result is always reproductible in serial
it’s not in parallel
#DV13LBDS
@JosePaumard
65. So far
A lambda expression is an alternative
to write instances of anonymous inner classes
#DV13LBDS
@JosePaumard
66. Other syntaxes
Very often one writes
mapper = person -> person.getAge() ;
This syntax is also possible :
mapper = Person::getAge ; // non static method
#DV13LBDS
@JosePaumard
67. Other syntaxes
Other example :
sum = (i1, i2) -> i1 + i2 ;
sum = Integer::sum ; // static method, new !
Or :
max = (i1, i2) -> i1 > i2 ? i1 : i2 ;
max = Integer::max ; // static method, new !
#DV13LBDS
@JosePaumard
68. Other syntaxes
Other example :
toLower = String::toLowerCase ;
// !!!! NO NO NO !!!!
toLowerFR = String::toLowerCase(Locale.FRANCE) ;
#DV13LBDS
@JosePaumard
71. Questions :
What model for a lambda ?
Can I put a lambda in a variable ?
#DV13LBDS
@JosePaumard
72. Questions :
What model for a lambda ?
Can I put a lambda in a variable ?
Is a lambda an object ?
#DV13LBDS
@JosePaumard
73. Modelization
A lambda is an instance of a « functional interface »
@FunctionalInterface
public interface Consumer<T> {
public void accept(T t) ;
}
#DV13LBDS
@JosePaumard
74. Modelization
A lambda is an instance of a « functional interface »
@FunctionalInterface
public interface Consumer<T> {
public void accept(T t) ;
}
- only one abstract method (methods from Object dont
count)
- can be annotated by @FunctionalInterface (optional)
#DV13LBDS
@JosePaumard
75. Modelization
A lambda is an instance of « functional interface »
@FunctionalInterface
public interface Consumer<T> {
public void accept(T t) ;
public default Consumer<T> chain(Consumer<? super T> other) {
Objects.requireNonNull(other);
return (T t) -> { accept(t); other.accept(t); };
}
}
#DV13LBDS
@JosePaumard
76. Putting a lambda in a variable
Example of a consumer
Consumer<String> c = new Consumer<String>() {
@Override
public void accept(String s) {
System.out.println(s) ;
}
} ;
So :
Consumer<String> c = s -> System.out.println(s) ;
#DV13LBDS
@JosePaumard
77. Putting a lambda in a variable
Example of a consumer
Consumer<String> c = new Consumer<String>() {
@Override
public void accept(String s) {
System.out.println(s) ;
}
} ;
Question : what is s ?
So :
Consumer<String> c = s -> System.out.println(s) ;
#DV13LBDS
@JosePaumard
78. Putting a lambda in a variable
Example of a consumer
Consumer<String> c = new Consumer<String>() {
@Override
public void accept(String s) {
System.out.println(s) ;
}
} ;
So :
Answer : the compiler
infers that it is a String
Consumer<String> c = s -> System.out.println(s) ;
#DV13LBDS
@JosePaumard
79. Putting a lambda in a variable
Side question : can one write this ?
int i = ... ;
Consumer<String> c = new Consumer<String>() {
@Override
public void accept(String s) {
System.out.println("i = " + i) ;
}
} ;
#DV13LBDS
@JosePaumard
80. Putting a lambda in a variable
Side question : can one write this ?
int i = ... ;
Consumer<String> c = new Consumer<String>() {
@Override
public void accept(String s) {
System.out.println("i = " + i) ;
}
} ;
JDK 7 : i should be final
#DV13LBDS
@JosePaumard
81. Putting a lambda in a variable
Side question : can one write this ?
int i = ... ;
Consumer<String> c = new Consumer<String>() {
@Override
public void accept(String s) {
System.out.println("i = " + i) ;
}
} ;
JDK 8 : Yes ! (no final needed)
#DV13LBDS
@JosePaumard
82. Putting a lambda in a variable
But this code …
int i = ... ;
Consumer<String> c = new Consumer<String>() {
@Override
public void accept(String s) {
System.out.println("i = " + i) ;
}
} ;
i = i + 1 ;
does not compile
#DV13LBDS
@JosePaumard
83. Putting a lambda in a variable
Because i is an effectively final variable
final int i = ... ;
Consumer<String> c = new Consumer<String>() {
@Override
public void accept(String s) {
System.out.println("i = " + i) ;
}
} ;
i = i + 1 ; // Cant modify
#DV13LBDS
@JosePaumard
84. And what about this ?
JDK 7 : this is the anonymous instance
Consumer<String> c = new Consumer<String>() {
@Override
public void accept(String s) {
System.out.println("this = " + this) ;
}
public String toString() {
return "I’m in c" ;
}
} ;
Calling accept() will print « I’m in c »
#DV13LBDS
@JosePaumard
85. And what about this ?
JDK 8 : no change, calling accept() still prints « I’m in c »
Consumer<String> c = new Consumer<String>() {
@Override
public void accept(String s) {
System.out.println("this = " + this) ;
}
public String toString() {
return "I’m in c" ;
}
} ;
But …
#DV13LBDS
@JosePaumard
86. And what about this ?
Consumer can also be instanced with a lambda
Consumer<String> c = s -> System.out.println(this) ;
Here this is the enclosing instance
#DV13LBDS
@JosePaumard
87. Questions :
What model for a lambda ?
answer : with a functional interface
Can I put a lambda in a variable ?
answer : yes
Is a lambda an object ?
#DV13LBDS
@JosePaumard
88. Is a lambda an objet ?
Let’s play the game of 7 errors (with only 1 error)
Consumer<String> c = new Consumer<String>() {
@Override
public void accept(String s) {
System.out.println(s) ;
}
} ;
Consumer<String> c = s -> System.out.println(s) ;
#DV13LBDS
@JosePaumard
89. Is a lambda an objet ?
Let’s play the game of 7 errors (with only 1 error)
Consumer<String> c = new Consumer<String>() {
@Override
public void accept(String s) {
System.out.println(s) ;
}
} ;
Consumer<String> c = s -> System.out.println(s) ;
#DV13LBDS
@JosePaumard
90. Questions :
What model for a lambda ?
answer : with a functionnal interface
Can I put a lambda in a variable ?
answer : yes
Is a lambda an object ?
answer : no
#DV13LBDS
@JosePaumard
95. Consumer
A consumer just … accepts an object
public interface Consumer<T> {
void accept(T t) ;
}
Consumer<String> c1 = s -> System.out.println(s) ;
Consumer<String> c2 = ... ;
Consumer<String> c3 = c1.andThen(c2) ;
persons.stream().forEach(c3) ;
#DV13LBDS
@JosePaumard
96. BiConsumer
Takes 2 arguments instead of one
Can be chained
Primitive types :
- ObjIntConsumer
- ObjLongConsumer
- ObjDoubleConsumer
ObjIntConsumer takes an object and an int as arguments
#DV13LBDS
@JosePaumard
97. Function
A function takes an object and returns another one
public interface Function<T, R> {
R apply(T t) ;
}
Can be chained and / or composed
#DV13LBDS
@JosePaumard
98. BiFunction
A function that takes 2 arguments
Can be chained
Primitive type versions :
- ToIntBiFunction
- ToLongBiFunction
- ToDoubleBiFunction
#DV13LBDS
@JosePaumard
99. UnaryOperator
A UnaryOperator is a Function that operates on one type
Other UnaryOperators :
- IntUnaryOperator
- LongUnaryOperator
- DoubleUnaryOperator
#DV13LBDS
@JosePaumard
100. BinaryOperator
A BinaryOperator is a BiFunction that operates on one type
Other BinaryOperators :
- IntBinaryOperator
- LongBinaryOperator
- DoubleBinaryOperator
IntBinaryOperator takes 2 int, and returns an int
#DV13LBDS
@JosePaumard
101. Predicate
A Predicate takes an object and returns a boolean
public interface Predicate<T> {
boolean test(T t) ;
}
Can be negated, composed with and / or
#DV13LBDS
@JosePaumard
102. BiPredicate
A BiPredicate takes two object and returns a boolean
public interface BiPredicate<T, U> {
boolean test(T t, U u) ;
}
#DV13LBDS
@JosePaumard
103. Predicate & BiPredicate
Primitive types :
- IntPredicate
- LongPredicate
- DoublePredicate
Not for BiPredicate
#DV13LBDS
@JosePaumard
105. Back to the
map / filter / reduce
pattern
#DV13LBDS
@JosePaumard
106. The JDK 7 way
How to apply map / filter / reduce
to List<Person> ?
#DV13LBDS
@JosePaumard
107. The JDK 7 way
How to apply map / filter / reduce
to List<Person> ?
The legal way is to iterate over the elements
and apply the pattern
#DV13LBDS
@JosePaumard
108. The JDK 7 way
How to apply map / filter / reduce
to List<Person> ?
The legal way is to iterate over the elements
and apply the pattern
One can create a helper method
#DV13LBDS
@JosePaumard
109. Applying map to a List
With a helper method, JDK 7
List<Person> persons = new ArrayList<>() ;
List<Integer> ages = // ages is a new list
Lists.map(
persons,
new Mapper<Person, Integer>() {
public Integer map(Person p) {
return p.getAge() ;
}
}
) ;
#DV13LBDS
@JosePaumard
110. Applying map to a List
With a helper method, JDK 8 & lambdas
List<Person> persons = new ArrayList<>() ;
List<Integer> ages =
Lists.map(
persons,
person -> person.getAge()
) ;
#DV13LBDS
@JosePaumard
111. Applying map to a List
With a helper method, JDK 8 & lambdas
List<Person> persons = new ArrayList<>() ;
List<Integer> ages =
Lists.map(
persons,
Person::getAge
) ;
#DV13LBDS
@JosePaumard
112. Applying map to a List
Caveats : beware the generics : what about managers ?
List<Manager> managers = new ArrayList<>() ;
List<Integer> ages =
Lists.map(
managers,
person -> person.getAge()
) ;
We want that to compile !
#DV13LBDS
@JosePaumard
113. Applying map to a List
Caveats : beware the generics
// in Lists
public static <T, V> List<V> map(
List<? extends T> list, Mapper<T, V> mapper) {
List<V> result = new ArrayList<>() ;
for (T t : list) {
V v = mapper.map(t) ;
result.add(v) ;
}
return result ;
}
#DV13LBDS
@JosePaumard
114. Putting things together
The map / filter / reduce pattern would look like this
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<Integer> ages
= Lists.map(
persons, p -> p.getAge()) ;
List<Integer> agesGT20 = Lists.filter(ages,
a -> a > 20) ;
int sum
= Lists.reduce(agesGT20, (i1, i2) -> i1 + i2) ;
#DV13LBDS
@JosePaumard
115. Putting things together
The map / filter / reduce pattern would look like this
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<Integer> ages
= Lists.map(
persons, p -> p.getAge()) ;
List<Integer> agesGT20 = Lists.filter(ages,
a -> a > 20) ;
int sum
= Lists.reduce(agesGT20, (i1, i2) -> i1 + i2) ;
The idea is to push lambdas to the API, and let it apply
them on its content
#DV13LBDS
@JosePaumard
116. Putting things together
The map / filter / reduce pattern would look like this
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<Integer> ages
= Lists.map(
persons, p -> p.getAge()) ;
List<Integer> agesGT20 = Lists.filter(ages,
a -> a > 20) ;
int sum
= Lists.reduce(agesGT20, (i1, i2) -> i1 + i2) ;
Pro : the API can be optimized
without touching the code : great !
#DV13LBDS
@JosePaumard
117. Putting things together
The map / filter / reduce pattern would look like this
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<Integer> ages
= Lists.map(
persons, p -> p.getAge()) ;
List<Integer> agesGT20 = Lists.filter(ages,
a -> a > 20) ;
int sum
= Lists.reduce(agesGT20, (i1, i2) -> i1 + i2) ;
Pro : the API can be optimized
without touching the code
Cons …
#DV13LBDS
@JosePaumard
118. Putting things together
1) Suppose persons is a really BIG list
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<Integer> ages
= Lists.map(
persons, p -> p.getAge()) ;
List<Integer> agesGT20 = Lists.filter(ages,
a -> a > 20) ;
int sum
= Lists.reduce(agesGT20, (i1, i2) -> i1 + i2) ;
#DV13LBDS
@JosePaumard
119. Putting things together
1) Suppose persons is a really BIG list
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<Integer> ages
= Lists.map(
persons, p -> p.getAge()) ;
List<Integer> agesGT20 = Lists.filter(ages,
a -> a > 20) ;
int sum
= Lists.reduce(agesGT20, (i1, i2) -> i1 + i2) ;
2 duplications : ages & agesGT20
#DV13LBDS
@JosePaumard
120. Putting things together
1) Suppose persons is a really BIG list
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<Integer> ages
= Lists.map(
persons, p -> p.getAge()) ;
List<Integer> agesGT20 = Lists.filter(ages,
a -> a > 20) ;
int sum
= Lists.reduce(agesGT20, (i1, i2) -> i1 + i2) ;
2 duplications : ages & agesGT20
What do we do with them ? Send them to the GC
#DV13LBDS
@JosePaumard
121. Putting things together
2) Suppose the algorithms to compute the map / filter /
reduce are optimal
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<Integer> ages
= Lists.map(
persons, p -> p.getAge()) ;
List<Integer> agesGT20 = Lists.filter(ages,
a -> a > 20) ;
int sum
= Lists.reduce(agesGT20, (i1, i2) -> i1 + i2) ;
#DV13LBDS
@JosePaumard
122. Putting things together
2) Suppose the algorithms to compute the map / filter /
reduce are optimal
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<Integer> ages
= Lists.map(
persons, p -> p.getAge()) ;
List<Integer> agesGT20 = Lists.filter(ages,
a -> a > 20) ;
int sum
= Lists.reduce(agesGT20, (i1, i2) -> i1 + i2) ;
… but I need to go even faster !
#DV13LBDS
@JosePaumard
123. Putting things together
2) Suppose the algorithms to compute the map / filter /
reduce are optimal
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<Integer> ages
= Lists.map(
persons, p -> p.getAge()) ;
List<Integer> agesGT20 = Lists.filter(ages,
a -> a > 20) ;
int sum
= Lists.reduce(agesGT20, (i1, i2) -> i1 + i2) ;
Only solution : go parallel !
#DV13LBDS
@JosePaumard
124. Putting things together
2) Suppose we want to go parallel
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<Integer> ages
= Lists.map(
persons, p -> p.getAge()) ;
List<Integer> agesGT20 = Lists.filter(ages,
a -> a > 20) ;
int sum
= Lists.reduce(agesGT20, (i1, i2) -> i1 + i2) ;
ages & agesGT20 would better be concurrent aware !
Because they will be accessed concurrently
#DV13LBDS
@JosePaumard
125. Putting things together
3) Suppose we have a allMatch() reducer
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<String> names =
Lists.map(persons, p -> p.getName()) ;
boolean allMatch =
Lists.allMatch(names, n -> n.length() < 20) ;
allMatch is true if all the names are shorter than 20 chars
let’s see a possible implementation of allMatch()
#DV13LBDS
@JosePaumard
126. How could one write allMatch ?
Here is a basic implementation of allMatch
public static <T> boolean allMatch(
List<? extends T> list, Filter<T> filter) {
for (T t : list) {
if (!filter.filter(t)) {
return false ;
}
}
return true ;
}
#DV13LBDS
@JosePaumard
127. How could one write allMatch ?
Here is a basic implementation of allMatch
public static <T> boolean allMatch(
List<? extends T> list, Filter<T> filter) {
for (T t : list) {
if (!filter.filter(t)) {
return false ;
}
}
No need to iterate
over the whole list
return true ;
}
#DV13LBDS
@JosePaumard
128. How could one write allMatch ?
Here is a basic implementation of allMatch
public static <T> boolean allMatch(
List<? extends T> list, Filter<T> filter) {
for (T t : list) {
if (!filter.filter(t)) {
return false ;
}
}
But…
return true ;
}
#DV13LBDS
@JosePaumard
129. Putting things together
When we apply the allMatch() reducer…
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<String> names =
Lists.map(persons, p -> p.getName()) ;
boolean allMatch =
Lists.allMatch(names, n.length() < 20) ;
… the list names has already been evaluated !
#DV13LBDS
@JosePaumard
130. Putting things together
When we apply the allMatch() reducer…
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<String> names =
Lists.map(persons, p -> p.getName()) ;
boolean allMatch =
Lists.allMatch(names, n.length() < 20) ;
… the list names has already been evaluated !
We lost a big opportunity for optimization !
#DV13LBDS
@JosePaumard
131. Putting things together
When we apply the allMatch() reducer…
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<String> names =
Lists.map(persons, p -> p.getName()) ;
boolean allMatch =
Lists.allMatch(names, n.length() < 20) ;
… we should have wait to apply the filter
A good way to do that : apply the filter step lazily !
#DV13LBDS
@JosePaumard
132. Conclusion
Pro : 1
Cons : 3 (at least)
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<Integer> ages
= Lists.map(
persons, p -> p.getAge()) ;
List<Integer> agesGT20 = Lists.filter(ages,
a -> a > 20) ;
int sum
= Lists.reduce(agesGT20, (i1, i2) -> i1 + i2) ;
#DV13LBDS
@JosePaumard
133. Conclusion
Pro : 1
Cons : 3 (at least)
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<Integer> ages
= Lists.map(
persons, p -> p.getAge()) ;
List<Integer> agesGT20 = Lists.filter(ages,
a -> a > 20) ;
int sum
= Lists.reduce(agesGT20, (i1, i2) -> i1 + i2) ;
#DV13LBDS
@JosePaumard
134. Conclusion
And the same goes for this one
// applying the map / filter / reduce pattern
List<Person> persons = ... ;
List<Integer> ages
= persons.map(p -> p.getAge()) ;
List<Integer> agesGT20 = ages.filter(a -> a > 20) ;
int sum
= agesGT20.reduce(ages, (i1, i2) -> i1 + i2) ;
#DV13LBDS
@JosePaumard
135. What about « in place » operations ?
Not possible for the map step, because the type of the list
changes
Could be evaluated for the filter, with concurrency issues
Doesnt make sense for the reduction
#DV13LBDS
@JosePaumard
137. Conclusion (again)
We need a new concept to handle BIG lists efficiently
The Collection framework is not the right one…
#DV13LBDS
@JosePaumard
138. Conclusion (again)
We need a new concept to handle BIG lists efficiently
The Collection framework is not the right one
We need something else !
#DV13LBDS
@JosePaumard
140. Introduction
Putting map / filter / reduce methods on Collection
would have led to :
// map / filter / reduce pattern on collections
int sum = persons
.map(p -> p.getAge())
.filter(a -> a > 20)
.reduce(0, (a1, a2) -> a1 + a2) ;
And even if it doesnt lead to viable patterns,
it is still a pleasant way of writing things
#DV13LBDS
@JosePaumard
141. Introduction
Let’s keep the same kind of pattern and add a stream()
// map / filter / reduce pattern on collections
int sum = persons.stream()
.map(p -> p.getAge())
.filter(a -> a > 20)
.reduce(0, (a1, a2) -> a1 + a2) ;
#DV13LBDS
@JosePaumard
142. Introduction
Collection.stream() returns a Stream : new interface
// map / filter / reduce pattern on collections
int sum = persons.stream()
.map(p -> p.getAge())
.filter(a -> a > 20)
.reduce(0, (a1, a2) -> a1 + a2) ;
New interface = our hands are free !
#DV13LBDS
@JosePaumard
143. New Collection interface
So we need a new method on Collection
public interface Collection<E> {
// our good old methods
Stream<E> stream() ;
}
#DV13LBDS
@JosePaumard
144. New Collection interface
Problem : ArrayList doesnt compile anymore…
Something has to be done !
public interface Collection<E> {
// our good old methods
Stream<E> stream() ;
}
#DV13LBDS
@JosePaumard
145. Interfaces
Problem : ArrayList doesnt compile anymore…
Something has to be done !
Something that does not need to change or recompile all
the existing implementations of Collection
#DV13LBDS
@JosePaumard
148. Java 8 interfaces
Problem : ArrayList doesnt compile anymore…
Something has to be done !
Something that does not need to change or recompile all
the existing implementations of Collection
Problem : how to add methods to an interface, without
changing any of the implementations ?
#DV13LBDS
@JosePaumard
149. Java 8 interfaces
Problem : how to add methods to an interface, without
changing any of the implementations ?
Solution : change the way interfaces work in Java
#DV13LBDS
@JosePaumard
150. Java 8 interfaces
ArrayList needs to know the implementation of stream()…
public interface Collection<E> {
// our good old methods
Stream<E> stream() ;
}
#DV13LBDS
@JosePaumard
151. Java 8 interfaces
Let’s put it in the interface !
public interface Collection<E> {
// our good old methods
default Stream<E> stream() {
return ... ;
}
}
#DV13LBDS
@JosePaumard
152. Java 8 interfaces
Let’s introduce default methods in Java
public interface Collection<E> {
// our good old methods
default Stream<E> stream() {
return ... ;
}
}
Default methods are about interface evolution
Allows the extension of old interfaces
#DV13LBDS
@JosePaumard
154. Default methods
Does it bring multiple inheritance to Java ?
Yes ! But we already have it
#DV13LBDS
@JosePaumard
155. Default methods
Does it bring multiple inheritance to Java ?
Yes ! But we already have it
public class String
implements Serializable, Comparable<String>, CharSequence {
// ...
}
#DV13LBDS
@JosePaumard
157. Default methods
What we have so far is multiple inheritance of type
What we get in Java 8 is multiple inheritance of behavior
#DV13LBDS
@JosePaumard
158. Default methods
What we have so far is multiple inheritance of type
What we get in Java 8 is multiple inheritance of behavior
What we dont have is multiple inheritance of state
#DV13LBDS
@JosePaumard
159. Default methods
What we have so far is multiple inheritance of type
What we get in Java 8 is multiple inheritance of behavior
What we dont have is multiple inheritance of state
… and this is where the trouble is !
#DV13LBDS
@JosePaumard
164. Default methods
public class C
implements A, B {
// ...
}
public interface A {
default String a() { ... }
}
#DV13LBDS
public interface B {
default String a() { ... }
}
@JosePaumard
165. Default methods
public class C
implements A, B {
public String a() { ... }
}
Example #0
public interface A {
default String a() { ... }
}
public interface B {
default String a() { ... }
}
Class wins !
No default if an implementation is provided
#DV13LBDS
@JosePaumard
166. Default methods
public class C
implements A, B {
// ...
}
Example #1
public interface A {
default String a() { ... }
}
#DV13LBDS
public interface B {
default String a() { ... }
}
@JosePaumard
167. Default methods
public class C
implements A, B {
// ...
}
Example #1
public interface A {
default String a() { ... }
}
public interface B {
default String a() { ... }
}
Compile time error :
class C inherits unrelated defaults for a() from types A and B
#DV13LBDS
@JosePaumard
168. Default methods
public class C
implements A, B {
// ...
}
Example #2
public interface A extends B {
default String a() { ... }
}
public interface B {
default String a() { ... }
}
A is more specific than B : A.a() is chosen
#DV13LBDS
@JosePaumard
169. Default methods
public class D extends C
implements B {
// ...
}
public class C
implements A {
// ...
}
Example #2
public interface A extends B {
default String a() { ... }
}
#DV13LBDS
public interface B {
default String a() { ... }
}
@JosePaumard
170. Default methods
public class D extends C
implements B {
// ...
}
public class C
implements A {
// ...
}
Example #2
public interface A extends B {
default String a() { ... }
}
public interface B {
default String a() { ... }
}
A is still more specific than B : A.a() is chosen
#DV13LBDS
@JosePaumard
171. Default methods
public class C
implements A, B {
public String a() { return B.super.a() ; }
}
Back to example #1
public interface A {
default String a() { ... }
}
public interface B {
default String a() { ... }
}
We can also make an explicit call
#DV13LBDS
@JosePaumard
172. 3 simple rules
To handle the conflict of multiple inheritance of behavior
#DV13LBDS
@JosePaumard
173. 3 simple rules
To handle the conflict of multiple inheritance of behavior
1) Class wins !
#DV13LBDS
@JosePaumard
174. 3 simple rules
To handle the conflict of multiple inheritance of behavior
1) Class wins !
2) More specific interfaces wins over less specific
#DV13LBDS
@JosePaumard
175. 3 simple rules
To handle the conflict of multiple inheritance of behavior
1) Class wins !
2) More specific interfaces wins over less specific
3) In fact there’s no rule #3
#DV13LBDS
@JosePaumard
176. An example
Everybody who writes an implementation of Iterator
writes this :
public class MyIterator implements Iterator<E> {
// some critical business code
public void remove() {
throw new UnsupportedOperationException("You shouldnt do that") ;
} ;
}
#DV13LBDS
@JosePaumard
177. An example
Thanks to Java 8 :
public interface Iterator<E> {
default void remove() {
throw new UnsupportedOperationException("remove") ;
} ;
}
No silly implementation of remove() anymore !
#DV13LBDS
@JosePaumard
178. Java 8 interfaces
So the concept of « interface » changed in Java 8
public class HelloWorld {
// souvenir souvenir
public static void main(String[] args) {
System.out.println("Hello world!") ;
} ;
}
#DV13LBDS
@JosePaumard
179. Java 8 interfaces
So the concept of « interface » changed in Java 8
I mean, really
public interface HelloWorld {
// souvenir souvenir
public static void main(String[] args) {
System.out.println("Hello world!") ;
} ;
}
#DV13LBDS
@JosePaumard
180. Java 8 interfaces
1) Default methods, multiple behavior inheritance
Rules to handle conflicts, solved at compile time
2) Static methods in interfaces
Will bring new patterns and new ways to write APIs
#DV13LBDS
@JosePaumard
181. Where are we ?
1) We have a new syntax
2) We have a new pattern to implement
3) We have new interfaces to keep the backward
compatibily
#DV13LBDS
@JosePaumard
182. Where are we going to ?
1)
2)
3)
4)
The Stream API
Reductions
Performances
Collectors
But for now…
#DV13LBDS
@JosePaumard
185. What is a Stream
From a technical point of view : a typed interface
« a stream of String »
#DV13LBDS
@JosePaumard
186. What is a Stream
From a technical point of view : a typed interface
« a stream of String »
Other classes for primitive types :
IntStream, LongStream, DoubleStream
#DV13LBDS
@JosePaumard
187. It’s a new notion
It‘ looks like a collection, but…
- It is built on a « source », that can be a collection
- No data needs to be provided at build time
- No limit on what the source can produce
#DV13LBDS
@JosePaumard
188. It’s a new notion
A stream can be built on :
- A collection or an array
- An iterator
- An I/O source
Some of those can be « infinite »
#DV13LBDS
@JosePaumard
189. It’s a new notion
1) A stream doesnt hold any data
It’s just an object on which one can declare operations
Streams are about « specifying computations »
#DV13LBDS
@JosePaumard
190. It’s a new notion
1) A stream doesnt hold any data
2) A stream cant modify its source
Consequence : it can use parallel processing
#DV13LBDS
@JosePaumard
191. It’s a new notion
1) A stream doesnt hold any data
2) A stream cant modify its source
3) A source can be infinite
So we need a way to guarantee that computations
will be conducted in finite time
#DV13LBDS
@JosePaumard
192. It’s a new notion
1)
2)
3)
4)
A stream doesnt hold any data
A stream cant modify its source
A source can be infinite
A stream works lazily
Then optimizations can be made among the different
operations
We need a way / convention to trigger the computations
#DV13LBDS
@JosePaumard
193. Is a Stream the same as a collection ?
Answer is no
#DV13LBDS
@JosePaumard
194. Is a Stream the same as a collection ?
Answer is no
A Collection provides means to access the elements one
by one
#DV13LBDS
@JosePaumard
195. Is a Stream the same as a collection ?
Answer is no
A Collection provides means to access the elements one
by one
A Stream does not
#DV13LBDS
@JosePaumard
196. Is a Stream the same as a collection ?
Answer is no
A Collection does not provide any mean to process its
elements with lambdas
#DV13LBDS
@JosePaumard
197. Is a Stream the same as a collection ?
Answer is no
A Collection does not provide any mean to process its
elements with lambdas
A Stream does
#DV13LBDS
@JosePaumard
198. How can we build a stream ?
Many ways…
1) From a collection : method Collection.stream
Collection<String> collection = ... ;
Stream<String> stream = collection.stream() ;
#DV13LBDS
@JosePaumard
199. How can we build a stream ?
Many ways…
1) From a collection : method Collection.stream
2) From an array : Arrays.stream(Object [])
Stream<String> stream2 =
Arrays.stream(new String [] {"one", "two", "three"}) ;
#DV13LBDS
@JosePaumard
200. How can we build a stream ?
Many ways…
1) From a collection : method Collection.stream
2) From an array : Arrays.stream(Object [])
3) From factory methods in Stream, IntStream, …
Stream<String> stream1 = Stream.of("one", "two", "three") ;
#DV13LBDS
@JosePaumard
201. How can we build a stream ?
Some last patterns
Stream.empty() ; // returns an empty Stream
Stream.of(T t) ; // a stream with a single element
Stream.generate(Supplier<T> s) ;
Stream.iterate(T seed, UnaryOperator<T> f) ;
#DV13LBDS
@JosePaumard
202. How can we build a stream ?
Other ways scattered in the JDK
string.chars() ; // returns a IntStream
lineNumberReader.lines() ; // returns a Stream<String>
random.ints() ; // return a IntStream
#DV13LBDS
@JosePaumard
203. How can we build a stream ?
Factory methods : a closer look
Stream.build() ; // returns a Stream.Builder
A stream builder is a consumer
- one sends it elements, that are accepted
- it builds back a stream on the build() call with the
accepted elemts
- it has a life cycle
#DV13LBDS
@JosePaumard
204. How can we build a stream ?
And as a last resort : StreamSupport + Spliterator
« This class is mostly for library writers
presenting stream views of data structures;
most static stream methods intended for end
users are in the various Stream classes »
#DV13LBDS
@JosePaumard
205. A 1st example
Back to our map / filter / reduce example
Build a stream
collections
on a List
// map / filter / reduce pattern on
int sum = persons.stream()
.map(p -> p.getAge())
.filter(a -> a > 20)
.reduce(0, (a1, a2) -> a1 + a2) ;
#DV13LBDS
@JosePaumard
206. A 1st example
Back to our map / filter / reduce example
Declare
collections
operations
// map / filter / reduce pattern on
int sum = persons.stream()
.map(p -> p.getAge())
.filter(a -> a > 20)
.reduce(0, (a1, a2) -> a1 + a2) ;
#DV13LBDS
@JosePaumard
207. A 1st example
Back to our map / filter / reduce example
Triggers the
collections
computation
// map / filter / reduce pattern on
int sum = persons.stream()
.map(p -> p.getAge())
.filter(a -> a > 20)
.reduce(0, (a1, a2) -> a1 + a2) ;
#DV13LBDS
@JosePaumard
208. A 1st example
Back to our map / filter / reduce example
// map / filter / reduce pattern on collections
int sum = persons.stream()
.map(p -> p.getAge())
.filter(a -> a > 20)
.reduce(0, (a1, a2) -> a1 + a2) ;
Default value, in case the
stream is empty
#DV13LBDS
@JosePaumard
209. Two types of operations
One can declare operations on a Stream
Two types of operations :
1) Intermediate operations : declarations, processed lazily
ex : map, filter
#DV13LBDS
@JosePaumard
210. Two types of operations
One can declare operations on a Stream
Two types of operations :
1) Intermediate operations : declarations, processed lazily
ex : map, filter
2) Terminal operations : trigger the computations
ex : reduce
#DV13LBDS
@JosePaumard
211. State of a stream
The implementation of Stream has a state :
- SIZED : the number of elements is known
- ORDERED : the order matters (List)
- DISTINCT : all elements are unique (Set)
- SORTED : all elements have been sorted (SortedSet)
#DV13LBDS
@JosePaumard
212. State of a stream
Some operations change that :
- filtering removes the SIZED
- mapping removes DISTINCT and SORTED
#DV13LBDS
@JosePaumard
213. State of a stream
And it allows nice optimizations !
- a HashSet cant have doubles so…
#DV13LBDS
@JosePaumard
214. State of a stream
And it allows nice optimizations !
- a HashSet cant have doubles so…
- distinct() is NOPed on a stream built on a HashSet
#DV13LBDS
@JosePaumard
215. State of a stream
And it allows nice optimizations !
- a HashSet cant have doubles so…
- distinct() is NOPed on a stream built on a HashSet
- A TreeSet is sorted so…
#DV13LBDS
@JosePaumard
216. State of a stream
And it allows nice optimizations !
- a HashSet cant have doubles so…
- distinct() is NOPed on a stream built on a HashSet
- A TreeSet is sorted so…
- sorted() is NOPed on a stream built on a TreeSet
#DV13LBDS
@JosePaumard
217. A 2nd example
Sorting a list of String
List<String> strings = new ArrayList<>() ;
// populate the list with strings
// then somewhere else in the code :
List<String> result = strings.stream()
.sorted()
.collect(Collector.toList()) ;
#DV13LBDS
@JosePaumard
218. A 2nd example
Sorting a list of String
// then some smart guy changes the implementation
SortedSet<String> strings = new TreeSet<>() ;
// populate the list with strings
// then somewhere else in the code :
List<String> result = strings.stream()
.sorted()
.collect(Collector.toList()) ;
#DV13LBDS
@JosePaumard
219. A 2nd example
Sorting a list of String
// then some smart guy changes the implementation
SortedSet<String> strings = new TreeSet<>() ;
// populate the list with strings
// then somewhere else in the code :
List<String> result = strings.stream()
.sorted()
.collect(Collector.toList()) ;
In this case, no sorting is required, and executed !
#DV13LBDS
@JosePaumard
220. Stateless / statefull operations
Stateless / statefull operations
Some operations are stateless :
persons.stream().map(p -> p.getAge()) ;
It means that they dont need more information than the
one held by their parameters
#DV13LBDS
@JosePaumard
221. Stateless / statefull operations
Stateless / statefull operations
Some operations are stateless :
persons.stream().map(p -> p.getAge()) ;
Some need to retain information :
stream.limit(10_000_000) ; // select the 10M first elements
#DV13LBDS
@JosePaumard
222. Example
Let’s sort an array of String
Random rand = new Random() ;
String [] strings = new String[10_000_000] ;
for (int i = 0 ; i < strings.length ; i++) {
strings[i] = Long.toHexString(rand.nextLong()) ;
}
#DV13LBDS
@JosePaumard
223. Example
Let’s sort an array of String
Random rand = new Random() ;
String [] strings = new String[10_000_000] ;
for (int i = 0 ; i < strings.length ; i++) {
strings[i] = Long.toHexString(rand.nextLong()) ;
}
Soooo Java 7…
#DV13LBDS
@JosePaumard
224. Example
Let’s sort an array of String
Random rand = new Random() ;
Stream<String> stream =
Stream.generate(
() ->
Long.toHexString(rand.nextLong())
) ;
Better ?
#DV13LBDS
@JosePaumard
225. Example
Let’s sort an array of String
// Random rand = new Random() ;
Stream<String> stream =
Stream.generate(
() ->
Long.toHexString(ThreadLocalRandom.current().nextLong())
) ;
Better !
#DV13LBDS
@JosePaumard
226. Example
Let’s sort an array of String
// other way
Stream<String> stream =
ThreadLocalRandom
.current()
.longs() // returns a LongStream
.mapToObj(l -> Long.toHexString(l)) ;
#DV13LBDS
@JosePaumard
227. Example
Let’s sort an array of String
// other way
Stream<String> stream =
ThreadLocalRandom
.current()
.longs()
.mapToObj(Long::toHexString) ;
#DV13LBDS
@JosePaumard
228. Example
Let’s sort an array of String
// other way
Stream<String> stream =
ThreadLocalRandom
.current()
.longs()
.mapToObj(Long::toHexString)
.limit(10_000_000)
.sorted() ;
T = 4 ms
#DV13LBDS
@JosePaumard
229. Example
Let’s sort an array of String
// other way
Stream<String> stream =
ThreadLocalRandom
.current()
.longs()
.mapToObj(Long::toHexString)
.limit(10_000_000)
.sorted() ;
Object [] sorted = stream.toArray() ;
#DV13LBDS
@JosePaumard
230. Example
Let’s sort an array of String
// other way
Stream<String> stream =
ThreadLocalRandom
.current()
.longs()
.mapToObj(Long::toHexString)
.limit(10_000_000)
.sorted() ;
Object [] sorted = stream.toArray() ;
T = 14 s
#DV13LBDS
@JosePaumard
231. Example
Let’s sort an array of String
// other way
Stream<String> stream =
ThreadLocalRandom
.current()
.longs()
.mapToObj(Long::toHexString)
.limit(10_000_000)
.sorted() ;
Intermediate calls !
Object [] sorted = stream.toArray() ;
T = 14 s
#DV13LBDS
@JosePaumard
232. More on stream operations
1) There are intermediate and terminal operations
2) A stream is processed on a terminal operation call
#DV13LBDS
@JosePaumard
233. More on stream operations
1)
2)
3)
4)
There are intermediate and terminal operations
A stream is processed on a terminal operation call
Only one terminal operation is allowed
It cannot be processed again
If needed another stream should be built on the source
#DV13LBDS
@JosePaumard
235. Optimization
The first optimization (well, laziness was first !) we need is
parallelism
The fork / join enable parallel programming since JDK 7
But writing tasks is hard and does not always lead to better
performances
#DV13LBDS
@JosePaumard
236. Optimization
The first optimization (well, laziness was first !) we need is
parallelism
The fork / join enable parallel programming since JDK 7
But writing tasks is hard and does not always lead to better
performances
Using the parallel stream API is much more secure
#DV13LBDS
@JosePaumard
237. How to build a parallel stream ?
Two patterns
1) Call parallelStream() instead of stream()
Stream<String> s = strings.parallelStream() ;
2) Call parallel() on an existing stream
Stream<String> s = strings.stream().parallel() ;
#DV13LBDS
@JosePaumard
238. Can we decide the # of cores ?
Answer is no
#DV13LBDS
@JosePaumard
239. Can we decide the # of cores ?
Answer is no
But… do we really need it ?
#DV13LBDS
@JosePaumard
242. Is parallelism really that simple ?
Example 1 :
persons.stream().limit(10_000_000) ;
« returns the 10M first elements of the source »
#DV13LBDS
@JosePaumard
243. Is parallelism really that simple ?
Example 1 :
persons.parallelStream().limit(10_000_000) ;
« returns the 10M first elements of the source »
#DV13LBDS
@JosePaumard
244. Is parallelism really that simple ?
Example 1 :
persons.parallelStream().limit(10_000_000) ;
« returns the 10M first elements of the source »
But how can I track the first 10M of the source on several
cores of my CPU ?
#DV13LBDS
@JosePaumard
245. Is parallelism really that simple ?
Example 1 : performances
Code 1
List<Long> list = new ArrayList<>(10_000_100) ;
for (int i = 0 ; i < 10_000_000 ; i++) {
list1.add(ThreadLocalRandom.current().nextLong()) ;
}
#DV13LBDS
@JosePaumard
246. Is parallelism really that simple ?
Example 1 : performances
Code 2
Stream<Long> stream =
Stream.generate(() -> ThreadLocalRandom.current().nextLong()) ;
List<Long> list1 =
stream.limit(10_000_000).collect(Collectors.toList()) ;
#DV13LBDS
@JosePaumard
247. Is parallelism really that simple ?
Example 1 : performances
Code 3
Stream<Long> stream =
ThreadLocalRandom.current().longs(10_000_000).mapToObj(Long::new) ;
List<Long> list = stream.collect(Collectors.toList()) ;
#DV13LBDS
@JosePaumard
248. Is parallelism really that simple ?
Example 1 : performances
Code 1 (for)
Code 2 (limit)
Code 3 (longs)
#DV13LBDS
Serial
270 ms
310 ms
250 ms
Parallel
@JosePaumard
249. Is parallelism really that simple ?
Example 1 : performances
Code 1 (for)
Code 2 (limit)
Code 3 (longs)
#DV13LBDS
Serial
270 ms
310 ms
250 ms
Parallel
500 ms
320 ms
@JosePaumard
250. Is parallelism really that simple ?
Example 2 :
Stream s3 = Stream.concat(stream1, stream2) ;
« returns a stream with all the elements of the first stream
followed by all the elements of the second stream »
#DV13LBDS
@JosePaumard
251. Parallelism
Parallelism implies overhead most of the time
Badly configured parallel operations can lead to unneeded
computations that will slow down the overall process
Unnecessary ordering of a stream will lead to performance
hits
#DV13LBDS
@JosePaumard
252. Back to our sorting example
Parallel version
Stream<String> stream = ... ; // created before the test
Object [] sorted =
stream.parallel()
.sorted() // or not
.toArray() ;
#DV13LBDS
@JosePaumard
253. Sorting performances
In fact it’s not that simple
#
10
100
1K
10K
100K
1M
10M
20M
30M
#DV13LBDS
Serial
33 ms
1 ms
2 ms
5 ms
65 ms
540 ms
9,7 s
19,1 s
31,0 s
Parallel
32 ms
1 ms
1 ms
7 ms
49 ms
153 ms
2,2 s
4,78 s
8,13 s
Ratio
Ratio n.log(n)
@JosePaumard
254. Sorting performances
In fact it’s not that simple
#
10
100
1K
10K
100K
1M
10M
20M
30M
#DV13LBDS
Serial
33 ms
1 ms
2 ms
5 ms
65 ms
540 ms
9,7 s
19,1 s
31,0 s
Parallel
32 ms
1 ms
1 ms
7 ms
49 ms
153 ms
2,2 s
4,78 s
8,13 s
Ratio
1
1
2
0,7
1,3
3,5
4,4
4,0
3,8
Ratio n.log(n)
@JosePaumard
255. Sorting performances
In fact it’s not that simple
#
10
100
1K
10K
100K
1M
10M
20M
30M
#DV13LBDS
Serial
33 ms
1 ms
2 ms
5 ms
65 ms
540 ms
9,7 s
19,1 s
31,0 s
Parallel
32 ms
1 ms
1 ms
7 ms
49 ms
153 ms
2,2 s
4,78 s
8,13 s
Ratio
1
1
2
0,7
1,3
3,5
4,4
4,0
3,8
Ratio n.log(n)
1
664
4982
26,6K
25,6K
36,9K
23,9K
25,3K
24,1K
@JosePaumard
258. What is a reduction
2 kinds of reduction :
1) The reduction
« A reduction operation (also called a fold) takes a
sequence of input elements and combines them
into a single summary result by repeated
application of a combining operation »
#DV13LBDS
@JosePaumard
259. What is a reduction
2 kinds of reduction :
2) The mutable reduction
« A mutable reduction operation accumulates
input elements into a mutable result container,
such as a Collection or StringBuilder, as it
processes the elements in the stream »
#DV13LBDS
@JosePaumard
260. A simple reduction
Sum of ages
// map / filter / reduce pattern on collections
int sum = persons.stream()
.map(p -> p.getAge())
.filter(a -> a > 20)
.reduce(0, (a1, a2) -> a1 + a2) ;
#DV13LBDS
@JosePaumard
261. A simple reduction
It would be nice to be able to write :
// map / filter / reduce pattern on collections
int sum = persons.stream()
.map(p -> p.getAge())
.filter(a -> a > 20)
.sum() ;
But there’s no sum() on Stream<T>
What would be the sense of « adding persons » ?
#DV13LBDS
@JosePaumard
262. A simple reduction
It would be nice to be able to write :
// map / filter / reduce pattern on collections
int sum = persons.stream()
.map(p -> p.getAge())
.filter(a -> a > 20)
.sum() ;
But there’s no sum() on Stream<T>
There’s one on IntStream !
#DV13LBDS
@JosePaumard
263. A simple reduction
2nd version :
// map / filter / reduce pattern on collections
int sum = persons.stream()
.map(Person::getAge)
.filter(a -> a > 20)
.mapToInt(Integer::intValue)
.sum() ;
Value if persons is an empty list : 0
#DV13LBDS
@JosePaumard
264. A simple reduction
2nd version (even better) :
// map / filter / reduce pattern on collections
int sum = persons.stream()
.mapToInt(Person::getAge)
.filter(a -> a > 20)
// .mapToInt(Integer::intValue)
.sum() ;
Value if persons is an empty list : 0
#DV13LBDS
@JosePaumard
265. A simple reduction
What about max() and min() ?
// map / filter / reduce pattern on collections
....... = persons.stream()
.mapToInt(Person::getAge)
.filter(a -> a > 20)
.max() ;
How can one chose a default value ?
#DV13LBDS
@JosePaumard
266. Problem with the default value
The « default value » is a tricky notion
#DV13LBDS
@JosePaumard
267. Problem with the default value
The « default value » is a tricky notion
1) The « default value » is the reduction of the empty set
#DV13LBDS
@JosePaumard
268. Problem with the default value
The « default value » is a tricky notion
1) The « default value » is the reduction of the empty set
2) But it’s also the identity element for the reduction
#DV13LBDS
@JosePaumard
277. Problem with the default value
The « default value » is a tricky notion
1) The « default value » is the reduction of the empty set
2) But it’s also the identity element for the reduction
#DV13LBDS
@JosePaumard
278. Problem with the default value
Problem : max() and min() have no identity element
eg : an element e for which max(e, a) = a
#DV13LBDS
@JosePaumard
279. Problem with the default value
Problem : max() and min() have no identity element
eg : an element e for which max(e, a) = a
0 cant be, max(0, -1) is not -1…
−∞ is not an integer
#DV13LBDS
@JosePaumard
280. Problem with the default value
So what is the default value of max() and min() ?
#DV13LBDS
@JosePaumard
281. Problem with the default value
So what is the default value of max() and min() ?
Answer : there is no default value for max() and min()
#DV13LBDS
@JosePaumard
282. Problem with the default value
So what is the returned value of the max() reduction ?
// map / filter / reduce pattern on collections
....... = persons.stream()
.mapToInt(Person::getAge)
.filter(a -> a > 20)
.max() ;
#DV13LBDS
@JosePaumard
283. Problem with the default value
So what is the returned value of the max() reduction ?
// map / filter / reduce pattern on collections
....... = persons.stream()
.mapToInt(Person::getAge)
.filter(a -> a > 20)
.max() ;
If it’s int, then the default value will be 0…
#DV13LBDS
@JosePaumard
284. Optionals
Since there’s none, we need another notion
// map / filter / reduce pattern on collections
OptionalInt optionalMax = persons.stream()
.mapToInt(Person::getAge)
.filter(a -> a > 20)
.max() ;
« there might be no
result »
#DV13LBDS
@JosePaumard
285. Optionals
What can I do with OptionalInt ?
1st pattern : test if it holds a value
OptionalInt optionalMax = ... ;
int max ;
if (optionalMax.isPresent()) {
max = optionalMax.get() ;
} else {
max = ... ; // decide a « default value »
}
#DV13LBDS
@JosePaumard
286. Optionals
What can I do with OptionalInt ?
2nd pattern : read the held value, can get an exception
OptionalInt optionalMax = ... ;
// throws NoSuchElementException if no held value
int max = optionalMax.getAsInt() ;
#DV13LBDS
@JosePaumard
287. Optionals
What can I do with OptionalInt ?
3rd pattern : read the held value, giving a default value
OptionalInt optionalMax = ... ;
// get 0 if no held value
int max = optionalMax.orElse(0) ;
#DV13LBDS
@JosePaumard
288. Optionals
What can I do with OptionalInt ?
4th pattern : read the held value, or throw an exception
OptionalInt optionalMax = ... ;
// exceptionSupplier will supply an exception, if no held value
int max = optionalMax.orElseThrow(exceptionSupplier) ;
#DV13LBDS
@JosePaumard
292. Available reductions
SummaryStatistics is an object that holds :
- Count, Min, Max, Sum, Average
It’s a consumer, so it can be easily extended to compute
other statistics
#DV13LBDS
@JosePaumard
293. Mutable reductions : example 1
Use of the helper class : Collectors
ArrayList<String> strings =
stream
.map(Object::toString)
.collect(Collectors.toList()) ;
#DV13LBDS
@JosePaumard
294. Mutable reductions : example 2
Concatenating strings with a helper
String names = persons
.stream()
.map(Person::getName)
.collect(Collectors.joining()) ;
#DV13LBDS
@JosePaumard
295. Mutable reductions
Common things :
- have a container : Collection or StringBuilder
- have a way to add an element to the container
- have a way to merge two containers (for parallelization)
#DV13LBDS
@JosePaumard
296. Mutable reductions
The general form works with 3 objects :
- a supplier : supplies a new instance of the mutable result
container
Supplier<ArrayList<String>> supplier = () -> new ArrayList<String>() ;
#DV13LBDS
@JosePaumard
297. Mutable reductions
The general form works with 3 objects :
- a supplier : supplies a new instance of the mutable result
container
- an accumulator : takes an partly filled container and
adds an element to it
BiConsumer<ArrayList<String>, ? super String> accumulator =
(suppliedList, s) -> suppliedList.add(s) ;
#DV13LBDS
@JosePaumard
298. Mutable reductions
The general form works with 3 objects :
- a supplier : supplies a new instance of the mutable result
container
- an accumulator : takes an partly filled container and
adds an element to it
- a combiner : takes two partly filled containers and
combines them
BiConsumer<ArrayList<String>, ArrayList<String>> combiner =
(supplied1, supplied2) -> supplied1.addAll(supplied2) ;
#DV13LBDS
@JosePaumard
299. Mutable reductions : example 1
Putting things together :
ArrayList<String> strings =
stream
.map(Object::toString)
.collect(
() -> new ArrayList<String>(), // the supplier
(suppliedList, s) -> suppliedList.add(s), // the accumulator
(supplied1, supplied2) -> supplied1.addAll(supplied2) // the combiner
) ;
#DV13LBDS
@JosePaumard
300. Mutable reductions : example 1
Putting things together :
ArrayList<String> strings =
stream
.map(Object::toString)
.collect(
ArrayList::new,
// the supplier
ArrayList::add,
// the accumulator
ArrayList::addAll // the combiner
) ;
#DV13LBDS
@JosePaumard
302. The Collectors class
A rich toolbox (37 methods) for various types of reductions
- counting, minBy, maxBy
- summing, averaging, summarizing
- joining
- toList, toSet
And
- mapping, groupingBy, partionningBy
#DV13LBDS
@JosePaumard
303. The Collectors class
Average, Sum, Count
persons
.stream()
.collect(Collectors.averagingDouble(Person::getAge)) ;
persons
.stream()
.collect(Collectors.counting()) ;
#DV13LBDS
@JosePaumard
304. The Collectors class
Concatenating the names in a String
String names = persons
.stream()
.map(Person::getName)
.collect(Collectors.joining(", ")) ;
#DV13LBDS
@JosePaumard
305. The Collectors class
Accumulating in a List, Set
Set<Person> setOfPersons = persons
.stream()
.collect(
Collectors.toSet()) ;
#DV13LBDS
@JosePaumard
306. The Collectors class
Accumulating in a custom collection
TreeSet<Person> treeSetOfPersons = persons
.stream()
.collect(
Collectors.toCollection(TreeSet::new)) ;
#DV13LBDS
@JosePaumard
307. The Collectors class
Getting the max according to a comparator
Optional<Person> optionalPerson = persons
.stream()
.collect(
Collectors.maxBy(
Comparator.comparing(Person::getAge)) ;
Bonus : new API to build comparators
#DV13LBDS
@JosePaumard
308. Building comparators
New API to build comparators in a declarative way
Comparator<Person> comp =
Comparator.comparing(Person::getLastName)
.thenComparing(Person::getFirstName)
.thenComparing(Person:getAge) ;
#DV13LBDS
@JosePaumard
309. The Collectors class : mapping
The mapping helper method takes 2 parameter :
- a function, that maps the elements of the stream
- a collector, called a « downstream », that is applied to
the mapped values
#DV13LBDS
@JosePaumard
310. The Collectors class : mapping
Accumulating names in a set
Set<String> set = persons
.stream()
.collect(
Collectors.mapping(
Person::getLastName,
Collectors.toSet())) ;
#DV13LBDS
@JosePaumard
311. The Collectors class : mapping
Mapping the stream, then accumulating in a collection
TreeSet<String> set = persons
.stream()
.collect(
Collectors.mapping(
Person::getLastName,
Collectors.toCollection(TreeSet::new))
) ;
#DV13LBDS
@JosePaumard
312. The Collector API : groupingBy
« Grouping by » builds hash maps
- must explain how the keys are built
- by default the values are put in a list
- may specify a downstream (ie a collector)
#DV13LBDS
@JosePaumard
313. The Collector API : groupingBy
Grouping a list of persons by their age
Map<Integer, List<Person>> map =
persons
.stream()
.collect(
Collectors.groupingBy(Person::getAge)) ;
#DV13LBDS
@JosePaumard
314. The Collector API : groupingBy
Grouping a list of persons by their age
Map<Integer, Set<Person>> map =
persons
.stream()
.collect(
Collectors.groupingBy(
Person::getAge,
Collectors.toSet() // the downstream
) ;
#DV13LBDS
@JosePaumard
315. The Collector API : groupingBy
Grouping a list of persons names by their age
Map<Integer, Set<String>> map =
persons
.stream()
.collect(
Collectors.groupingBy(
Person::getAge,
Collectors.mapping(
//
Person::getLastName, // the downstream
Collectors.toSet()
//
)
) ;
#DV13LBDS
@JosePaumard
316. The Collector API : groupingBy
Grouping a list of persons names by their age
Map<Integer, TreeSet<String>> map =
persons
.stream()
.collect(
Collectors.groupingBy(
Person::getAge,
Collectors.mapping(
Person::getLastName,
Collectors.toCollection(TreeSet::new)
)
) ;
#DV13LBDS
@JosePaumard
317. The Collector API : groupingBy
Grouping a list of blah blah blah
TreeMap<Integer, TreeSet<String>> map =
persons
.stream()
.collect(
Collectors.groupingBy(
Person::getAge,
TreeMap::new,
Collectors.mapping(
Person::getLastName,
Collectors.toCollection(TreeSet::new)
)
) ;
#DV13LBDS
@JosePaumard
318. The Collector API : groupingBy
Example : creating an age histogram
Map<Integer, Long> map =
persons
.stream()
.collect(
Collectors.groupingBy(Person::getAge, Collectors.counting())
) ;
Gives the # of persons by age
#DV13LBDS
@JosePaumard
319. The Collector API : partionningBy
Creates a Map<Boolean, …> on a predicate
- the map has 2 keys : TRUE and FALSE
- may specify a downstream
#DV13LBDS
@JosePaumard
320. The Collector API : partionningBy
Creates a Map<Boolean, …> on a predicate
Map<Boolean, List<Person>> map =
persons
.stream()
.collect(
Collectors.partitioningBy(p -> p.getAge() > 20)
) ;
map.get(TRUE) returns the list people older than 20
#DV13LBDS
@JosePaumard
321. The Collector API : partionningBy
Can further process the list of persons
Map<Boolean, TreeSet<String>> map =
persons
.stream()
.collect(
Collectors.partitioningBy(
p -> p.getAge() > 20,
Collectors.mapping(
Person::getLastName,
Collectors.toCollection(TreeSet::new))
)
)
) ;
#DV13LBDS
@JosePaumard
322. The Collector API : partionningBy
Can further process the list of persons
Map<Boolean, TreeSet<String>> map =
Partition the persons
persons
.stream()
on age > 20
.collect(
Collectors.partitioningBy(
p -> p.getAge() > 20,
Collectors.mapping(
Person::getLastName,
Collectors.toCollection(TreeSet::new))
)
)
) ;
#DV13LBDS
@JosePaumard
323. The Collector API : partionningBy
Can further process the list of persons
Map<Boolean, TreeSet<String>> map =
Gather their
persons
.stream()
.collect(
Collectors.partitioningBy(
p -> p.getAge() > 20,
Collectors.mapping(
Person::getLastName,
Collectors.toCollection(TreeSet::new))
)
)
) ;
#DV13LBDS
names…
@JosePaumard
324. The Collector API : partionningBy
Can further process the list of persons
Map<Boolean, TreeSet<String>> map =
… in TreeSets
persons
.stream()
.collect(
Collectors.partitioningBy(
p -> p.getAge() > 20,
Collectors.mapping(
Person::getLastName,
Collectors.toCollection(TreeSet::new))
)
)
) ;
#DV13LBDS
@JosePaumard
325. The Collector API : collectingAndThen
Collect data with a downstream
Then apply a function called a « finisher »
Useful for putting the result in a immutable collection
#DV13LBDS
@JosePaumard
326. The Collector API : collectingAndThen
Set<Map.Entry<Integer, List<Person>>> set =
persons
.stream()
.collect(
Collectors.collectingAndThen(
Collectors.groupingBy(
Person::getAge),
// downstream, builds a map
Map::entrySet
// finisher, applied on the map
) ;
In this case « Map::entrySet » is a finisher
#DV13LBDS
@JosePaumard
330. 1st example
A Stream of movies
Optional<Entry<Integer, Long>> opt =
movies.stream().parallel()
.collect(
Collectors.collectingAndThen(
Collectors.groupingBy(
movie -> movie.releaseYear(),
Collectors.counting()
),
Map::entrySet
)
)
.stream()
.max(Map.Entry.comparingByValue()) ;
#DV13LBDS
@JosePaumard
331. 1st example
Building a map
of year / # of movies
Optional<Entry<Integer, Long>> opt =
movies.stream().parallel()
.collect(
Collectors.collectingAndThen(
Collectors.groupingBy(
movie -> movie.releaseYear(),
Collectors.counting()
),
Map::entrySet
)
)
.stream()
.max(Map.Entry.comparingByValue()) ;
#DV13LBDS
@JosePaumard
332. 1st example
Optional<Entry<Integer, Long>> opt =
movies.stream().parallel()
.collect(
Collectors.collectingAndThen(
Collectors.groupingBy(
movie -> movie.releaseYear(),
Collectors.counting()
),
Map::entrySet
)
)
.stream()
.max(Map.Entry.comparingByValue()) ;
#DV13LBDS
Then getting the
entry set
@JosePaumard
333. 1st example
Optional<Entry<Integer, Long>> opt =
movies.stream().parallel()
.collect(
Collectors.collectingAndThen(
Collectors.groupingBy(
movie -> movie.releaseYear(),
Collectors.counting()
),
Map::entrySet
)
)
.stream()
.max(Map.Entry.comparingByValue()) ;
#DV13LBDS
Then getting the
max by value
@JosePaumard
334. 1st example
Optional<Entry<Integer, Long>> opt =
movies.stream().parallel()
.collect(
Collectors.collectingAndThen(
Collectors.groupingBy(
movie -> movie.releaseYear(),
Collectors.counting()
),
Map::entrySet
)
)
.stream()
.max(Map.Entry.comparingByValue()) ;
#DV13LBDS
Returns the
year with the most
movies released
@JosePaumard
336. Going parallel
To ways to do it :
- create a parallel stream up front
- call the parallel() method on a given stream, or
sequential() to go sequential
When a terminal operation is initiated, the operations are
executed in parallel or not, depending on the mode of the
stream
#DV13LBDS
@JosePaumard
337. Things that will go wrong
Some operations, like findAny() dont give reproductible
results in parallel
Changing the source (which is forbidden) can lead to
unpredictable results even if the source is concurrent
aware
#DV13LBDS
@JosePaumard
338. Things that will not go wrong, but…
The parallel processing of a stream has to be freed from
constraints
- ordering is costly, relaxing ordering may lead to better
performances
- Collecting must be concurrent aware
#DV13LBDS
@JosePaumard
344. Conclusion
Why are lambdas introduced in Java 8 ?
Because it’s in the mood !
Because it allows to write more compact code !
#DV13LBDS
@JosePaumard
345. Compact code is better !
An example of compact code (in C)
#include "stdio.h"
main() {
int b=0,c=0,q=60,_=q;for(float i=-20,o,O=0,l=0,j,p;j=O*O,p=l*l,
(!_--|(j+p>4)?fputc(b?q+(_/3):10,(i+=!b,p=j=O=l=0,c++,stdout)),
_=q:l=2*O*l+i/20,O=j-p+o),b=c%q,c<2400;o=-2+b*.05) ;
}
http://www.codeproject.com/Articles/2228/Obfuscating-your-Mandelbrot-code
#DV13LBDS
@JosePaumard
347. Conclusion
Why are lambdas introduced in Java 8 ?
Because it’s in the mood !
Because it allows to write more compact code !
#DV13LBDS
@JosePaumard
348. Conclusion
Why are lambdas introduced in Java 8 ?
Because it brings new and efficient patterns,
that enable easy and safe parallelization,
- needed by modern applications
- needed by API writers
#DV13LBDS
@JosePaumard
349. Conclusion
Java 8 is coming, it’s the biggest update in 15 years
#DV13LBDS
@JosePaumard
350. Conclusion
Java 8 is coming, it’s the biggest update in 15 years
Moving to Java 8 means a lot of work for us developers !
- Self training
- Changing our habits
#DV13LBDS
@JosePaumard
351. Conclusion
Java 8 is coming, it’s the biggest update in 15 years
Moving to Java 8 means a lot of work for us developers !
- Self training
- Changing our habits
- Convincing our bosses (sigh…)
#DV13LBDS
@JosePaumard
352. Conclusion
Java 8 is coming, it’s the biggest update in 15 years
Download the develop preview version on Open JDK site
Release date : 18th march 2014 !
#DV13LBDS
@JosePaumard
353. Conclusion
Java 8 is coming, it’s the biggest update in 15 years
Devoxx openning keynote
Brian Goetz : « Lamda a peek under the hood », Wed
12:00
Paul Sandoz : « In full flow : Java 8 lambdas in the
stream », Wed 14:00
#DV13LBDS
@JosePaumard
354. Conclusion
Java 8 is coming, it’s the biggest update in 15 years
« Java is a blue collar language. It’s not PhD thesis
material but a language for a job » – James Gosling, 1997
#DV13LBDS
@JosePaumard
355. Conclusion
Java 8 is coming, it’s the biggest update in 15 years
« Language features are not a goal unto themselves;
language features are enablers, encouraging or
discouraging certain styles and idioms » – Brian Goetz,
2013
#DV13LBDS
@JosePaumard
356. Conclusion
Java 8 is coming, it’s the biggest update in 15 years
The good news is :
Java is still Java !
#DV13LBDS
@JosePaumard