Talk form IWST 2023

1. Garbage Collector Tuning
in Pathological Allocation Pattern Applications
nahuel.palumbo@inria.fr
Nahuel Palumbo - Sebastian Jordan Montaño - Guillermo Polito - Pablo Tesone - Stéphane Ducasse
1
IWST ’23

2. Motivation
“Pharo is slow”
VM Developers
Pharo-AI Developers
My application takes >1h30m
What are you
doing?
Loading a 3GB
dataset
Ok, let’s see the
memory management
2

3. Motivation
“Pharo The GC is slow”
DataFrame
3

4. Motivation
“Pharo The GC is slow”
~1min
=2.5min
>1h30m
DataFrame
4

5. Motivation
“Pharo The GC is slow”
3x
2x
1.5x
3.6x
DataFrame
5

6. Motivation
“Pharo The GC is slow”
>1h30m !!!!
DataFrame
~1h25m
6

7. Memory Management
Garbage Collection
7

8. Manually Memory Management
A work for devs?
data = malloc(size)
...
... use data ...
...
free(data)
Manually
Memory Management
8

9. Automatic Memory Management
Garbage Collectors
data = malloc(size)
...
... use data ...
...
free(data)
Manually
Memory Management
data = Data new
...
... use data ...
...
??????????
Compute the size
Allocate in the memory
Maybe move the data for
better use of the memory
Free the space when data
is not used anymore
9

10. Automatic Memory Management
Garbage Collectors
data = malloc(size)
...
... use data ...
...
free(data)
Manually
Memory Management
data = Data new
...
... use data ...
...
??????????
Compute the size
Allocate in the memory
Maybe move the data for
better use of the memory
Free the space when data
is not used anymore
Developer 10

11. Automatic Memory Management
Garbage Collectors
data = malloc(size)
...
... use data ...
...
free(data)
Manually
Memory Management
data = Data new
...
... use data ...
...
??????????
Compute the size
Allocate in the memory
Maybe move the data for
better use of the memory
Free the space when data
is not used anymore
Developer 11

12. Application’s Allocation Patterns
How do the applications use the memory?
Memory
Allocations are particular for each application
Hard to predict
There are some general heuristics
Easy to predict
12

13. Application’s Allocation Patterns
Weak generational hypothesis
data = Data new
... first use of data ...
??????????
Will you use it again?
No, you can free
the memory
Yes, keep it!
>90%
<10%
“Most of the objects die young”
13

14. Application’s Allocation Patterns
When an object dies?
“Must be accessible from the roots”
ROOT
14

15. Application’s Allocation Patterns
When an object dies?
“Must be accessible from the roots”
ROOT
15

16. Application’s Allocation Patterns
When an object dies?
“Must be accessible from the roots”
ROOT
16

17. Application’s Allocation Patterns
When an object dies?
“Must be accessible from the roots”
ROOT
17

18. Application’s Allocation Patterns
When an object dies?
“Must be accessible from the roots”
ROOT
18

19. Application’s Allocation Patterns
When an object dies?
“Must be accessible from the roots”
ROOT
19

20. Application’s Allocation Patterns
When an object dies?
“Must be accessible from the roots”
ROOT
20

21. Application’s Allocation Patterns
When an object dies?
“Must be accessible from the roots”
ROOT
21

22. Generational Garbage Collector
High-Performance Automatic Memory Management for OOP
D. Ungar, Generation scavenging: A non-disruptive high performance storage reclamation algorithm
New Space Old Space
HEAP
ROOT
Remembered
Set Roots of the New Space (write barrier)
22

23. Generational Garbage Collector
High-Performance Automatic Memory Management for OOP
D. Ungar, Generation scavenging: A non-disruptive high performance storage reclamation algorithm
New Space Old Space
HEAP
ROOT
Remembered
Set Roots of the New Space (write barrier)
Can grow!
23

24. Generational Garbage Collector
High-Performance Automatic Memory Management for OOP
D. Ungar, Generation scavenging: A non-disruptive high performance storage reclamation algorithm
New Space Old Space
HEAP
ROOT
Scavenge
Remembered
Set Roots of the New Space (write barrier)
Can grow!
FAST
(often)
24

25. Generational Garbage Collector
High-Performance Automatic Memory Management for OOP
D. Ungar, Generation scavenging: A non-disruptive high performance storage reclamation algorithm
New Space Old Space
HEAP
ROOT
FULL GC
Scavenge
Remembered
Set Roots of the New Space (write barrier)
Can grow!
FAST
(often)
SLOW
(occasionally)
25

26. Long lifetimes
Memory-starved
Pathological Allocation Pattern
Garbage Collectors’ problem
Weak generational
Stable memory use
Few Full GCs
Fast Scavengers
Many Full GCs
Scavenger overhead
26

27. Long lifetimes
Memory-starved
Pathological Allocation Pattern
Garbage Collectors’ problem
Weak generational
Stable memory use
Few Full GCs
Fast Scavengers
Many Full GCs
Scavenger overhead
27

28. Pathological Allocation Pattern
Tuning the Garbage Collector
New Space Old Space
HEAP
ROOT
FULL GC
Scavenge
Remembered
Set
28

29. Pathological Allocation Pattern
Tuning the Garbage Collector
New Space Old Space
HEAP
ROOT
FULL GC
Scavenge
Remembered
Set
29

30. Pathological Allocation Pattern
Tuning the Garbage Collector
New Space Old Space
HEAP
ROOT
FULL GC
Scavenge
Remembered
Set
30

31. Pathological Allocation Pattern
Tuning the Garbage Collector
New Space Old Space
HEAP
ROOT
FULL GC
Scavenge
Remembered
Set
31

32. Pathological Allocation Pattern
Tuning the Garbage Collector
New Space Old Space
HEAP
ROOT
FULL GC
Scavenge
Remembered
Set
Developer 32

33. How should I tune the GC
parameters for my application?
33

34. Our methodology for GC tuning
Pro
fi
le GC events
From Scavenges:
•Amount of memory used (before and after).
•Size of the Remembered Set (before and after).
•Tenuring info (amount of data - threshold).
•Executed time.
From FullGC:
•Time spent marking/sweeping/compacting.
•Executed time.
DataFrame
Logs
Plots
Tuning
Developer
34

35. How Memory Grows
The overhead
Time
35

36. How Memory Grows
The overhead
Time
I run some FullGCs
when memory grows
so much Don’t do that
36

37. How Memory Grows
The tuning solution
FullGC Ratio - Threshold for triggering a FullGC
when the old space grows more than expected
Grow Headroom - Minimum amount of
memory that the GC will order from the OS
DataFrame
I will load 3GB
of data
37

38. How Memory Grows
The tuning solution
Time
38

39. How Memory Grows
The tuning solution
Time
Developer
1% - 5% faster!
Just that?
39

40. Deeper in the allocation pattern
Generational clash
40

41. Deeper in the allocation pattern
Generational clash
BIG
OVERHEAD
41

42. Deeper in the allocation pattern
Generational clash
New Space Old Space
HEAP
Scavenge
Remembered
Set
Roots of the New Space
42

43. Deeper in the allocation pattern
Generational clash
New Space Old Space
HEAP
Scavenge
Remembered
Set
Roots of the New Space
43

44. Deeper in the allocation pattern
Generational clash
New Space Old Space
HEAP
Scavenge
Remembered
Set
Roots of the New Space
44

45. Deeper in the allocation pattern
Generational clash
New Space Old Space
HEAP
Scavenge
Remembered
Set
Roots of the New Space
45

46. Deeper in the allocation pattern
Remembered Set overhead
The Remembered Set is large (lot of objects) * No chart :( *
The objects in the Remembered Set are large
DataFrame
46

47. Deeper in the allocation pattern
THE DataFrame
DataFrame
In the New
Space
47
In the Old
Space

48. Long Scavenges
The tuning solution
How? I don’t have any
algorithm for that
We need to avoid having large
objects in the Remembered Set
Then, close the New Space
48

49. Long Scavenges
The tuning solution
Tenuring threshold - Desired number of objects
already in the New Space for tenuring to the Old Space
DataFrame
Close the
door
New
Space
49

50. Long Scavenges
The tuning solution
50

51. Long Scavenges
The tuning solution
12.5x faster!
Developer 51

52. Conclusions
52

53. Final result
>1h30m
~7mins
53

54. Conclusions
New Old Space
HEAP
ROOT
FULL GC
Scavenge
Remembered
Set
54

55. Conclusions
New Old Space
HEAP
ROOT
FULL GC
Scavenge
Remembered
Set
DataFrame
Tuning
Developer
55

56. Conclusions
New Old Space
HEAP
ROOT
FULL GC
Scavenge
Remembered
Set
DataFrame
Tuning
Developer
56

57. My questions
• How much should devs know about their applications?
• How much should devs know about Garbage Collection algorithms?
• How much should devs know about the running VM?
57

58. Conclusions
New Old Space
HEAP
ROOT
FULL GC
Scavenge
Remembered
Set
DataFrame
Tuning
Developer
58

59. Conclusions
New Old Space
HEAP
ROOT
FULL GC
Scavenge
Remembered
Set
DataFrame
Tuning
Developer
Questions?
59