Backpressure--resistance opposing the desired flow of data through software--from massive scale, to death by a thousand cuts, it's something nearly every software engineer will have to deal with at some point. For some, it’s a frequent problem. But the term itself isn’t nearly as understood and recognized as such. In this talk, we'll discover what backpressure really is, when it happens, and the strategies you can apply to deal with it. (presented at Uphill Conf 2019)

1. Backpressure?
Resistance is not futile.
Jay Phelps | @_jayphelps

2. Jay Phelps | @_jayphelps
Jay Phelps
@_jayphelps
Senior Software Engineer
Citadel

3. Jay Phelps | @_jayphelps
Backpressure

4. Jay Phelps | @_jayphelps
A term borrowed from fluid dynamics,
like in automotive exhaust and house
plumbing.

5. Jay Phelps | @_jayphelps
“Resistance or force opposing the
desired flow of fluid through pipes.”

6. Jay Phelps | @_jayphelps
“Resistance or force opposing the
desired flow of fluid through pipes.”

7. Jay Phelps | @_jayphelps
“Resistance or force opposing the
desired flow of fluid through pipes.”
^ data through software.

8. Jay Phelps | @_jayphelps
“Resistance or force opposing the
desired flow of data through software.”

9. Jay Phelps | @_jayphelps
i.e. “input is coming in faster
than we can output”

10. Jay Phelps | @_jayphelps
Cool

11. Jay Phelps | @_jayphelps
…but…what is Backpressure?

12. Jay Phelps | @_jayphelps
Let’s start with an analogy!

13. Jay Phelps | @_jayphelps
I Love Lucy: Chocolate Factory

14. "I Love Lucy" Job Switching - 1952 - CBS
"I Love Lucy" Job Switching - 1952 - CBS
video: https://bit.ly/1eUkJgG

15. Jay Phelps | @_jayphelps
Lucy has Backpressure!

16. Jay Phelps | @_jayphelps
She tries to set them to the side
(buffering)

17. Jay Phelps | @_jayphelps
Then she tries eating and hiding them
(dropping)

18. Jay Phelps | @_jayphelps
She needs to slow down the conveyor belt
(producer control)

19. Jay Phelps | @_jayphelps
Let’s talk software

20. Jay Phelps | @_jayphelps
Reading and writing files
File Systems

21. Jay Phelps | @_jayphelps
Writing is slower than reading

22. Jay Phelps | @_jayphelps
Read: 150 MB/s
Write: 100 MB/s
Example hard drive

23. Jay Phelps | @_jayphelps
Example hard drive
150 MB/s - 100 MB/s = 50 MB/sdeficit

24. Jay Phelps | @_jayphelps
Imagine needing to read/write a 6 GB file
Example hard drive

25. Jay Phelps | @_jayphelps
Example hard drive
6 GB / 150 MB read = 40 seconds

26. Jay Phelps | @_jayphelps
Example hard drive
50 MB deficit x 40 sec = 2 GB memory!

27. Jay Phelps | @_jayphelps
That’s a lot of wasted memory

28. Jay Phelps | @_jayphelps
Solution: only read as fast as you can write
(control the producer)

29. Jay Phelps | @_jayphelps
Most I/O libraries do this for you, automatically
Node.js Streams is a great example

30. Jay Phelps | @_jayphelps
const zlib = require('zlib');
const fs = require('fs');
const gzip = zlib.createGzip();
const input = fs.createReadStream('input.txt');
const output = fs.createWriteStream('input.txt.gz');
// handles backpressure for you
input.pipe(gzip).pipe(output);

31. Jay Phelps | @_jayphelps
Server-to-Server Communication

32. A
Server
B
Server
C
Server
Jay Phelps | @_jayphelps

33. A
Server
B
Server
C
Server
100 rps 75 rps
Jay Phelps | @_jayphelps

34. A B C
Server Server Server
60 sec * 25 rps = 1,500 rpm!
100 rps 75 rps
Jay Phelps | @_jayphelps

35. A B C
Server Server Server
100 rps 75 rps
60 sec * 60 min * 25 rps = 90,000 rph!
Jay Phelps | @_jayphelps

36. Jay Phelps | @_jayphelps
Solution: control the producer (or scale up)

37. Jay Phelps | @_jayphelps
…unfortunately, that isn’t easy

38. Jay Phelps | @_jayphelps
RSocket, gRPC, etc

39. Jay Phelps | @_jayphelps
UI Rendering

40. Jay Phelps | @_jayphelps
Throttling/debouncing keyboard input

41. Jay Phelps | @_jayphelps
High volume WebSockets

44. Jay Phelps | @_jayphelps
Events are coming in faster
than we can render them

45. Jay Phelps | @_jayphelps
Solution: control the producer?

46. Jay Phelps | @_jayphelps
Is this even a good User Experience?

47. Jay Phelps | @_jayphelps
Performance problems are often UX problems!

48. Jay Phelps | @_jayphelps
Buffer or Drop/Sample?

49. Jay Phelps | @_jayphelps
Maybe table virtualization too?

50. VirtualizedNOT Virtualized vs.

51. Jay Phelps | @_jayphelps
Backpressure Strategies

52. Jay Phelps | @_jayphelps
If you can, scale up your resources

53. Jay Phelps | @_jayphelps
Three fundamental strategies

54. Jay Phelps | @_jayphelps
Control the producer
Buffer
Drop

55. Jay Phelps | @_jayphelps
Control the producer
slow down/speed up is decided by consumer

56. Jay Phelps | @_jayphelps
const source = connectToSource();
source.pull(response1 => {
console.log(response1);
// later…
source.pull(response2 => {
console.log(response2);
});
});

57. Jay Phelps | @_jayphelps
Controlling is usually the ideal option
but not always viable

58. Jay Phelps | @_jayphelps
Buffer
accumulate incoming data spikes temporarily

59. Jay Phelps | @_jayphelps
Be careful with unbounded buffers!

60. Jay Phelps | @_jayphelps
Drop
sample a percentage of the incoming data

61. Jay Phelps | @_jayphelps
Throttle, debounce, etc

62. Jay Phelps | @_jayphelps
Not always acceptable to lose data

63. Jay Phelps | @_jayphelps
Libraries for handling Backpressure

64. Jay Phelps | @_jayphelps
Probably streams, but maybe not!

65. Jay Phelps | @_jayphelps
Push-based streams
RxJS, Bacon.js, xstream

66. Jay Phelps | @_jayphelps
Push: producer is in control

67. Jay Phelps | @_jayphelps
Pull-based streams
Node.js Streams, Web Streams, Async Iterators, IxJS

68. Jay Phelps | @_jayphelps
Pull: consumer is in control

69. Jay Phelps | @_jayphelps
It’s not unusual to use both push
and pull streams in the same app

70. Jay Phelps | @_jayphelps
“Resistance opposing the
desired flow of data through software.”

71. Jay Phelps | @_jayphelps
i.e. “input is coming in faster
than we can output”

72. Jay Phelps | @_jayphelps
Control the producer
Buffer
Drop

73. Jay Phelps | @_jayphelps
Thanks!
@_jayphelps