Presented 15/11/2018 at Devoxx Be

1. Delivering Beautiful and Fast
Images and Video
Doug Sillars
@DougSillars
Devoxx Belgium
November 15, 2018

2. Contact Me:
@DougSillars
Doug.Sillars@gmail.com
www.dougsillars.com
Doug Sillars
Freelance Developer Relations
Performance Audits: Web/Native
Workshops:
Performance/Images/Video
http://bit.ly/HighPerformanceAndroidApps

4. 0.5
0.6
0.7
0.8
0.9
Standing in Line Standing on the
edge of a virtual
cliff
Experiencing
Mobile Delays
Solving a Math
Problem
https://www.ericsson.com/res/docs/2016/mobility-report/emr-feb-2016-the-stress-of-steaming-delays.pdf
Stress

5. Large Downloads Induce Delays in Rendering
3s: 53% of Users Abandon Mobile Sites
500ms: 26% Frustration
8% Engagement
100ms: 1% Revenue Walmart & Amazon (Desktop 2001)
4% Mobile Users Throw Their Phones
https://www.doubleclickbygoogle.com/articles/mobile-speed-matters
http://bit.ly/mobileWebStress
http://www.globaldots.com/how-website-speed-affects-conversion-rates/
https://www.mobilejoomla.com/blog/172-responsive-design-vs-server-side-solutions-infographic.html

6. Images & Video make up 75% of Web Content
HTTPArchive mobile websites 15/02/18

7. 7.9 MB

8. 3.5 MB

9. 4 Simple Image Optimizations

10. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

11. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

12. 4 Simple Image Optimizations
Httparchive.org Webpagetest.org

13. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

14. Image Quality
magick -quality 85 riga.jpg riga85.jpg
http://res.cloudinary.com/dougsillars/image/upload/q_85/v1520504964/IMG_20180301_114117_tzasan.jpg
Lighthouse:
Recommends 85% quality on all images
https://developers.google.com/speed/docs/insights/OptimizeImages

15. 100%
3.6 MB
http://res.cloudinary.com/dougsillars/image/upload/v1529005982/IMG_20180614_184507_ssuk1i.jpg

16. 85%
1.87 MB
q_85
http://res.cloudinary.com/dougsillars/image/upload/q_85/v1529005982/IMG_20180614_184507_ssuk1i.jpg

17. Image Quality Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

18. Image Quality Use “In The Wild”
Median Savings (50th percentile):
• 2.83 seconds faster page load
• 419KB less data

19. 50%
914 KB
q_50
http://res.cloudinary.com/dougsillars/image/upload/q_50/v1529005982/IMG_20180614_184507_ssuk1i.jpg

20. 20%
513 KB
q_20
http://res.cloudinary.com/dougsillars/image/upload/q_20/v1529005982/IMG_20180614_184507_ssuk1i.jpg

21. Graphing the Results
https://developers.google.com/speed/docs/insights/OptimizeImages

22. Graphing the Results
https://developers.google.com/speed/docs/insights/OptimizeImages

23. Automate Quality vs. File Size
• Butteraugli
• SSIM: Structural SIMilarity
https://github.com/technopagan/cjpeg-dssim
cjpeg-dssim jpegoptim riga.jpg
http://res.cloudinary.com/dougsillars/image/upload/q_auto/v1529005982/IMG_20180614_184507_ssuk1i.jpg

24. SSIM
1.46 MB
http://res.cloudinary.com/dougsillars/image/upload/q_auto/v1529005982/IMG_20180614_184507_ssuk1i.jpg

25. Graphing the Results
https://developers.google.com/speed/docs/insights/OptimizeImages

26. Results:
• Test Load on Motorola G4:

27. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

28. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

29. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

30. Scalable Vector Graphics (SVG)
Images drawn as shapes
infinitely scalable
XML - Can be added inline to HTML document
https://gist.github.com/hail2u/2884613?short_path=66a60ff

31. Scalable Vector Graphics (SVG)
Images drawn as shapes
infinitely scalable
XML - Can be added inline to HTML document
https://gist.github.com/hail2u/2884613?short_path=66a60ff

32. Scalable Vector Graphics (SVG)

33. Scalable Vector Graphics (SVG)

34. Scalable Vector Graphics (SVG)
https://github.com/google/brotli

35. Image Formats – Average Size
HTTPArchive mobile websites 15/02/18

36. File Format: WebP
https://caniuse.com/#feat=webp

38. SSIM + WebP
986 KB
http://res.cloudinary.com/dougsillars/image/upload/q_auto,f_auto/v1529005982/IMG_20180614_184507_

39. File Format: Web
<picture>
<source width = "100%" type="image/webp" srcset=”riga.webp">
<img width = "100%" src=”riga_cjpeg_dssim.jpg"
alt=”Riga, Latvia">
</picture>

40. Image Format Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

41. Image Format Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

42. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

43. Image Sizing
-

44. Image Sizing

45. Image Sizing
-

46. Image Sizing
624
832
-

47. Image Sizing
624
832
12,979,000 pixels
519,000 pixels
_________________
-
12,460,000 pixels

48. Image Sizing
624
832
12,979,000 pixels
519,000 pixels
_________________
-
12,460,000 pixels

49. Image Sizing
https://twitter.com/paulcalvano/status/928751141843808256

50. Image Sizing
-
Responsive Images:
Generate a set of images
25 KB difference in size

51. Responsive Images
624
832
625,000 pixels
-
106,000 pixels
519,000 pixels
_________________

52. Responsive Breakpoint Generation
https://github.com/cloudinary/responsive_breakpoints_generator
http://www.responsivebreakpoints.com/

53. Responsive Images

54. Responsive Images

55. Responsive Images Use “In The Wild”
500,000 mobile sites
Analyzed 3/15/18

56. Responsive Images Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

57. 4 Simple Image Optimizations
1. Quality
2. Format
3. Sizing
4. Lazy Loading

58. Lazy Load
https://calendar.perfplanet.com/2017/progressive-image-loading-using-intersection-observer-and-sqip/

59. Lazy Load
https://calendar.perfplanet.com/2017/progressive-image-loading-using-intersection-observer-and-sqip/

60. Lazy Loading Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

61. Lazy Loading Use “In The Wild”
442,000 mobile sites
Analyzed 3/15/18

62. Preview Images

63. Optimizing Content Delivery
Preview Images
https://github.com/technopagan/sqip

64. Lazy Loading: Experiments

65. Lazy Loading: Experiments

66. Lazy Loading: Experiments

67. Animated GIFs
Original MP4
1.4 MB

68. Animated GIFs

69. Animated GIFs
Animated GIF
3.8 MB
270% larger

70. Animated GIFs

71. Animated GIFs
MP4: 256 colors
247KB
93% smaller

72. Animated GIFs
Video Tags:
<video loop autoplay muted playsinline controls = "false” src="goats.mp4”/>
Video is not pre-loaded, will be last to download
Img tags are fast!
<picture>
<source type="video/mp4" srcset=”goats.mp4">
<source type="image/webp" srcset=”goats.webp">
<img src=”goats.gif">
</picture>
https://calendar.perfplanet.com/2017/animated-gif-without-the-gif/

73. Animated GIFs

74. Video

75. 75
Video
Buffer Rage: a state of uncontrollable
fury or violent anger induced by delayed
or interrupted streaming video content
http://www.ineoquest.com/press-releases/new-research-reveals-buffer-rage-as-techs-newest-epidemic

76. Video Quality Metrics
1.Does the Video Start?
2.Does the Video Stall?
3.Does it Look Good?

77. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018:

78. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
16.9B total Video plays
400M Fail to Start
2B Abandoned before Start
~800M hours of video
playback lost

79. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
16.9B total Video plays
400M Fail to Start
2B Abandoned before Start
~800M hours of video
playback lost

80. Video Fails To Start

81. Video Startup Failure

82. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
16.9B total Video plays
400M Fail to Start
2B Abandoned before Start

83. Video Startup Delay
After 2 seconds,
every additional
second corresponds
to 5.8% increase in
abandonment
https://www.akamai.com/kr/ko/multimedia/documents/technical-publication/video-stream-quality-impacts-viewer-behavior-inferring-causality-using-quasi-experimental-designs-technical-publication.pdf

84. Video Startup Delay
https://www.akamai.com/kr/ko/multimedia/documents/technical-publication/video-stream-quality-impacts-viewer-behavior-inferring-causality-using-quasi-experimental-designs-technical-publication.pdf

85. Preload When It Makes Sense
Ad Playback Video
Download

86. Preload When It Makes Sense
Ad Playback Video
Download

87. Preload When It Makes Sense
Ad Playback
Video
Download

88. ..be Careful with Video Preload
Preload = “auto”

89. …be Careful with Preload

90. Background Video

91. Video Background
Video (as Downloaded): 5.3 MB
Video 5 MB
Audio 250 KB 5% of file
Best Practice:
To save bandwidth, remove the audio stream from videos that
are played silently.

92. Video Background: Mobile
Best Practice:
If Viewport will not support Video…
Don’t Download it

93. Video Background

94. Video Background
33.6 MB
27s
2560 x 1226
10 MBPS

95. Video Background
Best Practice:
Resize Video to reasonable size.
33.6 MB
27s
2560 x 1226
10 MBPS

96. Video Background
Best Practice:
Resize Video to reasonable size.
PROTIP: Renaming the file is not enough…

97. Video Resizing
• 1920x1080: 8.1 MB
• 1280x720: 4.3 MB
• 1080x608: 3.3 MB
• 720x405: 1.76 MB
http://res.cloudinary.com/dougsillars/video/upload/vc_auto,w_720/v1533591785/depend_p2ryou.mp4

98. Video Third Parties

99. Video Third Parties

100. Video Third Parties

101. Video Third Parties

102. Video Streaming
Manifest File:
List of Available
Streams
Player Chooses a Stream Stream Manifest:
List of Video Segments
Player Downloads
Segments into buffer
Video Plays

103. Video Plays
Optimizing Video Delivery
Video Streaming
Player can estimate
network throughput
Optimal video Bitrate
HTTPArchive
Mobile: 1.6 MBPS
Desktop 5 MBPS

104. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8
Video Tracks
Video Tracks (iFrames)
Audio Tracks &
Subtitles

105. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8
Video Tracks

106. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8

107. Optimizing Video Delivery
Video Streaming
Manifest File:
List of Available
Streams
Player selects high
bitrate stream Stream Manifest:
List of Video Segments
Buffer takes a long
time to fill
Video Does Not Play
Player chooses low
bitrate
Buffer Fills Quickly
Video Plays

108. Video Startup
Conviva creen_Streaming_TV_Census_Report_FINAL.pdf
Q1 2018: Video Startup
Average Video Start Time:
N. America: 4.91s
Asia: 3.00s
Europe: 4.30s

109. Video Streaming: Startup Time
Low -> High
High => Low
Middle (Goldilocks)

110. Video Startup
10s 11s 12s 13s 14s 15s 16s 17s
4k Low->High:
4K Middle
(Goldilocks):
4k:High->Low:

111. Video Startup
4k Low->High:
4K Middle
(Goldilocks):
4k:High->Low:
PROs: CONS:
Fast Startup Initial Quality: Low
Other Connections:
Initial Quality: Low
Very slow startup
Fast Connections:
Initial Quality: High
Initial Quality: Good
Initial Startup: Not
fast, but not slow

112. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8 Testing with WebPageTest 3G

113. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8

114. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8

115. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8
Testing with WebPageTest 3G

116. Video Streaming: Manifest File
600k.m3u8

117. Video Streaming: Manifest File
600k.m3u8

118. Video Streaming: Manifest File
https://hls.ted.com/talks/2208.m3u8

119. Conclusion
Optimize Image:
Quality
Format
Sizing
Lazy Load if Possible
Only download Video when displayed
Strip audio if silent
Resize Videos for Mobile
Audit 3rd Party Videos
Streaming: Start with lower bitrates to speed video playback
Streaming: Conservative bitrates *may* reduce stalls, but will lower quality
Images
Video

120. Summary
Testing:
WebPageTest https://www.webpagetest.org
HttpArchive https://httparchive.org
Images:
ImageMagick https://www.imagemagick.org
SSIM https://github.com/technopagan/cjpeg-dssim
LazySizes https://github.com/aFarkas/lazysizes
Responsive Breakpoints http://www.responsivebreakpoints.com/
Cloudinary https://www.cloudinary.com
Tooling

121. Conclusion
Images and Video
Can Be Beautiful AND Fast