"The Distributed Web" A talk about building distributed web apps for the second Research & Development Meetup

1. RDM#2
LISBON
Research & Development Meetup

2. LISBON
David Dias,
P2P SE&R @ Protocol Labs
João Ascensão,
Data Scientist @ Uniplaces
tw: @daviddias gh: @diasdavid
tw: @jtascensao gh: @jtascensao

3. LISBON
What is RDM?

4. LISBON
Space to invest time in learning
things that are hard to grasp

5. LISBON
world of possibilities

6. LISBON
what actually
gets researched

7. LISBON
what gets
developed

8. LISBON
what gets
deployed

9. LISBON
what most people are doing

10. PROTOCOL LABS

11. LISBON
FORMAT
- New topics / Research
- Hacking + Digesting + Publishing

12. LISBON
TODAY
- The Distributed Web
- Machine Learning / AI

14. The Distributed Web

22. Serve data to everywhere

23. • Motivations
• IPFS, the Protocol
• MerkleDAG (Directed Acyclic Graph)
• DHT (Distributed Hash Table)

24. • Motivations
• IPFS, the Protocol,
MerkleDAG, IPNS,
Network
• IPFS in action
• IPFS, the project
• filecoin
• libp2p
• starship
• gx
• full IPFS JavaScript
implementation
• mutable file system
and the files API
• Other DHT
• Structure P2P Networks
• Unstructured P2P
Networks
• NAT Traversal
• Relay/Turn
• registry-mirror
• smart chunking
• Transports
• Multiplexers
• Self Describing Data
Types
• Search
• Station
• and more

28. Location Addressing

29. http://10.20.30.40/foo/bar/baz.png
location path
http://example.com/foo/bar/baz.png

30. http://10.20.30.40/foo/bar/baz.png
you

31. http://10.20.30.40/foo/bar/baz.png
10.20.30.40
you

32. http://10.20.30.40/foo/bar/baz.png
10.20.30.40
you

37. 200 MB

38. 200 MB x 30 x 8 = 48 GB

39. 2,544,837,694

40. 2,387,990,609
508.968 PB

41. Bandwidth

43. https://www.backblaze.com/blog/price-gap-storage-vs-bandwidth/

44. Latency

45. Amazon Google
DataCenters

46. Disconnected / Offline

53. low bw interference congestion travelisp outagedatacenter

54. Permanence Security
IoT
Control
Offline
Bandwidth
Disconnected

55. InterPlanetary FileSystem

56. a protocol to upgrade the web

57. InterPlanetary FileSystem

59. InterPlanetary FileSystem

60. DISTRIBUTED
PERMANENT
MERKLE
THE WEB

61. PermanentDistributed Safer
SmarterOffline
Faster

62. SFS
web
DHT+

63. web
DHT
SFS
routing
network
exchange
merkledag
naming
applications
The Stack

64. routing
network
exchange
merkledag
naming
applicationsThe Stack
Defining the Data
Moving the Data
Using the Data
IPLD
libp2p
IPNS

66. 8AyaTQMhJklMo
Ga3WaDgwcAEPc
KQeipgDu64qaj
kr3fNcVt1LVMD
KQsXSobGtbNhF
sv3qA44E7iXAy
gcuSfVhW8RV3z
R9F3MFvbVaqf9

67. 8AyaTQMhJklMo
Ga3WaDgwcAEPc
KQeipgDu64qaj
kr3fNcVt1LVMD
hash( ) -> KQsXSobGtbNhF
KQsXSobGtbNhF
sv3qA44E7iXAy
gcuSfVhW8RV3z
R9F3MFvbVaqf9

68. 8AyaTQMhJklMo
Ga3WaDgwcAEPc
KQeipgDu64qaj
kr3fNcVt1LVMD
hash( ) -> KQsXSobGtbNhF
KQsXSobGtbNhF
sv3qA44E7iXAy
gcuSfVhW8RV3z
R9F3MFvbVaqf9
MERKLE LINK

75. cvs/svn

76. cvs/svn

77. cvs/svn

83. merkledag

84. in IPFS
data forms a dag
it’s called the merkle dag
because the links are hashes

85. any data structures are
represented as dags

86. unix files
and dirs
any data structures are
represented as dags

87. blob, tree, commit
unix files
and dirs
any data structures are
represented as dags

88. blob, tree, commit
unix files
and dirs
any data structures are
represented as dags
blocks, txns, wallets

89. blob, tree, commit
unix files
and dirs
any data structures are
represented as dags
blocks, txns, wallets
kv-stores

90. blob, tree, commit
unix files
and dirs
any data structures are
represented as dags
blocks, txns, wallets
kv-stores

91. unixfs as a dag

92. files are dag nodes
unixfs as a dag

93. files are dag nodes
BIG files may be
split into many
unixfs as a dag

94. directories are
also dag nodes
files are dag nodes
BIG files may be
split into many
unixfs as a dag

95. which link
to others
directories are
also dag nodes
files are dag nodes
BIG files may be
split into many
unixfs as a dag

96. http://10.20.30.40/foo/bar/baz.png
location path
http://example.com/foo/bar/baz.png

97. http://10.20.30.40/foo/bar/baz.png
location path
http://example.com/foo/bar/baz.png
/ipns/example.com/foo/bar/baz.png
/ipfs/QmW98pJrc6FZ6/foo/bar/baz.png
content path

98. 10.20.30.40
you
HTTP
http://10.20.30.40/foo/bar/baz.png
/ipfs/QmW98pJrc6FZ6/foo/bar/baz.png

99. http://10.20.30.40/foo/bar/baz.png
10.20.30.40
you
/ipfs/QmW98pJrc6FZ6/foo/bar/baz.png
IPFS

100. /ipfs/QmW98pJrc6FZ6/foo/bar/baz.png
content path

101. /ipfs/QmW98pJrc6FZ6/foo/bar/baz.png
content path

102. /ipfs/QmW98pJrc6FZ6/foo/bar/baz.png
content path
foo
bar
baz.png
QmW98pJrc6FZ6

103. /ipfs/QmW98pJrc6FZ6/foo/bar/baz.png
content path
foo:HB
HB
HC
HD
bar:HC
baz.png:HDQmW98pJrc6FZ6
foo
bar
baz.png

104. DHT

105. DHT are the Holy Grail* of P2P Networks
*from a survey paper

106. 1
5
6
10
50
55

107. 1 10 12 20 30 50 54 58 70 88 81 83 88 90 92 95 97 99 101105 300411 500

108. 1 10 12 20 30 50 54 58 70 88 81 83 88 90 92 95 97 99 101105 300411 500

109. 1 10 12 20 30 50 54 58 70 88 81 83 88 90 92 95 97 99 101105 300411 500
hash( )

110. 1 10 12 20 30 50 54 58 70 88 81 83 88 90 92 95 97 99 101105 300411 500
hash( ) = 91

111. 1 10 12 20 30 50 54 58 70 88 81 83 88 90 92 95 97 99 101105 300411 500
hash( ) = 91

112. 1 10 12 20 30 50 54 58 70 88 81 83 88 90 92 95 97 99 101105 300411 500
Who has fim , which hash( ) = 91?

113. The Distributed Web

114. • Applications with no-origin
• Leverage network cache
• Resilient to network splits

115. Serve data to everywhere

116. Serve data from everywhere

117. David Dias - http://daviddias.me IPFS http://ipfs.io Protocol Labs http://ipn.io

118. Thank you!
David Dias - http://daviddias.me IPFS http://ipfs.io Protocol Labs http://ipn.io