SlideShare a Scribd company logo
1 of 102
Download to read offline
Introduction to the IPFS
170629
SFC Arch B3 gentam
• IPFS
• IPFS
• IPFS
2
• IPFS
• IPFS
• IPFS
3
• IPFS
• IPFS
• IPFS
4
• The InterPlanetary File System
• : "Content Addressed, Versioned, P2P File System"
• Protocol Labs < https://protocol.ai> Juan Benet 2013
12
• Go < https://github.com/ipfs/go-ipfs>
5
• IPFS P2P
• DHTs, BitTorrent, Git, SFS ...
• IPFS 1 Merkle DAG
6
• 1
• Web IPFS Git
BitTorrent
• IPFS
• Permanent Web
7
• HTTP Web
• Web
• Faster: P2P ?
• Safer:
• Open:
8
• Blockchain
‣ IPFS
‣
‣ Blockchain IPFS
‣ IPFS Blockchain
9
– Juan Benet
“The contribution of IPFS is simplifying, evolving, and connecting proven
techniques into a single cohesive system, greater than the sum of its
parts.”
10
2 IPFS HP (ipfs.io)
11
• IPFS
• IPFS
• IPFS
12
• HTTP 1 1
• IPFS
• P2P
60%
(
• IPFS
13
•
• 100
• IPFS git
14
• Web
•
• IPFS
15
•
•
• IPFS
16
• IPFS
• IPFS
• IPFS
17
• IPFS
• IPFS
18
19
•
•
20
21
22
IPNS
23
• IPFS
• IPFS
• IPFS
24
IPFS
25
• IPFS
• IPFS
26
• IPFS
• IPFS
27
28
IPFS IPFS
• : "Content Addressed, Versioned, P2P File System"
• Protocol Labs <https://protocol.ai> Juan Benet 2013
12
• Go <https://github.com/ipfs/go-ipfs>
29
IPFS IPFS
• : "Content Addressed, Versioned, P2P File System"
• Protocol Labs < https://protocol.ai> Juan Benet 2013
12
• Go <https://github.com/ipfs/go-ipfs>
DHT, SFS
Git (+Blockchain)
SFS
• Routing - DHT
• Block Exchanges - BitTorrent
• Version Control Systems - Git
• Self-Certified Filesystems - SFS
30
• Routing - DHT
• Block Exchanges - BitTorrent
• Version Control Systems - Git
• Self-Certified Filesystems - SFS
31
• /
P2P
• ID
32
Kademlia
• ID XOR
• log(n) lookup (e.g. 10,000,000 20 )
•
• DoS
Kademlia DHT Erlang
33
S/Kademlia DHT
• Kademlia
• PKI PoW Sybil
• 0.85
34
Coral DSHT
• Distributed Sloppy Hash Table
• Kademlia ID " "
• → " " ( )
• " "
35
• Routing - DHT
• Block Exchanges - BitTorrent
• Version Control Systems - Git
• Self-Certified Filesystems - SFS
36
• BitTorrent P2P
• Bram Cohen 2001
• " *"
*
37
1. ( )
" "
2.
3. BitTorrent "tit-for-tat" ( =1 )
PropShare
38
• Routing - DHT
• Block Exchanges - BitTorrent
• Version Control Systems - Git
• Self-Certified Filesystems - SFS
39
•
• Git Merkle DAG*
* Merkle Directed Acyclic Graph ( ?) –
40
1. (blob), (tree), (commit)
2.
3. Merkle DAG
4. ( etc)
5.
6.
41
: Git
<https://www.slideshare.net/ktateish/git-concept1>
42
• Routing - DHT
• Block Exchanges - BitTorrent
• Version Control Systems - Git
• Self-Certified Filesystems - SFS
43
SFS
• SFS
/sfs/<Location>:<HostID>
Location =
HostID = hash(public_key || Location)
44
/sfs/<Location>:<HostID>
• SFS
• SFS
45
• IPFS
• IPFS
46
• IPFS
Identities, Network, Routing, Exchange, Objects, Files, Naming
47
1. Iden99es - ID
2. Network -
3. Rou9ng -
4. Exchange - (BitSwap)
5. Objects - Merkle DAG
6. Files - Git
7. Naming - self-certifying
48
1. Identities
2. Network
3. Routing
4. Exchange
5. Objects
6. Files
7. Naming
49
• NodeID
• S/Kademlia
• NodeID ID (
ID
type NodeId Multihash
type Multihash []byte
// self-describing cryptographic hash digest
type PublicKey []byte
type PrivateKey []byte
// self-describing keys
type Node struct {
NodeId NodeID
PubKey PublicKey
PriKey PrivateKey
}
50
• S/Kademlia IPFS ID :
difficulty = <integer parameter>
n = Node{}
do {
n.PubKey, n.PrivKey = PKI.genKeyPair()
n.NodeId = hash(n.PubKey)
p = count_preceding_zero_bits(hash(n.NodeId))
} while (p < difficulty)
• hash(other.PublicKey)
other.NodeID
51
• IPFS
• multihash :
<function code><digest length><digest bytes>
•
52
1. Identities
2. Network
3. Routing
4. Exchange
5. Objects
6. Files
7. Naming
53
• IPFS
•
• Transport, Reliability, Connectivity, Integrity, Authenticity
54
• Transport: IPFS
WebRTC DataChannl ( )
uTP (LEDBAT)
• Reliability:
IPFS uTP SCTP
55
• Connectivity: ICE (Interactive Connectivity Establishment) NAT
• Integrity:
( )
• Authenticity: HMAC
( )
56
• multihash
multiaddr
• IP
# an SCTP/IPv4 connection
/ip4/10.20.30.40/sctp/1234/
# an SCTP/IPv4 connection proxied over TCP/IPv4
/ip4/5.6.7.8/tcp/5678/ip4/1.2.3.4/sctp/1234/
57
1. Identities
2. Network
3. Routing
4. Exchange
5. Objects
6. Files
7. Naming
58
• IPFS :
•
•
• IPFS S/Kademlia Coral DSHT
• 1KB DHT (
NodeID)
59
type IPFSRouting interface {
FindPeer(node NodeId)
// gets a particular peer's network address
SetValue(key []bytes, value []bytes)
// stores a small metadata value in DHT
GetValue(key []bytes)
// retrieves small metadata value from DHT
ProvideValue(key Multihash)
// announces this node can serve a large value
FindValuePeers(key Multihash, min int)
// gets a number of peers serving a large value
}
•
60
1. Identities
2. Network
3. Routing
4. Exchange
5. Objects
6. Files
7. Naming
61
• BitSwap: BitTorrent
• have_list
• want_list
• BitTorrent 1
‣
( )
62
• BitSwap :
• Seeder ( )
• Leecher ( )
• " "
• ignore_cooldown
(10 sec)
63
• BitTorrent "tit-for-tat"
:
• BitTyrant: Selfish Peer
• BitThief:
• PropShare:
‣
64
• BitSwap debt ratio: r
( )
• r :
65
P(send | r) =
1
1 + e6 3r
r =
bytes send
bytes recv + 1
• ledger
• ledger
type Ledger struct {
owner NodeId
partner NodeId
bytes_sent int
bytes_recv int
timestamp Timestamp
}
• ledger
66
1. Open: ledgers
2. Sending: want_lists
3. Close:
4. Ignored( ):
67
1. Identities
2. Network
3. Routing
4. Exchange
5. Objects
6. Files
7. Naming
68
• DHT BitSwap P2P
• IPFS Git Merkle DAG
69
1. Content Addressing:
multihash
2. Tamper resistance:
3. Deduplica9on:
70
IPFS :
type IPFSLink struct {
Name string
// name or alias of this link
Hash Multihash
// cryptographic hash of target
Size int
// total size of target
}
type IPFSObject struct {
links []IPFSLink
// array of links
data []byte
// opaque content data
}
71
• ipfs (brew install ipfs)
• :
> ipfs ls /XLZ1625Jjn7SubMDgEyeaynFuR84ginqvzb
XLYkgq61DYaQ8NhkcqyU7rLcnSa7dSHQ16x 189458 less
XLHBNmRQ5sJJrdMPuu48pzeyTtRo39tNDR5 19441 script
XLF4hwVHsVuZ78FZK6fozf8Jj9WEURMbCX4 5286 template
<object multihash> <object size> <link name>
•
72
• UNIX Web
# format
/ipfs/<hash-of-object>/<name-path-to-object>
# example
/ipfs/XLYkgq61DYaQ8NhkcqyU7rLcnSa7dSHQ16x/foo.txt
73
• /ipfs
• " " /ipfs/
‣ :
/ipfs/<hash-of-foo>/bar/baz
/ipfs/<hash-of-bar>/baz
/ipfs/<hash-of-baz>
74
• ( )
‣ (
• " "
• DHT
‣
75
• IPFS
•
type EncryptedObject struct {
Object []bytes
// raw object data encrypted
Tag []bytes
// optional tag for encryption groups
}
type SignedObject struct {
Object []bytes
// raw object data signed
Signature []bytes
// hmac signature
PublicKey []multihash
// multihash identifying key
}
76
1. Identities
2. Network
3. Routing
4. Exchange
5. Objects
6. Files
7. Naming
77
• IPFS :
1. block: a variable-size block of data.
2. list: a collection of blocks or other lists.
3. tree: a collection of blocks, lists, or other trees.
4. commit: a snapshot in the version history of a tree.
78
• Git
:
1. (→objects )
2. (→ list )
3. commits tree
• IPFS Git
• JSON protobuf
79
• blob
• IPFS blob Git
{
"data": "some data here",
// blobs have no links
}
80
• list blob
{
"data": ["blob", "list", "blob"],
// lists have an array of object types as data
"links": [
{ "hash": "XLYkgq61DYaQ8NhkcqyU7rLcnSa7dSHQ16x",
"size": 189458 },
{ "hash": "XLHBNmRQ5sJJrdMPuu48pzeyTtRo39tNDR5",
"size": 19441 },
{ "hash": "XLWVQDqxo9Km9zLyquoC9gAP8CL1gWnHZ7z",
"size": 5286 }
// lists have no names in links
]
}
81
• tree : Git
{
"data": ["blob", "list", "blob"],
// trees have an array of object types as data
"links": [
{ "hash": "XLYkgq61DYaQ8NhkcqyU7rLcnSa7dSHQ16x",
"name": "less", "size": 189458 },
{ "hash": "XLHBNmRQ5sJJrdMPuu48pzeyTtRo39tNDR5",
"name": "script", "size": 19441 },
{ "hash": "XLWVQDqxo9Km9zLyquoC9gAP8CL1gWnHZ7z",
"name": "template", "size": 5286 }
// trees do have names
]
}
82
• commit
> ipfs file-cat <ccc111-hash> --json
{
"data": {
"type": "tree",
"date": "2014-09-20 12:44:06Z",
"message": "This is a commit message."
},
"links": [
{ "hash": "<ccc000-hash>",
"name": "parent", "size": 25309 },
{ "hash": "<ttt111-hash>",
"name": "object", "size": 5198 },
{ "hash": "<aaa111-hash>",
"name": "author", "size": 109 }
]
}
83
> ipfs file-cat <ttt111-hash> --json
{
"data": ["tree", "tree", "blob"],
"links": [
{ "hash": "<ttt222-hash>",
"name": "ttt222-name", "size": 1234 },
{ "hash": "<ttt333-hash>",
"name": "ttt333-name", "size": 3456 },
{ "hash": "<bbb222-hash>",
"name": "bbb222-name", "size": 22 }
]
}
84
1. Identities
2. Network
3. Routing
4. Exchange
5. Objects
6. Files
7. Naming
85
•
•
IPFS
• :
‣ (= ID)
‣
‣ (= ID)
‣ ...
86
‣ SFS
self-certified
87
• IPFS
1. NodeId = hash(node.PubKey)
2. NodeId " " :
/ipns/<NodeId>
3.
:
/ipns/XLF2ipQ4jD3UdeX5xp1KBgeHRhemUtaA8Vm/docs/hello
4. NodeId
88
• IPFS
1. NodeId = hash(node.PubKey)
2. NodeId " " :
/ipns/<NodeId>
3.
:
/ipns/XLF2ipQ4jD3UdeX5xp1KBgeHRhemUtaA8Vm/docs/hello
4. NodeId
88
f n
89
IPFS → 5. Object
• UNIX Web
# format
/ipfs/<hash-of-object>/<name-path-to-object>
# example
/ipfs/XLYkgq61DYaQ8NhkcqyU7rLcnSa7dSHQ16x/foo.txt
89
IPFS → 5. Object
• UNIX Web
# format
/ipfs/<hash-of-object>/<name-path-to-object>
# example
/ipfs/XLYkgq61DYaQ8NhkcqyU7rLcnSa7dSHQ16x/foo.txt
ipfs "
"
• "ipns" "ipfs"
• IPNS
:
1. IPFS
2.
routing.setValue(NodeId, <ns-object-hash>)
90
• IPNS
1. Peer Links
2. DNS TXT IPNS Records
3. Proquint Pronounceable Identifiers
4. Name Shortening Services
91
•
• web of trust
# Alice links to bob Bob
ipfs link /<alice-pk-hash>/friends/bob /<bob-pk-hash>
# Eve links to Alice
ipfs link /<eve-pk-hash/friends/alice /<alice-pk-hash>
# Eve also has access to Bob
/<eve-pk-hash/friends/alice/friends/bob
# access Verisign certified domains
/<verisign-pk-hash>/foo.com
92
• /ipns/<domain> IPFS DNS
TXT :
• IPFS IPNS
:
# this DNS TXT record
ipfs.benet.ai. TXT "ipfs=XLF2ipQ4jD3U ..."
# behaves as symlink
ln -s /ipns/XLF2ipQ4jD3U /ipns/fs.benet.ai
93
• IPFS
Proquint :
# this proquint phrase
/ipns/dahih-dolij-sozuk-vosah-luvar-fuluh
# will resolve to corresponding
/ipns/KhAwNprxYVxKqpDZ
94
•
• DNS Web URL :
# User can get a link from
/ipns/shorten.er/foobar
# To her own namespace
/ipns/XLF2ipQ4jD3UdeX5xp1KBgeHRhemUtaA8Vm
95
• IPFS
• IPFS
• IPFS
96
1. As a mounted global filesystem, under /ipfs
and /ipns.
2. As a mounted personal sync folder that
automatically versions, publishes, and backs
up any writes.
3. As an encrypted file or data sharing system.
4. As a versioned package manager for all
software.
5. As the root filesystem of a Virtual Machine.
6. As the boot filesystem of a VM (under a
hypervisor).
97
7. As a database: applications can write directly to
the Merkle DAG data model and get all the
versioning, caching, and distribution IPFS
provides.
8. As a linked (and encrypted) communications
platform.
9. As an integrity checked CDN for large files
(without SSL).
10.As an encrypted CDN.
11.On webpages, as a web CDN.
12.As a new Permanent Web where links do not die.
• <https://github.com/ipfs/awesome-ipfs>
• IPFSBin - Pastebin IPFS
• Interplanetary Wiki - IPFS Wiki
• ...
98
• Storj <https://storj.io/>
• Gitchain <http://gitchain.org/>
• ZeroTier One <https://www.zerotier.com/>
• MaidSafe <https://maidsafe.net>
• SIA < http://sia.tech/>
• Upspin <https://upspin.io>
99
• IPFS HP <https://ipfs.io/>
• White Paper <https://github.com/ipfs/papers/raw/master/ipfs-
cap2pfs/ipfs-p2p-file-system.pdf>
• the morning paper<https://blog.acolyer.org/2015/10/05/ipfs-content-
addressed-versioned-p2p-file-system/>
100

More Related Content

What's hot

What's hot (20)

Deep Dive on ClickHouse Sharding and Replication-2202-09-22.pdf
Deep Dive on ClickHouse Sharding and Replication-2202-09-22.pdfDeep Dive on ClickHouse Sharding and Replication-2202-09-22.pdf
Deep Dive on ClickHouse Sharding and Replication-2202-09-22.pdf
 
Linux Profiling at Netflix
Linux Profiling at NetflixLinux Profiling at Netflix
Linux Profiling at Netflix
 
Apache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic DatasetsApache Iceberg - A Table Format for Hige Analytic Datasets
Apache Iceberg - A Table Format for Hige Analytic Datasets
 
Understanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage EngineUnderstanding InfluxDB’s New Storage Engine
Understanding InfluxDB’s New Storage Engine
 
Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...
Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...
Cloud Native ClickHouse at Scale--Using the Altinity Kubernetes Operator-2022...
 
忙しい人の5分で分かるMesos入門 - Mesos って何だ?
忙しい人の5分で分かるMesos入門 - Mesos って何だ?忙しい人の5分で分かるMesos入門 - Mesos って何だ?
忙しい人の5分で分かるMesos入門 - Mesos って何だ?
 
Rootlessコンテナ
RootlessコンテナRootlessコンテナ
Rootlessコンテナ
 
All about Zookeeper and ClickHouse Keeper.pdf
All about Zookeeper and ClickHouse Keeper.pdfAll about Zookeeper and ClickHouse Keeper.pdf
All about Zookeeper and ClickHouse Keeper.pdf
 
Infrastructure & System Monitoring using Prometheus
Infrastructure & System Monitoring using PrometheusInfrastructure & System Monitoring using Prometheus
Infrastructure & System Monitoring using Prometheus
 
ORC File - Optimizing Your Big Data
ORC File - Optimizing Your Big DataORC File - Optimizing Your Big Data
ORC File - Optimizing Your Big Data
 
Tutorial: Using GoBGP as an IXP connecting router
Tutorial: Using GoBGP as an IXP connecting routerTutorial: Using GoBGP as an IXP connecting router
Tutorial: Using GoBGP as an IXP connecting router
 
My first 90 days with ClickHouse.pdf
My first 90 days with ClickHouse.pdfMy first 90 days with ClickHouse.pdf
My first 90 days with ClickHouse.pdf
 
Linux Performance Analysis: New Tools and Old Secrets
Linux Performance Analysis: New Tools and Old SecretsLinux Performance Analysis: New Tools and Old Secrets
Linux Performance Analysis: New Tools and Old Secrets
 
A Day in the Life of a ClickHouse Query Webinar Slides
A Day in the Life of a ClickHouse Query Webinar Slides A Day in the Life of a ClickHouse Query Webinar Slides
A Day in the Life of a ClickHouse Query Webinar Slides
 
The Patterns of Distributed Logging and Containers
The Patterns of Distributed Logging and ContainersThe Patterns of Distributed Logging and Containers
The Patterns of Distributed Logging and Containers
 
PostgreSQL on EXT4, XFS, BTRFS and ZFS
PostgreSQL on EXT4, XFS, BTRFS and ZFSPostgreSQL on EXT4, XFS, BTRFS and ZFS
PostgreSQL on EXT4, XFS, BTRFS and ZFS
 
Service-mesh options with Linkerd, Consul, Istio and AWS AppMesh
Service-mesh options with Linkerd, Consul, Istio and AWS AppMeshService-mesh options with Linkerd, Consul, Istio and AWS AppMesh
Service-mesh options with Linkerd, Consul, Istio and AWS AppMesh
 
Velocity 2015 linux perf tools
Velocity 2015 linux perf toolsVelocity 2015 linux perf tools
Velocity 2015 linux perf tools
 
Airflow introduction
Airflow introductionAirflow introduction
Airflow introduction
 
Yahoo! JAPANのコンテンツプラットフォームを支えるSpring Cloud Streamによるマイクロサービスアーキテクチャ #jsug #sf_52
Yahoo! JAPANのコンテンツプラットフォームを支えるSpring Cloud Streamによるマイクロサービスアーキテクチャ #jsug #sf_52Yahoo! JAPANのコンテンツプラットフォームを支えるSpring Cloud Streamによるマイクロサービスアーキテクチャ #jsug #sf_52
Yahoo! JAPANのコンテンツプラットフォームを支えるSpring Cloud Streamによるマイクロサービスアーキテクチャ #jsug #sf_52
 

Similar to IPFS introduction

Git.From thorns to the stars
Git.From thorns to the starsGit.From thorns to the stars
Git.From thorns to the stars
Strannik_2013
 
Development of the irods rados plugin @ iRODS User group meeting 2014
Development of the irods rados plugin @ iRODS User group meeting 2014Development of the irods rados plugin @ iRODS User group meeting 2014
Development of the irods rados plugin @ iRODS User group meeting 2014
mgrawinkel
 
Lessons learned while building Omroep.nl
Lessons learned while building Omroep.nlLessons learned while building Omroep.nl
Lessons learned while building Omroep.nl
bartzon
 

Similar to IPFS introduction (20)

Workshop: Big Data Visualization for Security
Workshop: Big Data Visualization for SecurityWorkshop: Big Data Visualization for Security
Workshop: Big Data Visualization for Security
 
OSDC 2016 - Unifying Logs and Metrics Data with Elastic Beats by Monica Sarbu
OSDC 2016 - Unifying Logs and Metrics Data with Elastic Beats by Monica SarbuOSDC 2016 - Unifying Logs and Metrics Data with Elastic Beats by Monica Sarbu
OSDC 2016 - Unifying Logs and Metrics Data with Elastic Beats by Monica Sarbu
 
Gitbase, SQL interface to Git repositories
Gitbase, SQL interface to Git repositoriesGitbase, SQL interface to Git repositories
Gitbase, SQL interface to Git repositories
 
Thrombus Training Dec. 2013
Thrombus Training Dec. 2013Thrombus Training Dec. 2013
Thrombus Training Dec. 2013
 
Git.From thorns to the stars
Git.From thorns to the starsGit.From thorns to the stars
Git.From thorns to the stars
 
Extracting Forensic Information From Zeus Derivatives
Extracting Forensic Information From Zeus DerivativesExtracting Forensic Information From Zeus Derivatives
Extracting Forensic Information From Zeus Derivatives
 
Git
GitGit
Git
 
Development of the irods rados plugin @ iRODS User group meeting 2014
Development of the irods rados plugin @ iRODS User group meeting 2014Development of the irods rados plugin @ iRODS User group meeting 2014
Development of the irods rados plugin @ iRODS User group meeting 2014
 
Introduction to FIWARE IoT
Introduction to FIWARE IoTIntroduction to FIWARE IoT
Introduction to FIWARE IoT
 
Six3 Getting Git
Six3 Getting GitSix3 Getting Git
Six3 Getting Git
 
Provisioning Heterogenous Bare Metal with Stacki
Provisioning Heterogenous Bare Metal with StackiProvisioning Heterogenous Bare Metal with Stacki
Provisioning Heterogenous Bare Metal with Stacki
 
Lessons learned while building Omroep.nl
Lessons learned while building Omroep.nlLessons learned while building Omroep.nl
Lessons learned while building Omroep.nl
 
Hydra - Getting Started
Hydra - Getting StartedHydra - Getting Started
Hydra - Getting Started
 
Lessons learned while building Omroep.nl
Lessons learned while building Omroep.nlLessons learned while building Omroep.nl
Lessons learned while building Omroep.nl
 
PostgreSQL 9.4: NoSQL on ACID
PostgreSQL 9.4: NoSQL on ACIDPostgreSQL 9.4: NoSQL on ACID
PostgreSQL 9.4: NoSQL on ACID
 
Burp suite
Burp suiteBurp suite
Burp suite
 
Managing your Black Friday Logs - Antonio Bonuccelli - Codemotion Rome 2018
Managing your Black Friday Logs - Antonio Bonuccelli - Codemotion Rome 2018Managing your Black Friday Logs - Antonio Bonuccelli - Codemotion Rome 2018
Managing your Black Friday Logs - Antonio Bonuccelli - Codemotion Rome 2018
 
ElasticSearch - DevNexus Atlanta - 2014
ElasticSearch - DevNexus Atlanta - 2014ElasticSearch - DevNexus Atlanta - 2014
ElasticSearch - DevNexus Atlanta - 2014
 
Full stack visibility with elastic, KubeCon 2017
Full stack visibility with elastic, KubeCon 2017Full stack visibility with elastic, KubeCon 2017
Full stack visibility with elastic, KubeCon 2017
 
The Basics of Open Source Collaboration With Git and GitHub
The Basics of Open Source Collaboration With Git and GitHubThe Basics of Open Source Collaboration With Git and GitHub
The Basics of Open Source Collaboration With Git and GitHub
 

Recently uploaded

Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
panagenda
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
WSO2
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Safe Software
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Victor Rentea
 

Recently uploaded (20)

DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 AmsterdamDEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
DEV meet-up UiPath Document Understanding May 7 2024 Amsterdam
 
Why Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire businessWhy Teams call analytics are critical to your entire business
Why Teams call analytics are critical to your entire business
 
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost SavingRepurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
Repurposing LNG terminals for Hydrogen Ammonia: Feasibility and Cost Saving
 
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ..."I see eyes in my soup": How Delivery Hero implemented the safety system for ...
"I see eyes in my soup": How Delivery Hero implemented the safety system for ...
 
AWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of TerraformAWS Community Day CPH - Three problems of Terraform
AWS Community Day CPH - Three problems of Terraform
 
Platformless Horizons for Digital Adaptability
Platformless Horizons for Digital AdaptabilityPlatformless Horizons for Digital Adaptability
Platformless Horizons for Digital Adaptability
 
Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..Understanding the FAA Part 107 License ..
Understanding the FAA Part 107 License ..
 
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data DiscoveryTrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
TrustArc Webinar - Unlock the Power of AI-Driven Data Discovery
 
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWEREMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
EMPOWERMENT TECHNOLOGY GRADE 11 QUARTER 2 REVIEWER
 
Boost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdfBoost Fertility New Invention Ups Success Rates.pdf
Boost Fertility New Invention Ups Success Rates.pdf
 
Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)Introduction to Multilingual Retrieval Augmented Generation (RAG)
Introduction to Multilingual Retrieval Augmented Generation (RAG)
 
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
Apidays New York 2024 - Accelerating FinTech Innovation by Vasa Krishnan, Fin...
 
MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024MINDCTI Revenue Release Quarter One 2024
MINDCTI Revenue Release Quarter One 2024
 
MS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectorsMS Copilot expands with MS Graph connectors
MS Copilot expands with MS Graph connectors
 
Architecting Cloud Native Applications
Architecting Cloud Native ApplicationsArchitecting Cloud Native Applications
Architecting Cloud Native Applications
 
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers:  A Deep Dive into Serverless Spatial Data and FMECloud Frontiers:  A Deep Dive into Serverless Spatial Data and FME
Cloud Frontiers: A Deep Dive into Serverless Spatial Data and FME
 
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
Modular Monolith - a Practical Alternative to Microservices @ Devoxx UK 2024
 
FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024FWD Group - Insurer Innovation Award 2024
FWD Group - Insurer Innovation Award 2024
 
WSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering DevelopersWSO2's API Vision: Unifying Control, Empowering Developers
WSO2's API Vision: Unifying Control, Empowering Developers
 
DBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor PresentationDBX First Quarter 2024 Investor Presentation
DBX First Quarter 2024 Investor Presentation
 

IPFS introduction

  • 1. Introduction to the IPFS 170629 SFC Arch B3 gentam
  • 5. • The InterPlanetary File System • : "Content Addressed, Versioned, P2P File System" • Protocol Labs < https://protocol.ai> Juan Benet 2013 12 • Go < https://github.com/ipfs/go-ipfs> 5
  • 6. • IPFS P2P • DHTs, BitTorrent, Git, SFS ... • IPFS 1 Merkle DAG 6
  • 7. • 1 • Web IPFS Git BitTorrent • IPFS • Permanent Web 7
  • 8. • HTTP Web • Web • Faster: P2P ? • Safer: • Open: 8
  • 9. • Blockchain ‣ IPFS ‣ ‣ Blockchain IPFS ‣ IPFS Blockchain 9
  • 10. – Juan Benet “The contribution of IPFS is simplifying, evolving, and connecting proven techniques into a single cohesive system, greater than the sum of its parts.” 10
  • 11. 2 IPFS HP (ipfs.io) 11
  • 13. • HTTP 1 1 • IPFS • P2P 60% ( • IPFS 13
  • 19. 19
  • 21. 21
  • 22. 22
  • 28. 28 IPFS IPFS • : "Content Addressed, Versioned, P2P File System" • Protocol Labs <https://protocol.ai> Juan Benet 2013 12 • Go <https://github.com/ipfs/go-ipfs>
  • 29. 29 IPFS IPFS • : "Content Addressed, Versioned, P2P File System" • Protocol Labs < https://protocol.ai> Juan Benet 2013 12 • Go <https://github.com/ipfs/go-ipfs> DHT, SFS Git (+Blockchain) SFS
  • 30. • Routing - DHT • Block Exchanges - BitTorrent • Version Control Systems - Git • Self-Certified Filesystems - SFS 30
  • 31. • Routing - DHT • Block Exchanges - BitTorrent • Version Control Systems - Git • Self-Certified Filesystems - SFS 31
  • 33. Kademlia • ID XOR • log(n) lookup (e.g. 10,000,000 20 ) • • DoS Kademlia DHT Erlang 33
  • 34. S/Kademlia DHT • Kademlia • PKI PoW Sybil • 0.85 34
  • 35. Coral DSHT • Distributed Sloppy Hash Table • Kademlia ID " " • → " " ( ) • " " 35
  • 36. • Routing - DHT • Block Exchanges - BitTorrent • Version Control Systems - Git • Self-Certified Filesystems - SFS 36
  • 37. • BitTorrent P2P • Bram Cohen 2001 • " *" * 37
  • 38. 1. ( ) " " 2. 3. BitTorrent "tit-for-tat" ( =1 ) PropShare 38
  • 39. • Routing - DHT • Block Exchanges - BitTorrent • Version Control Systems - Git • Self-Certified Filesystems - SFS 39
  • 40. • • Git Merkle DAG* * Merkle Directed Acyclic Graph ( ?) – 40
  • 41. 1. (blob), (tree), (commit) 2. 3. Merkle DAG 4. ( etc) 5. 6. 41
  • 43. • Routing - DHT • Block Exchanges - BitTorrent • Version Control Systems - Git • Self-Certified Filesystems - SFS 43
  • 44. SFS • SFS /sfs/<Location>:<HostID> Location = HostID = hash(public_key || Location) 44
  • 47. • IPFS Identities, Network, Routing, Exchange, Objects, Files, Naming 47
  • 48. 1. Iden99es - ID 2. Network - 3. Rou9ng - 4. Exchange - (BitSwap) 5. Objects - Merkle DAG 6. Files - Git 7. Naming - self-certifying 48
  • 49. 1. Identities 2. Network 3. Routing 4. Exchange 5. Objects 6. Files 7. Naming 49
  • 50. • NodeID • S/Kademlia • NodeID ID ( ID type NodeId Multihash type Multihash []byte // self-describing cryptographic hash digest type PublicKey []byte type PrivateKey []byte // self-describing keys type Node struct { NodeId NodeID PubKey PublicKey PriKey PrivateKey } 50
  • 51. • S/Kademlia IPFS ID : difficulty = <integer parameter> n = Node{} do { n.PubKey, n.PrivKey = PKI.genKeyPair() n.NodeId = hash(n.PubKey) p = count_preceding_zero_bits(hash(n.NodeId)) } while (p < difficulty) • hash(other.PublicKey) other.NodeID 51
  • 52. • IPFS • multihash : <function code><digest length><digest bytes> • 52
  • 53. 1. Identities 2. Network 3. Routing 4. Exchange 5. Objects 6. Files 7. Naming 53
  • 54. • IPFS • • Transport, Reliability, Connectivity, Integrity, Authenticity 54
  • 55. • Transport: IPFS WebRTC DataChannl ( ) uTP (LEDBAT) • Reliability: IPFS uTP SCTP 55
  • 56. • Connectivity: ICE (Interactive Connectivity Establishment) NAT • Integrity: ( ) • Authenticity: HMAC ( ) 56
  • 57. • multihash multiaddr • IP # an SCTP/IPv4 connection /ip4/10.20.30.40/sctp/1234/ # an SCTP/IPv4 connection proxied over TCP/IPv4 /ip4/5.6.7.8/tcp/5678/ip4/1.2.3.4/sctp/1234/ 57
  • 58. 1. Identities 2. Network 3. Routing 4. Exchange 5. Objects 6. Files 7. Naming 58
  • 59. • IPFS : • • • IPFS S/Kademlia Coral DSHT • 1KB DHT ( NodeID) 59
  • 60. type IPFSRouting interface { FindPeer(node NodeId) // gets a particular peer's network address SetValue(key []bytes, value []bytes) // stores a small metadata value in DHT GetValue(key []bytes) // retrieves small metadata value from DHT ProvideValue(key Multihash) // announces this node can serve a large value FindValuePeers(key Multihash, min int) // gets a number of peers serving a large value } • 60
  • 61. 1. Identities 2. Network 3. Routing 4. Exchange 5. Objects 6. Files 7. Naming 61
  • 62. • BitSwap: BitTorrent • have_list • want_list • BitTorrent 1 ‣ ( ) 62
  • 63. • BitSwap : • Seeder ( ) • Leecher ( ) • " " • ignore_cooldown (10 sec) 63
  • 64. • BitTorrent "tit-for-tat" : • BitTyrant: Selfish Peer • BitThief: • PropShare: ‣ 64
  • 65. • BitSwap debt ratio: r ( ) • r : 65 P(send | r) = 1 1 + e6 3r r = bytes send bytes recv + 1
  • 66. • ledger • ledger type Ledger struct { owner NodeId partner NodeId bytes_sent int bytes_recv int timestamp Timestamp } • ledger 66
  • 67. 1. Open: ledgers 2. Sending: want_lists 3. Close: 4. Ignored( ): 67
  • 68. 1. Identities 2. Network 3. Routing 4. Exchange 5. Objects 6. Files 7. Naming 68
  • 69. • DHT BitSwap P2P • IPFS Git Merkle DAG 69
  • 70. 1. Content Addressing: multihash 2. Tamper resistance: 3. Deduplica9on: 70
  • 71. IPFS : type IPFSLink struct { Name string // name or alias of this link Hash Multihash // cryptographic hash of target Size int // total size of target } type IPFSObject struct { links []IPFSLink // array of links data []byte // opaque content data } 71
  • 72. • ipfs (brew install ipfs) • : > ipfs ls /XLZ1625Jjn7SubMDgEyeaynFuR84ginqvzb XLYkgq61DYaQ8NhkcqyU7rLcnSa7dSHQ16x 189458 less XLHBNmRQ5sJJrdMPuu48pzeyTtRo39tNDR5 19441 script XLF4hwVHsVuZ78FZK6fozf8Jj9WEURMbCX4 5286 template <object multihash> <object size> <link name> • 72
  • 73. • UNIX Web # format /ipfs/<hash-of-object>/<name-path-to-object> # example /ipfs/XLYkgq61DYaQ8NhkcqyU7rLcnSa7dSHQ16x/foo.txt 73
  • 74. • /ipfs • " " /ipfs/ ‣ : /ipfs/<hash-of-foo>/bar/baz /ipfs/<hash-of-bar>/baz /ipfs/<hash-of-baz> 74
  • 75. • ( ) ‣ ( • " " • DHT ‣ 75
  • 76. • IPFS • type EncryptedObject struct { Object []bytes // raw object data encrypted Tag []bytes // optional tag for encryption groups } type SignedObject struct { Object []bytes // raw object data signed Signature []bytes // hmac signature PublicKey []multihash // multihash identifying key } 76
  • 77. 1. Identities 2. Network 3. Routing 4. Exchange 5. Objects 6. Files 7. Naming 77
  • 78. • IPFS : 1. block: a variable-size block of data. 2. list: a collection of blocks or other lists. 3. tree: a collection of blocks, lists, or other trees. 4. commit: a snapshot in the version history of a tree. 78
  • 79. • Git : 1. (→objects ) 2. (→ list ) 3. commits tree • IPFS Git • JSON protobuf 79
  • 80. • blob • IPFS blob Git { "data": "some data here", // blobs have no links } 80
  • 81. • list blob { "data": ["blob", "list", "blob"], // lists have an array of object types as data "links": [ { "hash": "XLYkgq61DYaQ8NhkcqyU7rLcnSa7dSHQ16x", "size": 189458 }, { "hash": "XLHBNmRQ5sJJrdMPuu48pzeyTtRo39tNDR5", "size": 19441 }, { "hash": "XLWVQDqxo9Km9zLyquoC9gAP8CL1gWnHZ7z", "size": 5286 } // lists have no names in links ] } 81
  • 82. • tree : Git { "data": ["blob", "list", "blob"], // trees have an array of object types as data "links": [ { "hash": "XLYkgq61DYaQ8NhkcqyU7rLcnSa7dSHQ16x", "name": "less", "size": 189458 }, { "hash": "XLHBNmRQ5sJJrdMPuu48pzeyTtRo39tNDR5", "name": "script", "size": 19441 }, { "hash": "XLWVQDqxo9Km9zLyquoC9gAP8CL1gWnHZ7z", "name": "template", "size": 5286 } // trees do have names ] } 82
  • 83. • commit > ipfs file-cat <ccc111-hash> --json { "data": { "type": "tree", "date": "2014-09-20 12:44:06Z", "message": "This is a commit message." }, "links": [ { "hash": "<ccc000-hash>", "name": "parent", "size": 25309 }, { "hash": "<ttt111-hash>", "name": "object", "size": 5198 }, { "hash": "<aaa111-hash>", "name": "author", "size": 109 } ] } 83
  • 84. > ipfs file-cat <ttt111-hash> --json { "data": ["tree", "tree", "blob"], "links": [ { "hash": "<ttt222-hash>", "name": "ttt222-name", "size": 1234 }, { "hash": "<ttt333-hash>", "name": "ttt333-name", "size": 3456 }, { "hash": "<bbb222-hash>", "name": "bbb222-name", "size": 22 } ] } 84
  • 85. 1. Identities 2. Network 3. Routing 4. Exchange 5. Objects 6. Files 7. Naming 85
  • 86. • • IPFS • : ‣ (= ID) ‣ ‣ (= ID) ‣ ... 86
  • 88. • IPFS 1. NodeId = hash(node.PubKey) 2. NodeId " " : /ipns/<NodeId> 3. : /ipns/XLF2ipQ4jD3UdeX5xp1KBgeHRhemUtaA8Vm/docs/hello 4. NodeId 88
  • 89. • IPFS 1. NodeId = hash(node.PubKey) 2. NodeId " " : /ipns/<NodeId> 3. : /ipns/XLF2ipQ4jD3UdeX5xp1KBgeHRhemUtaA8Vm/docs/hello 4. NodeId 88 f n
  • 90. 89 IPFS → 5. Object • UNIX Web # format /ipfs/<hash-of-object>/<name-path-to-object> # example /ipfs/XLYkgq61DYaQ8NhkcqyU7rLcnSa7dSHQ16x/foo.txt
  • 91. 89 IPFS → 5. Object • UNIX Web # format /ipfs/<hash-of-object>/<name-path-to-object> # example /ipfs/XLYkgq61DYaQ8NhkcqyU7rLcnSa7dSHQ16x/foo.txt ipfs " "
  • 92. • "ipns" "ipfs" • IPNS : 1. IPFS 2. routing.setValue(NodeId, <ns-object-hash>) 90
  • 93. • IPNS 1. Peer Links 2. DNS TXT IPNS Records 3. Proquint Pronounceable Identifiers 4. Name Shortening Services 91
  • 94. • • web of trust # Alice links to bob Bob ipfs link /<alice-pk-hash>/friends/bob /<bob-pk-hash> # Eve links to Alice ipfs link /<eve-pk-hash/friends/alice /<alice-pk-hash> # Eve also has access to Bob /<eve-pk-hash/friends/alice/friends/bob # access Verisign certified domains /<verisign-pk-hash>/foo.com 92
  • 95. • /ipns/<domain> IPFS DNS TXT : • IPFS IPNS : # this DNS TXT record ipfs.benet.ai. TXT "ipfs=XLF2ipQ4jD3U ..." # behaves as symlink ln -s /ipns/XLF2ipQ4jD3U /ipns/fs.benet.ai 93
  • 96. • IPFS Proquint : # this proquint phrase /ipns/dahih-dolij-sozuk-vosah-luvar-fuluh # will resolve to corresponding /ipns/KhAwNprxYVxKqpDZ 94
  • 97. • • DNS Web URL : # User can get a link from /ipns/shorten.er/foobar # To her own namespace /ipns/XLF2ipQ4jD3UdeX5xp1KBgeHRhemUtaA8Vm 95
  • 99. 1. As a mounted global filesystem, under /ipfs and /ipns. 2. As a mounted personal sync folder that automatically versions, publishes, and backs up any writes. 3. As an encrypted file or data sharing system. 4. As a versioned package manager for all software. 5. As the root filesystem of a Virtual Machine. 6. As the boot filesystem of a VM (under a hypervisor). 97 7. As a database: applications can write directly to the Merkle DAG data model and get all the versioning, caching, and distribution IPFS provides. 8. As a linked (and encrypted) communications platform. 9. As an integrity checked CDN for large files (without SSL). 10.As an encrypted CDN. 11.On webpages, as a web CDN. 12.As a new Permanent Web where links do not die.
  • 100. • <https://github.com/ipfs/awesome-ipfs> • IPFSBin - Pastebin IPFS • Interplanetary Wiki - IPFS Wiki • ... 98
  • 101. • Storj <https://storj.io/> • Gitchain <http://gitchain.org/> • ZeroTier One <https://www.zerotier.com/> • MaidSafe <https://maidsafe.net> • SIA < http://sia.tech/> • Upspin <https://upspin.io> 99
  • 102. • IPFS HP <https://ipfs.io/> • White Paper <https://github.com/ipfs/papers/raw/master/ipfs- cap2pfs/ipfs-p2p-file-system.pdf> • the morning paper<https://blog.acolyer.org/2015/10/05/ipfs-content- addressed-versioned-p2p-file-system/> 100