Blockchain Technologies for Data Science
•
0 j'aime•970 vues
Bitcoin has brought about a true revolution in how we think about money. In one fell stroke it solved the main problems that afflicted previous attempts at a truly digital currency: distributed consensus, double spending, and external attacks. Perhaps more importantly, it provided the first working version of a blockchain or distributed ledger. However, despite their relative simplicity, the underlying concepts on which these technologies are built are not well known and often obscured by hype and technical jargon. Since the days of Bitcoin’s founding, many other crypto-currencies have been proposed and released. This tutorial will introduce these technologies in an intuitive way for data scientists, explaining their driving algorithms, motivations, and data structures.
Recommandé
Contenu connexe
Tendances
Tendances (20)
Similaire à Blockchain Technologies for Data Science
Similaire à Blockchain Technologies for Data Science (20)
Plus de Bruno Gonçalves
Plus de Bruno Gonçalves (12)
Dernier
Dernier (20)
Blockchain Technologies for Data Science
- 1. Bruno Gonçalves www.data4sci.com Blockchain Technologies for Data Scientists www.data4sci.com https://github.com/DataForScience/blockchain-data
- 2. www.data4sci.com@bgoncalves The views and opinions expressed in this article are those of the authors and do not necessarily reflect the official policy or position of my employers. The examples provided with this tutorial were chosen for their didactic value and are not mean to be representative of my day to day work. Disclaimer
- 4. www.data4sci.com@bgoncalves How Money Works • Many different types of currency have been used throughout the years • Gold • Cattle • Salt • Rocks • Sea Shells • etc… • Fundamental functions (and properties) of money: • Medium of Exchange (Fungibility, Portability) • Unit of Account (Cognizability) • Store of Value (Durability, Stability of Value / Scarcity) https://www.continentalcurrency.ca/wp-content/uploads/2015/06/continental-currency-exchange.jpg https://upload.wikimedia.org/wikipedia/commons/2/2b/Olaus_Magnus_-_On_Trade_Without_Using_Money.jpg
- 6. www.data4sci.com@bgoncalves The Ingredients of a Crypto-Currency • Fundamental functions (and properties) of money: • Medium of Exchange (Fungibility, Portability) • Unit of Account (Cognizability) • Store of Value (Durability, Stability of Value / Scarcity) • Digital currencies have obvious advantages: • Fungibility - A bit is a bit • Portability - We can easily cary terabytes in our pockets or digitally transfer them across the world • Cognizability - It’s just a number • Durability - Information can easily be stored for decades or even centuries • And obvious problems: • Stability of Value / Scarcity • There is no limit to how many you can create • They can be easily duplicated (no trust) • etc… https://pixabay.com/photos/bank-money-finance-shares-save-2907728/
- 7. www.data4sci.com@bgoncalves The Ingredients of a Crypto-Currency • How to avoid the need for a centralized authority? • Distributed Ledger • How to avoid double spending? • Consensus algorithm • How to prevent cheaters? • Reward miners for being honest https://pixabay.com/photos/bank-money-finance-shares-save-2907728/ https://news.bitcoin.com/10-years-ago-bitcoins-genesis-block-changed-the-course-of-history/
- 8. www.data4sci.com@bgoncalves The road to Bitcoin https://queue.acm.org/detail.cfm?id=3136559
- 9. www.data4sci.com@bgoncalves How does it work? • According to “Satoshi Nakamoto”: • Essentially: • Transactions are processed in blocks • Miners verify the transactions in each block [ Mining Reward ] • Blocks are cryptographically signed to prevent tampering [ Proof of Work ] • Each block points to the previous block to form a chain • The longest chain is defined as the correct one [ Consensus ] https://bitcoin.org/bitcoin.pdf
- 10. www.data4sci.com@bgoncalves Linked List Header 0: Null Pointer DATA Header 1: Pointer to 0 DATA Header 2: Pointer to 1 DATA Header N: Pointer to N-1 DATA⋯
- 11. www.data4sci.com@bgoncalves Blockchain Header 0: Null Hash Header 1: Hash of 0 Header 2: Hash of 1 Header N: Hash of N-1 DATA DATA DATA DATA⋯ • The hashing function ensures that any changes to a validated previous block invalidates it.
- 12. www.data4sci.com@bgoncalves Blockchain Header 0: Null Hash Header 1: Hash of Header 0 Header 2: Hash of Header 1 Header N: Hash of Header N-1 ⋯ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ • The hashing function ensures that any changes to a validated previous block invalidates it. • Keep hash only the previous header for efficiency
- 13. www.data4sci.com@bgoncalves Blockchain Header 0: Null Hash Txs Hash Header 1: Hash of Header 0 Txs Hash Header 2: Hash of Header 1 Txs Hash Header N: Hash of Header N-1 Txs Hash ⋯ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ • The hashing function ensures that any changes to a validated previous block invalidates it. • Keep hash only the previous header for efficiency • Header must include hash of data
- 14. www.data4sci.com@bgoncalves Proof Of Work • Each block is identified by a hash value • To control the rate at which blocks can be verified, and avoid malicious attacks the hash value of the block must obey certain conditions • Miners add values to the data in the header (a nonce) until the hash fulfills these conditions • The cryptographic properties of the hashing function guarantee that each value of the nonce is a roll of the dice (fairness) • The “difficulty” level of the chain establishes how hard it is to obtain a valid hash • Any change to the block data invalidates the hash • The difficulty is adjusted every two weeks to account for the hashing capability of the network
- 16. www.data4sci.com@bgoncalves Blockchain Header 0: Null Hash Difficulty 0 Nonce 0 Txs Hash Header 1: Hash of Header 0 Difficulty 1 Nonce 1 Txs Hash Header 2: Hash of Header 1 Difficulty 2 Nonce 2 Txs Hash Header N: Hash of Header N-1 Difficulty 3 Nonce N Txs Hash ⋯ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ • The hashing function ensures that any changes to a validated previous block invalidates it. • Keep hash only the previous header for efficiency • Header must include hash of data
- 17. www.data4sci.com@bgoncalves Blockchain Header 0: Null Hash Difficulty 0 Nonce 0 Merkle Root Header 1: Hash of Header 0 Difficulty 1 Nonce 1 Merkle Root Header 2: Hash of Header 1 Difficulty 2 Nonce 2 Merkle Root Header N: Hash of Header N-1 Difficulty N Nonce N Merkle Root ⋯ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ Tx 0 Tx 1 Tx 2 Tx 3 Tx N ⋮ • The hashing function ensures that any changes to a validated previous block invalidates it. • Keep hash only the previous header for efficiency • Header must include hash of data
- 18. www.data4sci.com@bgoncalves Hash 44 T4 + T4 Hash 4 Tx 4 Merkle Tree Hash 0 Tx 0 Hash 1 Tx 1 Hash 3 Tx 3 Hash 2 Tx 2 Hash 4 Tx 4 Hash 01 T0 + T1 Hash 23 T2 + T3 Hash 0123 Hash 01 + Hash 23 Hash 4444 Hash 44 + Hash 44 Hash 44 T4 + T4 Merkle Root Hash 01234444 Hash 0123 + Hash 4444
- 19. @bgoncalves Merkle Tree def get_root(transactions): missing = len(transactions) next_transactions = [] while missing > 1: print(missing) next_transactions = [] # If it's odd, repeat the last transaction if missing % 2 == 1: transactions.append(transactions[-1]) missing += 1 for i in range(0, missing, 2): input = transactions[i] + transactions[i+1] output = doubleSha256_new(input) next_transactions.append(output) print(i, i+1, next_transactions[-1]) missing = len(next_transactions) transactions = next_transactions[::] return invert_byteorder(next_transactions[0]) merkle_proof.py
- 20. www.data4sci.com@bgoncalves Merkle Proof - Tx 2 Hash 0 Tx 0 Hash 1 Tx 1 Hash 3 Tx 3 Hash 2 Tx 2 Hash 4 Tx 4 Hash 4 Tx 4 Hash 01 T0 + T1 Hash 23 T2 + T3 Hash 0123 Hash 01 + Hash 23 Hash 4444 Hash 44 + Hash 44 Hash 44 T4 + T4 Hash 44 T4 + T4 Merkle Root Hash 01234444 Hash 0123 + Hash 4444
- 21. www.data4sci.com@bgoncalves Merkle Proof - Tx 2 Hash 3 Tx 3 Hash 2 Tx 2 Hash 01 T0 + T1 Hash 23 T2 + T3 Hash 0123 Hash 01 + Hash 23 Hash 4444 Hash 44 + Hash 44 Merkle Root Hash 01234444 Hash 0123 + Hash 4444 • In general, with N transactions you need only hashes to verify if a transaction is part of the tree. • Building the tree, requires computing….. total hashes log2 (N)
- 22. @bgoncalves Merkle Proof def get_root(transactions): missing = len(transactions) next_transactions = [] while missing > 1: print(missing) next_transactions = [] # If it's odd, repeat the last transaction if missing % 2 == 1: transactions.append(transactions[-1]) missing += 1 for i in range(0, missing, 2): input = transactions[i] + transactions[i+1] output = doubleSha256_new(input) next_transactions.append(output) print(i, i+1, next_transactions[-1]) missing = len(next_transactions) transactions = next_transactions[::] return invert_byteorder(next_transactions[0]) merkle_proof.py
- 24. www.data4sci.com@bgoncalves Transactions Transaction HUF inputs outputs Transaction TYU inputs Tx AFD 3 Tx HUF 1 outputs Addr asd: 2.76 Addr otu: 5.50 Transaction AFD inputs outputs 1 03 2 1 0 01 5.15 B 3.14 B • All input values must be spent • Each input is a complete output • 0 or more inputs • 1 or more outputs • Inputs point to previous transaction outputs • Outputs are addresses Fee = ∑ in Vin − ∑ o u t Vo u t ≥ 0 • Difference between inputs and outputs is paid as a fee to the block validator Block Reward
- 25. www.data4sci.com@bgoncalves Transactions https://en.bitcoin.it/wiki/Transaction • All input values must be spent • Each input is a complete output • 0 or more inputs • 1 or more outputs • Inputs point to previous transaction outputs • Outputs are addresses Transaction inputs outputs • First transaction in a block is the block reward and goes to the miners (coinbase transaction) • coinbase transactions include all fees for the transactions included in the block • Base block reward is how new Bitcoins are produced
- 27. www.data4sci.com@bgoncalves Bitcoin transactions around the world http://www.vo.elte.hu/papers/2017/bitcoin/
- 32. www.data4sci.com@bgoncalves How much is it worth?
- 33. www.data4sci.com@bgoncalves Bitcoin interest over time 0 25 50 75 100 2009-022009-072009-122010-052010-102011-032011-082012-012012-062012-112013-042013-092014-022014-072014-122015-052015-102016-032016-082017-012017-062017-112018-042018-092019-02 blockchain bitcoin
- 35. Questions?