My presentation on 'Blockchain Scalability - Architectures and Algorithms' for the TechAthena Digital Community Webinar.
Blockchain Scalability is one of the most significant concern for Minimum Viable Blockchain Implementation. It is one of the key aspects determining the relevance and feasibility of Blockchain Technology for a particular use case.This session will cover the fundamental aspects of distributed computing that determine the contours of scalability.
Subsequently, the session will outline the parameters and metrics related to Blockchain Scalability in detail. In this context, the session will deep dive into architectural and algorithmic techniques that enables a scalable Blockchain.Architectural techniques such as vertical scaling and horizontal scaling will be explained in detail. Design techniques such as State Channels, Sharding, SideChains, Off chain computations, Block Size and Time Optimization etc. will be explained.
In summary, this session will conclude with the implications and trade-off between Blockchain Scalability, Security, Simplicity and Interoperability. Looking forward to your views and thoughts !
Blockchain Scalability Algorithms and Architectures
1. Gokul Alex, Associate Director, PwC India
Blockchain Scalability Algorithms and Architectures
2. What is the scalability challenge in Blockchain ?
3. –Vitalik Buterin
“Currently, all blockchain consensus protocols that are actively in use
have an important limitation: every fully participating node in the
network must process every transaction. ”
4. Interpretations of the Scalability Challenge
• This gives the blockchain a high
amount of security because of how
much validation goes into each
block
• At the same time it means that an
entire blockchain is only as fast as
its individual nodes and not the
sum of their parts.
5. Blockchain Scalability - Architectures and Algorithms
Blockchain
Trilemma
A Blockchain can have at most two of
these three properties :
• Decentralisation
• Scalability
• Security
10. Blockchain Scalability - Architectures and Algorithms
Understanding SpeedUp,
Efficiency and Scalability in Totality
• Speedup measures how the rate of doing work
increases with the number of processors
• Efficiency E measures the work rate per
processor
• Scalability ψ(k1
,k2
) from one scale k1
to another
scale k2
is the ratio of the efficiency figures
11. Blockchain Scalability - Architectures and Algorithms
Understanding Scalability The number of transactions per unity time
that the system can process
16. Blockchain Scalability - Architectures and Algorithms
Decentralised Scaling :
Cost and Constraints
It should be noted that scaling does not imply
a decrease in the latency of transactions.
Indeed, improvements to scaling may
sometimes come at the cost of increased
latency.
17. Blockchain Scalability - Architectures and Algorithms
Scalability and Consensus :
Myths and Misconceptions
It should also be noted that, on their own,
consensus mechanisms do not affect
transaction throughput in any meaningful way.
A common misconception is that “Proof-of-
Work (PoW) is slow” — that could not be
further from the truth.
18. Blockchain Scalability - Architectures and Algorithms
Strategic approach to
Scalability across Consensus
Systems that achieve greater scalability with different
consensus mechanisms do so almost universally by
reducing the size of the set of block producers.
If the root cause of the scalability problem is that every
transaction must be validated by every node, one way
forward is to simply reduce the number of nodes on the
network
19. Blockchain Scalability - Architectures and Algorithms
Understanding
Decentralisation
A system is decentralised if and only if it is:
• Distributed
• Trustless
• Permissionless
20. Blockchain Scalability - Architectures and Algorithms
Blockchain Scalability
Factors and Constraints
Size of transactions
Size of a block
How many transactions in a block
How often blocks get added to the chain
How nodes collaborate in the chain
How nodes add transactions to the chain
21. Blockchain Scalability - Architectures and Algorithms
Blockchain Scale in
comparison with Visa
Bitcoin - 90,000 Transactions / Day
Visa - 150 Million Transactions / Day
22. Blockchain Scalability - Architectures and Algorithms
Common approaches for
Blockchain Scalability
Off chain computations
Side Chains
State Channels
Sharding Protocols
New Consensus Protocols
Reducing the Block size
23. Blockchain Scalability - Architectures and Algorithms
Layer 1 - Layer 2
Scaling approaches
The class of proposals to properly scale decentralised
blockchains currently revolve around not having each node
in the network validate every transaction.
This is accomplished by either sharding the base
blockchain (so-called layer-1 scaling) or having separate
chains running alongside the base chain that only a subset
of users need to fully validate (so-called layer-2 scaling).
24. Blockchain Scalability - Architectures and Algorithms
Scalability Measures
• Maximum throughput - maximum rate at which the
blockchain can confirm transactions
• Latency - time for a transaction to be confirmed. The
transactions are confirmed once they are included in a
block, which is roughly 10 minutes for bitcoin.
• Bootstrap time - The time it takes for a new computer
node to download the history necessary to validate a
new transaction
25. Blockchain Scalability - Architectures and Algorithms
Scalability - Hard Truths
In all blockchain protocols, each node stores all states
with account balances, contract code, and entire
transaction history, in order to verify a transaction’s
validity for each transaction in the network without a
trusted third party.
This provides a large amount of security but limits
scalability to not being able to process more
transactions than any single node in the network can.
26. Blockchain Scalability - Architectures and Algorithms
Scalability Dimensions
Node Identity Management
Consensus Finality
Node Scalability
Client Scalability
Performance throughput
Performance latency
Power consumption
Tolerated power of an adversary
Network synchrony
Correctness proofs
27. Blockchain Scalability - Architectures and Algorithms
Node identity management
How nodes are identified in a consensus protocol in a
network through a unique identifier
In a proof of work based network, anyone can participate
and contribute to the system
In a BFT consensus, every participating node need to
identify to participate in the transactions
This results in a centralised identity management
requirement in a BFT protocol implementation
28. Blockchain Scalability - Architectures and Algorithms
Consensus Finality
• Consensus Finality is the property that ensures that a
valid block added to the blockchain is not removed
• PoW violates this property when two nodes append a
block to the same block which result in a fork
• In the case of Bitcoin the forks are handled by the
longest chain rule, thus breaking the consensus finality
by removing the shorter chain or the GHOST rule
• BFT satisfies consensus finality with immediate
confirmation
29. Blockchain Scalability - Architectures and Algorithms
Client Scalability
When it comes to scalability with the
number of clients, both PoW and BFT
support thousands of clients all
connected at once with good
concurrency and parallelism
30. Blockchain Scalability - Architectures and Algorithms
Node Scalability
Scalability in Proof of Work
Scalability in BFT
Scalability in DAG
Scalability in PoS
Scalability in DPoS
Scalability in DPoW
31. Blockchain Scalability - Architectures and Algorithms
Performance Issues in PoW
PoW scalability is reliant on the block size and the rate of
the block creations
If the block size is increased, potential trees are created in
the blockchain leading to double spend attacks
32. Blockchain Scalability - Architectures and Algorithms
Early Proposals for
Bitcoin Scalability
Removing old transactions from Blockchain and a database is used
to hold the non-empty address trees
In this way, nodes that are validating the transactions do not have to
store the previous transactions that are not relevant to them
Bitcoin Next Generation is another proposal where the blocks are
decoupled and there are leader blocks and micro blocks that handle
transactions
Miners would complete for the leader block, these would be the ones
in charge of generation of new micro blocks
33. Blockchain Scalability - Architectures and Algorithms
Double Spend attacks on
Blockchain Ledger
Double spend attacks is a method to override the
main chain to reverse transactions
The attacker pays a person and in secrecy builds a
chain of blocks where the payment is not included.
By releasing the chain the attacker can cause a
replacement in the ledger where the payments are
erased or redirects the payment to somewhere else
34. Blockchain Scalability - Architectures and Algorithms
Double Spend attack on
Bitcoin Network
This requires a lot of computational power which makes the attack
unlikely since the honest nodes have a lot of computational power,
however there was a case of a mining pool in Bitcoin having over
40% of the total computational power.
If the attacker has a lot of computational power there is a possibility
that the attacker can generate blocks that could replace the honest
longest chain and that enables the attacker to replace the main chain
at will.
When the attacker has more computational power than the honest
nodes, it is called the 51% attack (also known as the majority attack)
35. Blockchain Scalability - Architectures and Algorithms
Double Spend attack and
Bitcoin Scalability aspects
The Bitcoin protocol becomes more susceptible to
double-spend attacks when it scales upwards
and tries to meet the demand.
If we assume that the attacker creates blocks at a
rate that is faster than the rate of the honest main
chain, the attacker will always be successful with
these types of attacks regardless of the length of
the chain it aims to replace .
36. Blockchain Scalability - Architectures and Algorithms
State Channels approach to
Blockchain Scalability
State channels are the general form
of payment channels, applying the
same idea to any kind of state-
altering operation normally
performed on a blockchain.
38. Blockchain Scalability - Architectures and Algorithms
Prominent State Channel
Implementations
• Ligtning Network
• Raiden Network
• Trinity
• Spankchain
• Perun
• Counterfactual
• Celer Network
• Machinomy
• Fun Fair
• Liquidity
• Connext Network
39. Blockchain Scalability - Architectures and Algorithms
Bitcoin Scalability and
Lightning Network
The Lightning Network is supported by numerous
smart contracts put together in a system built on
the topmost tier of the Bitcoin Blockchain.
The protocol allows very fast transactions speeds
that are accompanied by very low transitioning
fees.
40. Blockchain Scalability - Architectures and Algorithms
Building Blocks of
Lightning Network
Unconfirmed Transactions
Double Spend Protection
Multi signature Addresses
Time Locks
Hash values and Secrets
41. Lightning Network Lifecycle
• Set up a wallet with a multi-signature feature with some amount in BTC
• Upload the Wallet’s address into the public Bitcoin Blockchain.
• This is accompanied by a smart contract that clearly states what amount
of BTC belongs to whom.
• Once the pay channel is instantiated, it opens up an avenue for the
parties therein to undertake unlimited transactions amongst themselves.
• The information in the wallet set is not updated onto the main
Blockchain. The transactions occur off-chain.
• Upon completion of every transaction, a balance is signed up by both
parties, and this is reflected on the balance sheet.
• At any given time, the multi-signature wallet will show the balances
owed to each party.
• In case of a dispute or should the payment channel be locked, the
contractual obligations terminate there and the involved parties pay each
other as per the balances reflected as a share in the Multi-signature
wallet.
43. Blockchain Scalability - Architectures and Algorithms
Unconfirmed Transactions
The Lightning Network is built up from more
or less regular Bitcoin transactions.
These transactions are typically not actually
broadcast over the Bitcoin network.
Instead, they are stored locally, on the nodes
of users - but they can be broadcast over the
network at any time.
44. Blockchain Scalability - Architectures and Algorithms
Double Spend Protection
If two transactions (or: inputs) rely
on the same output, only one can
confirm.
Even unconfirmed transactions can
be conflicting, meaning only one can
ever confirm.
45. Blockchain Scalability - Architectures and Algorithms
MultiSig ( P2SH) Addresses
Multisig addresses are Bitcoin addresses that
require multiple private keys to “unlock” and
spend bitcoins from.
The Lightning Network often uses two out of
two (2-of-2) multisig set-ups. Unlocking
bitcoins from 2-of-2 multisig addresses requires
two signatures, from two dedicated keys.
46. Blockchain Scalability - Architectures and Algorithms
Time Locks
Time-locks can “lock bitcoins up” in an output, to make them spendable (to
be included in a subsequent input) only at some point in the future.
There are two different types of time-locks: the absolute type, called
CheckLockTimeVerify (CLTV), and the relative type, CheckSequenceVerify
(CSV).
CLTV locks bitcoins up until a (more or less) concrete time in the future: an
actual time and date, or a specific block height. CSV, instead, uses relative
time.
Once a CVS-output is recorded on the blockchain, it takes a specific amount
of blocks from that point on before the bitcoins can be spent again.
47. Blockchain Scalability - Architectures and Algorithms
Hash Values and Secrets
In a bitcoin transactions, a hash can
be included in an output, and require
the subsequent input to include the
corresponding value in order to be
spendable.
50. Blockchain Scalability - Architectures and Algorithms
Sidechain approach to
Blockchain Scalability
A sidechain is a separate blockchain that is
attached to its parent blockchain using a
two-way peg.
In other words, you can move assets to the
sidechain and then back to the parent chain.
53. Blockchain Scalability - Architectures and Algorithms
Ethereum Plasma approach
to Blockchain Scalability
Plasma is a series of contracts that run
on top of a root chain (Ethereum main
chain) and consists of a network of
“child chains” connected to a root chain
in a hierarchical, tree-like structure.
54. Ethereum Plasma Lifecycle
• Initially, smart-contracts are created on the Ethereum
main-chain. These smart contracts serve as the “root” of
the Plasma child-chain.
• This main chain entry contains the basic rules of the child-
chain, records state hashes of the child-chain, and allows
users to move assets between the Ethereum main-chain
and the child-chain.
• After rooting the child-chain in the main chain, a child-
chain is created. This child-chain features its own
consensus algorithm, independently from the Ethereum
main-chain.
• Once the child-chain is up and running, the block creators
periodically commit a validation to the main-chain,
essentially proofing that the current state of the child-
chain is valid according to the consensus rules.
55. Plasma Consensus Protocol
• The consensus mechanism for this proof of stake system, is
again, enforced in an on- blockchain smart contract.
• Instead of enforcement of an incrementing nonce state (via
revocations), a system of fraud proofs is enforced for balances
and state transitions of these chain hierarchies.
• In effect, we are able to create state transitions which are only
periodically committed to parent chains (which then flows to the
root blockchain).
• Constructs computation in a MapReduce format to more easily
design computation and state transitions in a hierarchical tree.
• This creates economically enforceable computation at scale, with
only one block header/hash committed on the root chain to
encompass very high amount of data and work.
• It is only if a block is faulty that proof of invalidity is published,
otherwise extremely minimal amounts of data is submitted on
the root chain periodically.
59. Blockchain Scalability - Architectures and Algorithms
Ethereum Sharding approach
to Blockchain Scalability
Sharding is actually much older than blockchain technology
and has been implemented in a variety of systems from
business database optimizations to Google’s global Spanner
database.
Essentially, sharding is a particular method for horizontally
partitioning data within a database. More generally, the
database is broken into little pieces called “shards”, that when
aggregated together form the original database.
61. Sharding Data Structure
Sharding is often referred to as a Layer 1 scaling solution because it
is implemented at the base-level protocol of ethereum itself..
Ethereum breaks down the network into specific shards.
Each shard is assigned a specific group of transactions that is
determined by grouping specific accounts (including smart
contracts) into a shard. Each transaction group has the following
structure.
• Header
• The shard ID of the transaction group
• Assignment of validators through random sampling
• State Root (state of the merkle root of the shard before
and after transactions)
• Body
• All of the transactions that belong to the transaction
group that are part of the specific shard.
63. Sharding and Transactions
• Transactions are specific to each shard and occur between
accounts native to that shard.
• When transactions are verified, the state of the network changes
and account balances, storage, etc are updated.
• In order for the transaction group to verify as valid, the pre-
state root of the transaction group must match the shard root in
the global state.
• If they match, the transaction group is validated and the global
state is updated through the particular shard ID state root.
• Instead of only containing a state root, each block of the
Ethereum blockchain now contains both a state root and the
transaction group root.
• Basically, there is a merkle root of all of the different shards that
contain the updated and verified transaction groups. This root
is stored in the blockchain along with the updated state root.
65. Cross Shard Communication
• The cross-shard communication is achieved through
applying the concept of transaction receipts.
• The receipt for a transaction is stored in a merkle root
that can be easily verified but that is not part of the
state root.
• The shard receiving a transaction from another shard
checks the merkle root to ensure that the receipt has
not been spent.
• Essentially, the receipts are stored in a shared
memory that can be verified by other shards, but not
altered.
• Therefore, through a distributed storage of receipts,
shards are able to communicate with each other.
66. Sharding and 1% attack
• A major problem is the idea of a Single-Shard
Takeover Attack, where an attacker takes over the
majority of collators in a single shard to create a
malicious shard that can submit invalid collations.
• In a 100 shard system, it takes only 1% of network
hash rate to dominate the shard
• As a solution, random sampling of validators in
each shard is recommended
• Every shard will get assigned a bunch of
validators and the ones that will actually be
validating will be randomly sampled from that
set.
68. Blockchain Scalability - Architectures and Algorithms
Blockchain Architecture
Constraints and Challenges
In a Blockchain, every fully participating node in a network
must process every transaction
Every blockchain protocol that works in this way is forced to
choose between either limiting itself to a low maximum
transaction throughput, with a resulting high per-
transaction cost, or allowing a high degree of centralisation.
69. Blockchain Scalability - Architectures and Algorithms
Understanding Constraints
Physical resource constraints
Software constraints
Data communication is limited by the speed of light,
bandwidth transmission limits, CPU processing capacity,
network consistency, network availability, network
partition tolerance
Network latency is at the very base of the constraints
It measures how long it takes for data to travel
70. Blockchain Scalability - Architectures and Algorithms
Block Size Computation in
Bitcoin Network
At this theoretical rate of transactions that means the
nodes in the network must be able to process 500 bytes
x 2,000 tps = 1 MB amount of transactions per second.
Processing a transaction involves hashing and ECDSA
signature verifications. RIPEMD-160 and SHA256
(both hash algorithms) run at about 100 megabytes per
second, so 2,000 transactions could be processed in
about 10 milliseconds, so fast enough that we don’t
need to worry about it.
71. Blockchain Scalability - Architectures and Algorithms
Block Gas Limit
Computation in Ethereum
Ethereum is a little different in that it doesn’t have a traditional
block size but a per block gas limit.
Gas is the payment unit used to pay for the computations
required to process the transaction.
This is a consistently reevaluated value based on the current
processing, storage and bandwidth conditions of the network.
This is important because not every transaction is just used for
transferring an asset but may be used to signal a contract or carry
data to it which requires the network to do some level of extra
computation.
72. Blockchain Scalability - Architectures and Algorithms
Challenges of increasing
Blockchain Transaction Rate
If the rate of transactions increases by way of
block size increase or block generation rate, the
blockchain would grow at a corresponding rate.
Currently, the Bitcoin blockchain grows at a max
rate of about 1 GB a week (max rate of about 4
GB for SegWit enabled Bitcoin). If the block size
is doubled, that rate would be 2 GB a week,
tripled it would be 3 GB a week and so on.
73. If the Blockchain scalability demands larger computational
machines, it will result in centralisation of network among the
machines with better computation power and capacity
74. Blockchain Scalability - Architectures and Algorithms
Determinants of
Blockchain Throughput
The throughput of the blockchain can be affected by two
elements, the size of the block and the block creation rate.
The difficulty of the computational problem to create a
valid block can also be lowered so that the creation of
blocks is accelerated.
Increasing the block size or increasing the rate of creation
of blocks influence the blockchain in a negative way by
introducing more forks which in turn reduces the
security threshold of the blockchain.
75. Blockchain Scalability - Architectures and Algorithms
Block size incrementation
and Blockchain Scalability
BIP 100 is about changing the 1 MB block size to a new
floating block size which is determined by the consensus
mechanism of the miners.
Another suggestion is to increase the block size by 4.4%
each 97 days until the year 2063 which is a 17.7% increase
per year.
76. Blockchain Scalability - Architectures and Algorithms
Segregated Witness
SegWit is a protocol to increase the block capacity and to provide
protection from transaction malleability
Segregated witness approach does not increase the block size limit,
but it increases the amount of transactions that can be stored in a
block. This approach in the best case scenario increases the
throughput by four times.
The segregated witness approach resolves the transaction
malleability problem and thus allows new mechanisms to be
implemented which could provide powerful tools for the scalability
issues in blockchain implementations
77. Blockchain Scalability - Architectures and Algorithms
The Greedy Heaviest
Observed Sub Tree
The basic concept behind the protocol is that blocks that
are off the main chain can still contribute to its weight
It offers performance benefits over the standard longest
chain protocol as is implemented in Bitcoin
This allows the network to set higher rates for block
creation and increase the block size without worrying
about the 51% attacks which means a higher transaction
throughput can occur within the network
78. Blockchain Scalability - Architectures and Algorithms
What happens if you split
the data into sub chains
It has the two fundamental problems
it decreases security, to the point where the security of each
individual blockchain on average does not grow at all as total
usage of the ecosystem increases,
It massively reduces the potential for interoperability, as
applications no longer have the ability to talk to each other
with zero latency and total security over a single ledger
Alternatively we can use cumbersome ”atomic cross-chain”
interaction protocols
79. Blockchain Scalability - Architectures and Algorithms
Merge Mining
Proof of work miners mining multiple chains
simultaneously which could best offer a linear trade off
between a thousand chains and a single chain extreme
If each chain is mined by only one in a thousand miners,
then it is vulnerable to very cheap attacks
If each chain is mined by every miner then the consensus
participant must process every transaction anyway
80. Blockchain Scalability - Architectures and Algorithms
Checkpointing
Techniques involving “checkpointing” a
smaller blockchain inside a larger one also
fail to provide security, as they do not
prevent an attacker with majority
participation in the smaller blockchain’s
consensus process from creating a
checkpoint of fake or unavailable data.
81. Blockchain Scalability - Architectures and Algorithms
Scalability and Implicit
Validation Processes
The problem of simultaneously achieving
the best of both worlds: having only a
small portion of consensus participants
explicitly participate in validating each
transaction, but have the entire weight of
the network implicitly stand behind each
one, has proven surprisingly hard.
82. Blockchain Scalability - Architectures and Algorithms
Light Clients and the likelihood
of scalable validations
Nodes must necessarily employ statistical and
economic means in order to make sure that other
blocks are secure
In essence, every node can only fully keep track of at
most a few parts of the entire “state”, and must
therefore be a “light client”, downloading only a
small amount of header data and not performing
full verification checks,
83. Blockchain Scalability - Architectures and Algorithms
Data Availability issues and
impact on Blockchain Scalability
It is entirely possible for a block to
appear that looks valid, and is valid, but
for which the auxiliary data is
unavailable, leading to a situation where
no other validator can effectively validate
transactions or produce new blocks since
the necessary data is unavailable to them.
84. Blockchain Scalability - Architectures and Algorithms
Parallel processing and
State Transition Functions
When transactions are processed by different
nodes in parallel, it is not possible to
parallelise arbitrary computation quite easily
We must determine the set of restrictions to a
state transition function that will best
balance parallelisability and utility.