An introduction to SQL 2014 in-memory OLTP know as Hekaton (XTP) Introduction. I presented this session at SQL User Group Melbourne sometime back. Suitable for people who are getting to know more about Hekaton
2. Why Hekaton (XTP)?
Market need for ever higher throughput and
lower latency OLTP at a lower cost
HW trends demand architectural changes on
RDBMS to meet those demands
Stalled CPU’s, Attainment of Balanced Systems.
Hekaton is: High performance, memoryoptimized OLTP engine integrated into SQL Server
and architected for modern hardware trends
2
3. Why Hekaton (XTP)?
Market need for ever higher throughput and
lower latency OLTP at a lower cost
HW trends demand architectural changes on
RDBMS to meet those demands
Stalled CPU’s, Attainment of Balanced Systems.
Hekaton is: High performance, memoryoptimized OLTP engine integrated into SQL Server
and architected for modern hardware trends
3
4. What is Hekaton (XTP)?
Hekaton is Greek for “hundreds,” and it was given this name for its
ability to speed up database function 100x (possibly)
With a new latch-free technology Hekaton is claimed to
dramatically increase performance.
SQL Server 2014 allows you to migrate the most-used tables in an
existing database to memory-optimised 'Hekaton' technology.
It is also known as SQL Server In-Memory database, touted to
accelerate transaction throughput up to 30x performance increases
on existing hardware.
4
5. What is Hekaton (XTP)?
Hekaton is Greek for “hundreds,” and it was given this name for its
ability to speed up database function 100x (possibly)
With a new latch-free technology Hekaton is claimed to
dramatically increase performance.
Hekaton doesn’t use SQL’s Buffer Cache
SQLor VAS, it allows youeverything most-used tables in an
Server 2014 stores to migrate the in the RAM
existing database to memory-optimised 'Hekaton' technology.
It is also known as SQL Server In-Memory database, touted to
accelerate transaction throughput up to 30x performance increases
on existing hardware.
5
6. Drivers
Architectural Pillars
Customer
Benefits
In-Memory OLTP – Architectural Pillars
High performance
data operations
Efficient businesslogic processing
Frictionless scaleup
Hybrid engine and
integrated
experience
Main-Memory
Optimized
T-SQL Compiled to
Machine Code
High Concurrency
SQL Server
Integration
• Optimized for in-memory
data
• Indexes (hash and
range) exist only in
memory
• No buffer pool, B-trees
• Stream-based storage
• T-SQL compiled to
machine code via C code
generator and VC
• Invoking a procedure is
just a DLL entry-point
• Aggressive
optimizations @
compile-time
• Multi-version optimistic
concurrency control with
full ACID support
• Core engine uses lockfree algorithms
• No lock manager, latches
or spinlocks
Business
Hardware trends
Steadily declining
memory price, NVRAM
Stalling CPU clock rate
• Same
manageability, administr
ation & development
experience
• Integrated queries &
transactions
• Integrated HA and
backup/restore
Many-core processors
TCO
7. In-Memory OLTP Integration and Application
Migration
Client App
TDS Handler and Session Management
Natively Compiled
SPs and Schema
Parser, Cat
alog, Opti
mizer
Native
Compiler
Key
Existing
SQL
Component
T-SQL Query Execution
In-mem
OLTP
Component
Query
Interop
T1
T2
T3
Tables
Indexes
T1
Memory Optimized Data
Filegroup
Buffer Pool for Tables & Indexes
SQL Server.exe
Memory Optimized Tables & Indexes
T2
Transaction Log
T3
T1
T2
Data Filegroup
T3
Generated
.dll
Moore’s law for CPU and Transistors and CPU Stalled, Memory cheaper (think tank behind In-Memory OLTP)the DigitalAlpha 21164 microprocessor had 9.3 million transistors
Moore’s law for CPU and Transistors and CPU Stalled, Memory cheaper (think tank behind In-Memory OLTP)the DigitalAlpha 21164 microprocessor had 9.3 million transistors
Hekaton is a really interesting technology, but is a world away from the functionality that we know and love. The SQL team have done a great job of disguising this departure from us by integrating it inside the SQL Server engine but none the less is a different beast entirely. Although the ultimate aim, I would imagine, is a seamless integration where the user ( and developer) is not really concerned with the underlying storage technology there will be many real world issues occurring if the differences are not fully understood. Hekaton doesn’t use SQL’s Buffer Cache or VAS, it stores everything into the RAMThe way that Hekaton stores data is in hash buckets, this is a fundamental tenet. A hash is simply a function applied to some key data and the bucket is where the relating row is stored. For example : if our hash function was X%5 then our buckets for the values 1 through 10 would be populated thusly : Bucket Values 5,10 1,6 2,7 3,8 4,9 As % is the function for modulo (divide an return the remainder) 9%5 = 4. The hash function for SQL Server would be much more complicated than this and like the hash function used in a hash join will differ depending on the data being hashed. So this is quite interesting since when we define a hekaton table we need to specify the number of buckets that we think ( perhaps even assume that we need) upfront:
By taking advantage of modern HW trends, we will enable new applications with higher real-time processing needs, and hence increase the addressable market size. Pigeon hole: For mailboxes.. Collisions are ought to happen, hash bucket collisions.. Reduce it