Les index columnstore sont apparus avec SQL Server 2012 et bon nombre de limitations ou d'améliorations ont vu le jour avec SQL Server 2014 et bientôt SQL Server 2016. Il en va de même pour les tables In-Memory à partir de SQL Server 2014. Découvrez lors de cette session comment SQL Server 2016 répond aux besoins d'analyse opérationnelle en temps réel en introduisant et en mixant ces 2 technologies In-Memory

1. #JSS2015
Les journées
SQL Server 2015
Un événement organisé par GUSS
@GUSS_FRANCE

2. #JSS2015
Les journées
SQL Server 2015
Un événement organisé par GUSS
In-Memory 2016
Operational Analytics
David Barbarin
Frédéric Pichaut

3. #JSS2015
Merci à nos sponsors

4. #JSS2015
Speakers
David Barbarin
http://blog.developpez.com/mikedavem/
david.barbarin@dbi-services.com
@mikedavem
Frédéric Pichaut
Senior Escalation Engineer EMEA, Microsoft France
@FPichaut

5. #JSS2015
Columnstore Architecture
C1 C2 C3 C4 C5
Row group
Set of rows (typically 1 million rows)
Segment
Contains values from one column for the row group
Unit of transfer between disk and memory
1- Encoding
Dictionary
> Value: Base / Scale
> Bit packing
> RLE
…
Vertipaq
2- Binary
compression
(3)- Archive
compression
Xpress 8
+
+
C1 C2 C3
C2 C4
C1 C5
C3
C5
LOB
C1 C2 C3 C4 C5
Tuple mover

6. #JSS2015
Columnstore & Batch mode
Scan
Predicate
I/O significantly reduced by
> Encoding & compression
> Segment elimination
Next challenge: CPU usage – how
to improve performance in this area?
Get row
Filtered row
X
X
X
Batch object
Column vector
C1 C2
Scan
Predicate
Get batch
Filtered batch
Process multiple rows in batch for efficiency (~1K rows)
> Using of SIMD instructions
> Optimized for 64 bits values of the register
> Significant reduction in function calls = less CPU time(7 – 40x)
Reduction of CPU latency and cache misses
> Optimized for CPU L2 cache and avoiding cache misses
> Aggressive memory pre-fetch (sequential vs random)

7. #JSS2015
SQL 2016 Columnstore Improvements
• Tables:
• Primary key, foreign keys, CDC, Triggers, temporal tables
• Change tracking (NCCI only)
• Transaction isolation level SI and RCSI
• Availability groups and readonly secondary replicas support
• CI introduced in operational analytics
• Disk-based table + NCCI (updatable and filter capabilities)
• CCI + nonclustered indexes
• In-memory table + NCCI
• ALTER TABLE .. REORGANIZE for dealing with fragmentation
• New or enhanced DMVS
• sys.dm_db_column_store_row_group_*
• sys.dm_db_index_* and sys.dm_xtp_*

8. #JSS2015
SQL 2016 Columnstore Improvements
• Batch mode
• Sort operator
• Aggregate operators and distinct
• AVG, SUM, COUNT, CHECKSUM_AGG, STDEV, STDEVP
• Windows functions
• COUNT, COUNT_BIG, SUM, AVG, MIN, MAX
• Analysis functions
• LAG, LEAD, FIRST_VALUE, LAST_VALUE, PERCENTILE_CONT,
PERCENTILE_DISC, CUME_DIST, PERCENT_RANK
• Batch mode with maxdop 1
• Parallelized insert into the delta stores
• String predicate pushdown

9. #JSS2015
Relational Table
(Clustered Index/Heap)
Btree Index
Hot
Deletebitmap
Delta rowgroups
Operational analytics: columnstore indexes & In-Memory tables
Unified view for OLTP & DW developers
High performance past period DW
Data Lifecycle
Real-time operational OLTP
Create Columnstore only on cold data – using filtered predicate to minimize maintenance
create nonclustered columnstore index ….. where order_status = ‘SHIPPED’
CCI w / High compression
Deletebitmap
Delta rowgroups

10. #JSS2015

11. #JSS2015
Memory-optimized Table
Filegroup
SQL Server.exe
Hekaton Engine: Memory_optimized Tables
& Indexes
TDS Handler and Session Management
OLTP In-Memory Architecture
Natively Compiled
SPs and Schema
Buffer Pool for Tables & Indexes
Proc/Plan cache for ad-hoc T-
SQL and SPs
Client App
Transaction Log
Query
Interop
Non-durable TableT1
T4T3
T2
T1
T4T3
T2
T1
Tables
Indexes
Interpreter for TSQL, query plans,
expressions
T1 T4T3
T4T3
Checkpoint & Recovery
Access Methods
Parser,
Catalog,
Algebrizer,
Optimizer
Hekaton
Compiler
Hekaton
Component
Key
Existing SQL
Component
Generated .dll
Data File Group
T4T3
T4T3

12. #JSS2015
SQL 2016 - Improved Scaling - Storage
• SQL Server 2014
• SQL Server 2016
– Multiple offline checkpoint threads
– Goal: 1GB/s of log generation [work in progress]
–
Single Offline Checkpoint Thread
Del Tran2
(TS 450)
Del Tran3
(TS 250)
Del
Tran1(TS150)
Insert into
Hekaton T1
Log in SQL
Table
Del Tran1
(TS150)
Del Tran2
(TS 450)
Del Tran3
(TS 250)
Insert into
Hekaton T1SQL Transaction log
disk
SQL Transaction log Del Tran2
(TS 450)
Del Tran3
(TS 250)
Del
Tran1(TS150)
Insert into
Hekaton T1
Log in SQL
Table
Del Tran1
(TS150)
Del Tran2
(TS 450)
Del Tran3
(TS 250)
Insert into
Hekaton T1
disk
Offline Checkpoint Thread Offline Checkpoint Thread Offline Checkpoint Thread

13. #JSS2015
SQL 2016 In-Memory Improvements
• Tables:
• ALTER TABLE (offline, requires 2X memory)
• Identity columns
• Indexes on NULLable columns
• COLUMNSTORE indexes
• FOREIGN KEY, CHECK, UNIQUE constraints.
• Change HASH index bucket_count through index REBUILD
• Add/drop index supported
• Stats improvements: auto-update and sampled stats
• More than 8 indexes.
• DML Triggers (Only AFTER triggers; no INSTEADOF; Natively compiled)
• 2TB of user data in durable tables (in SQL 2014 Max 256GB)
• Full collations support

14. #JSS2015
SQL 2016 In-Memory Improvements
• Native Procedure:
• ALTER PROC and sp_recompile (online and recompilation Durant execution)
• Nested native procedures
• Natively Compiled Scalar UDFs (Access from both native and interop)
• Native Inline Table-Valued Functions
• EXECUTE AS CALLER
• Security and math built-ins
• Full collations support in native modules
• Query surface area
• Subqueries
• LIKE operator
• {LEFT|RIGHT} OUTER JOIN
• Disjunction (OR, NOT)
• UNION [ALL]
• SELECT DISTINCT
• Subqueries (EXISTS, IN, scalar)
• Parallel scan for memory-optimized indexes
• MARS (Multiple Active Result Sets) support
• TDE (Transparent Data Encryption) (on-disk data files encrypted once TDE is enabled)

15. #JSS2015
Remaining In-Memory Unsupported Features
Tables
• DDL triggers and Transactional DDL
• Data types: XML, Spatial, CLR types,
datetimeoffset, rowversion, sql_variant
• ALTER TABLE ONLINE
• Cross-container transaction limitations:
snapshot/snapshot, serializable/serializable;
SAVEPOINT
• Online Migration of disk-based tables to
memory-optimized
Interop: cross-database queries, MERGE
INTO, locking hints
Native Compilation
• CASE, MERGE, JOIN in UPDATE/DELETE; DML
with OUTPUT; VIEWs
• Natively Compiled Table-Valued Functions
• Automatic and statement-level recompile
• Access to disk-based tables
• Must be schema-bound; no dynamic T-SQL
• Parallelism; limitations in query operators
(hash join/agg, merge join)
Management: replication; DB
snapshot; CDC; compression

16. #JSS2015
In-Memory OLTP Table
Updateable CCI
Tail
DRT
Range Index
Hash Index
Hot
Relational Table
(Clustered Index/Heap)
Btree Index
Hot
Deletebitmap
Delta rowgroups
Operational analytics: columnstore indexes & In-Memory tables
Unified view for OLTP & DW developers
High performance past period DW
Data Lifecycle
Real-time operational OLTP
Create Columnstore only on cold data – using filtered predicate to minimize maintenance
create nonclustered columnstore index ….. where order_status = ‘SHIPPED’
CCI w / High compression
Deletebitmap
Delta rowgroups

17. #JSS2015

18. #JSS2015#JSS2015
Les évaluations des sessions,
c’est important !!
http://GUSS.Pro/jss

19. #JSS2015
Merci à nos volontaires…

20. #JSS2015#JSS2015