SlideShare a Scribd company logo
1 of 33
Download to read offline
Cache, cache, cache - Visionary’s Guide to
configuring Tableau Cache
Tableau Server & Cloud Admin TUG - June 2022
Mark Wu (Tableau Visionary)
enterprisetableau.com
markwu2000@gmail.com
2
What is Cache
Cache stores recently used query
results so that it can be quickly
rendered at a later time without
querying source data again
3
Cache always happens
•access the view
•run subscription
•API ….
4
Cache happens all layers
Data Query
Cache
Render Tile
Cache
Browser (client side)
Cache
Singlecachepolicy on server
Live Connection
Shorter cache
Extract
Longer cache
DefaultCachepolicyonTableauserver
Cache and reuse as long as
possible
DefaultCachepolicyonTableauCloud
12 hrs (not customizable)
(tsm data-access caching set -r low)
If mostly Live Connections
tsmdata-accesscachingset-ralways
Alwaysget the latest data : cache
refreshes each time a page is reloaded.
TableauServerCacheConfig
2
Server:MixedLive&Extract(mostcommon)
1
12+ Hrs
tsmdata-access
cachingset-r720
Live: Workbook level data refresh
policy by owner
3
(v2021.3)
Extract: View Acceleration by owner
4
(v2022.1)
Recommend: Longer Cache Policy (12-24 hrs) with cache warmup
tsm data-access caching set -r <value>
◦ low or empty string (""). Default. Aways use cached data
◦ <value>. "<value>" maximum minutes data should be cached
◦ always or 0 (zero). Always get the latest data
1. Server (not site)
2. Tableau Cloud 720 mins (12 hrs)
3. Mark’s server recommendation : 720-1440 mins
4. All workbooks (live or extract) except …
a. Row Level Security
b. Data fresh policy on live workbooks
#1:
10
warm up cache only for frequently used
workbooks (not all workbooks)
#2: Pre-compute Query Cache After Extract
Workbook
Performance
CPU and
memory
# of views of the workbook last 7 days
—————————————————— >= 2
# of refresh in next 7 days
#2: Pre-compute Query Cache After Extract
By default,
# of views of the workbook last 7 days
—————————————————— >= 2
# of refresh in next 7 days
#2: Pre-compute Query Cache After Extract
By default,
backgrounder.externalquerycachewarmup.view_threshold 2
Implement Pre-compute Query Cache After Extract
1.Enabled by default: backgrounder.externalquerycachewarmup.enabled true
2.Config: backgrounder.externalquerycachewarmup.view_threshold 2
(# of views of the workbook last 7 days / # of refresh in next 7 days)
3.Observe &Adjust
Deciding Cachewarmup Threshold?
https://enterprisetableau.com/cachewarmup/
Download workbook
#3: Live - Set Data Refresh Policy by owner
(v2021.3)
#4: Extract - View Acceleration by owner
(v2022.1)
View Acceleration : Force pre-compute cache after extract refresh for
the specific queries used in the view only
View Acceleration vs Pre-compute
Query Cache After Extract
What Pre-compute
Query Cache
View Acceleration
Who controls Admin Content Owner
Trigger Extract refresh Extract refresh
Scope Extract workbooks only Extract workbooks only
Exclude RLS workbooks Yes No but only for thumbnail
generation user
All frequently used workbook Yes Selected workbook
Exclude fast workbooks No Yes
Exclude refresh often
workbooks
In context of usage Yes - 6 times or more refresh/
daily
Exclude inactive owner
workbook
No Yes
If embedded credentials
expired
No warm up No warm up
If extract failed No warm up No warm up
Site level control Enable or not site limit
Admin blacklist Not possible Yes
tsm data-access caching set -r <value>
Workbooks do not follow the above server setting
1.Row Level Security - no cache (live or
extract)
2.Data freshness policy (live only)
Quiz 1:
tsm data-access caching set -r 720(or 1440)
Workbook w/o RLS, if
•7
:
00am: view accessed by an user
•8
:
00am: extract refreshed
•9
:
00am: John clicks the view,
would John get the changed
data?
Cache & Extract
Quiz 2:
if pre-compute workbooks flag set or not
tsm data-access caching set -r 720(or 1440)
Workbook w/o RLS and cache
warm-up flag checked, if
•7
:
00am view accessed by an user,
•8
:
00am: extract refreshed
•9
:
00am: John clicks the view,
would John get the changed
data?
Cache & Extract
Quiz 3:
if workbook is frequently used or not
tsm data-access caching set -r 720(or 1440)
Workbook w/o RLS and cache
warm-up flag checked and
workbook is the frequently used,
if
•7
:
00am view accessed by an user,
•8
:
00am: extract refreshed
•9
:
00am: John clicks the view,
would John get the changed
data?
Cache & Extract
Quiz 4:
possible browser cache, refresh
tsm data-access caching set -r 720(or 1440)
Workbook w/o RLS and cache
warm-up flag checked, if
•7
:
00am view accessed by an user,
•8
:
00am: extract refreshed
•9
:
00am: John clicks the view,
would get the changed data?
Cache & Extract
Frequently used
Less Frequently
used
Use Case:
Critical workbook, get
latest data always?
Answers:
1.Live: Data refresh policy
2021.3
2.Extract: View Acceleration
2022.1
Quiz 5:
24
Re-cap: Tableau Tools Available for Cache
Command Notes
1 tsm data-access caching set -r <value> Server level
2 Pre-compute recently viewed workbooks Server & Site level
3 backgrounder.subscription_image_caching Server level
4 No cache for Row Level Security workbook Workbook level
5 Data acceleration (for less frequently used workbook) Workbook level
6 Force a refresh
• ?:refresh=yes
• “Refresh”button
• Reload browser
Workbook level
7 Set workbook level policy (Live Connection only) Workbook level
ServerCPUandMemoryControlwith12-24hrsCache
Workbook
Performance
CPU and
memory
vizqlserver.querylimit-v180
3 mins
1 Shorter VizQL Session Timeout 3 mins (default 30 mins)
Avoidonebadlydesignedworkbook
bringingwholeservertoitsknees
Behavior of VizQL mem
80% limit : Reclamation
95-100% limit : Restart
2 VizQL Process Memory Recycle 60% system memory (default 80%)
Problem: Restart
happens too late
Question: How to let
VizQL restart earlier enough
(60% memory) before damage
made
native_api.memory_limit_enabled-vtrue-frc
native_api.memory_limit_per_process_gb-vxx -frc
Q: What is the value for host has 300GB with 4 VizQL w/o File Store?
2 VizQL Process Memory Recycle 60% system memory
1.No measurable impact on rolling
recycle VizQL process
2.Undocumented feature but
works perfect!
3.This setting applies to Hyper
process as well
A: 50 (50GB x 4 = 200GB is 66% of 300GB)
•HoSS to consume 210G
•Recommend 70% of system memory
(for example 300GB for HoSS only)
3 Hyper Process Memory ‘hyperd’ 70% system memory
Undocumented feature
but works perfect!
ProblemStatement:OnlyONE‘Hyperd’processon
eachHoSSnodesothememory_limit_per_process_gb
capHoSSnotabletoleveragesystemmemory
Solution:
hyper.srm_memory_limit_per_process_gb
-v210-frc
ServerManagementAdd-onfeature
•hyper.session_memory_limit-v10g-frc
10 GB
4 Hyper Session Memory Timeout 5-10G (default no limit)
Avoidonebadlydesignedworkbook
impactsmanyotherusers
•hyper.session_memory_limit-v10g-frc
10 GB
4 Hyper Session Memory Timeout 5-10G (default no limit)
2
Re-cap:MixedLive&Extract(mostcommon)
1
12+ Hrs
tsmdata-access
cachingset-r720
Live: Workbook level data refresh
policy by owner
3
(v2021.3)
Extract: View Acceleration by owner
4
(v2022.1)
Recommend: Longer Cache Policy (12-24 hrs) with cache warmup
How Cache Works on Tableau Server/Cloud.pdf

More Related Content

What's hot

20200811 AWS Black Belt Online Seminar CloudEndure
20200811 AWS Black Belt Online Seminar CloudEndure20200811 AWS Black Belt Online Seminar CloudEndure
20200811 AWS Black Belt Online Seminar CloudEndureAmazon Web Services Japan
 
LTO/オートローダー/仮想テープライブラリの基礎知識
LTO/オートローダー/仮想テープライブラリの基礎知識LTO/オートローダー/仮想テープライブラリの基礎知識
LTO/オートローダー/仮想テープライブラリの基礎知識MKT International Inc.
 
AWS市場動向と求められる人材、その育成方法について
AWS市場動向と求められる人材、その育成方法について AWS市場動向と求められる人材、その育成方法について
AWS市場動向と求められる人材、その育成方法について Trainocate Japan, Ltd.
 
HCL Notes and Nomad Troubleshooting for Dummies
HCL Notes and Nomad Troubleshooting for DummiesHCL Notes and Nomad Troubleshooting for Dummies
HCL Notes and Nomad Troubleshooting for Dummiespanagenda
 
Oracle Database: リリースモデルとアップグレード・パッチ計画 (2021年2月版)
Oracle Database: リリースモデルとアップグレード・パッチ計画 (2021年2月版)Oracle Database: リリースモデルとアップグレード・パッチ計画 (2021年2月版)
Oracle Database: リリースモデルとアップグレード・パッチ計画 (2021年2月版)オラクルエンジニア通信
 
20180704(20190520 Renewed) AWS Black Belt Online Seminar Amazon Elastic File ...
20180704(20190520 Renewed) AWS Black Belt Online Seminar Amazon Elastic File ...20180704(20190520 Renewed) AWS Black Belt Online Seminar Amazon Elastic File ...
20180704(20190520 Renewed) AWS Black Belt Online Seminar Amazon Elastic File ...Amazon Web Services Japan
 
猿でもわかるコンテナ
猿でもわかるコンテナ猿でもわかるコンテナ
猿でもわかるコンテナTsuyoshi Miyake
 
Active DirectoryでDHCPを使う ~DHCPサーバーとクライアントの設定~
Active DirectoryでDHCPを使う ~DHCPサーバーとクライアントの設定~Active DirectoryでDHCPを使う ~DHCPサーバーとクライアントの設定~
Active DirectoryでDHCPを使う ~DHCPサーバーとクライアントの設定~Michio Koyama
 
서버 성능에 대한 정의와 이해
서버 성능에 대한 정의와 이해서버 성능에 대한 정의와 이해
서버 성능에 대한 정의와 이해중선 곽
 
11st Legacy Application의 Spring Cloud 기반 MicroServices로 전환 개발 사례
11st Legacy Application의 Spring Cloud 기반 MicroServices로 전환 개발 사례11st Legacy Application의 Spring Cloud 기반 MicroServices로 전환 개발 사례
11st Legacy Application의 Spring Cloud 기반 MicroServices로 전환 개발 사례YongSung Yoon
 
[AWSマイスターシリーズ] Amazon ElastiCache
[AWSマイスターシリーズ] Amazon ElastiCache[AWSマイスターシリーズ] Amazon ElastiCache
[AWSマイスターシリーズ] Amazon ElastiCacheAmazon Web Services Japan
 
これで怖くない!?大規模環境で体験するDB負荷対策~垂直から水平の彼方へ~
これで怖くない!?大規模環境で体験するDB負荷対策~垂直から水平の彼方へ~これで怖くない!?大規模環境で体験するDB負荷対策~垂直から水平の彼方へ~
これで怖くない!?大規模環境で体験するDB負荷対策~垂直から水平の彼方へ~hideakikabuto
 
ストリーミングのげんざい
ストリーミングのげんざいストリーミングのげんざい
ストリーミングのげんざいTetsuya Morimoto
 
Veeam v12新機能セミナー!クラウドへの直接バックアップ、多要素認証など便利機能をまるっと解説
Veeam v12新機能セミナー!クラウドへの直接バックアップ、多要素認証など便利機能をまるっと解説Veeam v12新機能セミナー!クラウドへの直接バックアップ、多要素認証など便利機能をまるっと解説
Veeam v12新機能セミナー!クラウドへの直接バックアップ、多要素認証など便利機能をまるっと解説株式会社クライム
 
如何用k8s打造國產5G NFV平臺? 剖析經濟部5G核網技術的關鍵
如何用k8s打造國產5G NFV平臺?剖析經濟部5G核網技術的關鍵如何用k8s打造國產5G NFV平臺?剖析經濟部5G核網技術的關鍵
如何用k8s打造國產5G NFV平臺? 剖析經濟部5G核網技術的關鍵Jace Liang
 
20200818 AWS Black Belt Online Seminar AWS Shield Advanced
20200818 AWS Black Belt Online Seminar AWS Shield Advanced20200818 AWS Black Belt Online Seminar AWS Shield Advanced
20200818 AWS Black Belt Online Seminar AWS Shield AdvancedAmazon Web Services Japan
 
AWS Black Belt Online Seminar 2017 AWS Storage Gateway
AWS Black Belt Online Seminar 2017 AWS Storage GatewayAWS Black Belt Online Seminar 2017 AWS Storage Gateway
AWS Black Belt Online Seminar 2017 AWS Storage GatewayAmazon Web Services Japan
 

What's hot (20)

20200811 AWS Black Belt Online Seminar CloudEndure
20200811 AWS Black Belt Online Seminar CloudEndure20200811 AWS Black Belt Online Seminar CloudEndure
20200811 AWS Black Belt Online Seminar CloudEndure
 
LTO/オートローダー/仮想テープライブラリの基礎知識
LTO/オートローダー/仮想テープライブラリの基礎知識LTO/オートローダー/仮想テープライブラリの基礎知識
LTO/オートローダー/仮想テープライブラリの基礎知識
 
AWS市場動向と求められる人材、その育成方法について
AWS市場動向と求められる人材、その育成方法について AWS市場動向と求められる人材、その育成方法について
AWS市場動向と求められる人材、その育成方法について
 
HCL Notes and Nomad Troubleshooting for Dummies
HCL Notes and Nomad Troubleshooting for DummiesHCL Notes and Nomad Troubleshooting for Dummies
HCL Notes and Nomad Troubleshooting for Dummies
 
Oracle Database: リリースモデルとアップグレード・パッチ計画 (2021年2月版)
Oracle Database: リリースモデルとアップグレード・パッチ計画 (2021年2月版)Oracle Database: リリースモデルとアップグレード・パッチ計画 (2021年2月版)
Oracle Database: リリースモデルとアップグレード・パッチ計画 (2021年2月版)
 
20180704(20190520 Renewed) AWS Black Belt Online Seminar Amazon Elastic File ...
20180704(20190520 Renewed) AWS Black Belt Online Seminar Amazon Elastic File ...20180704(20190520 Renewed) AWS Black Belt Online Seminar Amazon Elastic File ...
20180704(20190520 Renewed) AWS Black Belt Online Seminar Amazon Elastic File ...
 
猿でもわかるコンテナ
猿でもわかるコンテナ猿でもわかるコンテナ
猿でもわかるコンテナ
 
Privilege Management Solution
Privilege Management SolutionPrivilege Management Solution
Privilege Management Solution
 
Active DirectoryでDHCPを使う ~DHCPサーバーとクライアントの設定~
Active DirectoryでDHCPを使う ~DHCPサーバーとクライアントの設定~Active DirectoryでDHCPを使う ~DHCPサーバーとクライアントの設定~
Active DirectoryでDHCPを使う ~DHCPサーバーとクライアントの設定~
 
서버 성능에 대한 정의와 이해
서버 성능에 대한 정의와 이해서버 성능에 대한 정의와 이해
서버 성능에 대한 정의와 이해
 
11st Legacy Application의 Spring Cloud 기반 MicroServices로 전환 개발 사례
11st Legacy Application의 Spring Cloud 기반 MicroServices로 전환 개발 사례11st Legacy Application의 Spring Cloud 기반 MicroServices로 전환 개발 사례
11st Legacy Application의 Spring Cloud 기반 MicroServices로 전환 개발 사례
 
今だからこそ見直そうAzureコスト最適化
今だからこそ見直そうAzureコスト最適化今だからこそ見直そうAzureコスト最適化
今だからこそ見直そうAzureコスト最適化
 
[AWSマイスターシリーズ] Amazon ElastiCache
[AWSマイスターシリーズ] Amazon ElastiCache[AWSマイスターシリーズ] Amazon ElastiCache
[AWSマイスターシリーズ] Amazon ElastiCache
 
これで怖くない!?大規模環境で体験するDB負荷対策~垂直から水平の彼方へ~
これで怖くない!?大規模環境で体験するDB負荷対策~垂直から水平の彼方へ~これで怖くない!?大規模環境で体験するDB負荷対策~垂直から水平の彼方へ~
これで怖くない!?大規模環境で体験するDB負荷対策~垂直から水平の彼方へ~
 
ストリーミングのげんざい
ストリーミングのげんざいストリーミングのげんざい
ストリーミングのげんざい
 
Veeam v12新機能セミナー!クラウドへの直接バックアップ、多要素認証など便利機能をまるっと解説
Veeam v12新機能セミナー!クラウドへの直接バックアップ、多要素認証など便利機能をまるっと解説Veeam v12新機能セミナー!クラウドへの直接バックアップ、多要素認証など便利機能をまるっと解説
Veeam v12新機能セミナー!クラウドへの直接バックアップ、多要素認証など便利機能をまるっと解説
 
如何用k8s打造國產5G NFV平臺? 剖析經濟部5G核網技術的關鍵
如何用k8s打造國產5G NFV平臺?剖析經濟部5G核網技術的關鍵如何用k8s打造國產5G NFV平臺?剖析經濟部5G核網技術的關鍵
如何用k8s打造國產5G NFV平臺? 剖析經濟部5G核網技術的關鍵
 
20200818 AWS Black Belt Online Seminar AWS Shield Advanced
20200818 AWS Black Belt Online Seminar AWS Shield Advanced20200818 AWS Black Belt Online Seminar AWS Shield Advanced
20200818 AWS Black Belt Online Seminar AWS Shield Advanced
 
20170621 aws-black belt-ads-sms
20170621 aws-black belt-ads-sms20170621 aws-black belt-ads-sms
20170621 aws-black belt-ads-sms
 
AWS Black Belt Online Seminar 2017 AWS Storage Gateway
AWS Black Belt Online Seminar 2017 AWS Storage GatewayAWS Black Belt Online Seminar 2017 AWS Storage Gateway
AWS Black Belt Online Seminar 2017 AWS Storage Gateway
 

Similar to How Cache Works on Tableau Server/Cloud.pdf

(DAT402) Amazon RDS PostgreSQL:Lessons Learned & New Features
(DAT402) Amazon RDS PostgreSQL:Lessons Learned & New Features(DAT402) Amazon RDS PostgreSQL:Lessons Learned & New Features
(DAT402) Amazon RDS PostgreSQL:Lessons Learned & New FeaturesAmazon Web Services
 
Oracle Enteprise Manager Cloud Control 12c - Setting Up Metrics and Monitorin...
Oracle Enteprise Manager Cloud Control 12c - Setting Up Metrics and Monitorin...Oracle Enteprise Manager Cloud Control 12c - Setting Up Metrics and Monitorin...
Oracle Enteprise Manager Cloud Control 12c - Setting Up Metrics and Monitorin...Leighton Nelson
 
Data Guard Deep Dive UKOUG 2012
Data Guard Deep Dive UKOUG 2012Data Guard Deep Dive UKOUG 2012
Data Guard Deep Dive UKOUG 2012Emre Baransel
 
Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion
Advanced caching techniques with ehcache, big memory, terracotta, and coldfusionAdvanced caching techniques with ehcache, big memory, terracotta, and coldfusion
Advanced caching techniques with ehcache, big memory, terracotta, and coldfusionColdFusionConference
 
Adding Value in the Cloud with Performance Test
Adding Value in the Cloud with Performance TestAdding Value in the Cloud with Performance Test
Adding Value in the Cloud with Performance TestRodolfo Kohn
 
Caching and tuning fun for high scalability
Caching and tuning fun for high scalabilityCaching and tuning fun for high scalability
Caching and tuning fun for high scalabilityWim Godden
 
Dynamics ax performance tuning
Dynamics ax performance tuningDynamics ax performance tuning
Dynamics ax performance tuningOutsourceAX
 
Lonestar php scalingmagento
Lonestar php scalingmagentoLonestar php scalingmagento
Lonestar php scalingmagentoMathew Beane
 
Oracle Database 12c features for DBA
Oracle Database 12c features for DBAOracle Database 12c features for DBA
Oracle Database 12c features for DBAKaran Kukreja
 
Managing and Monitoring TeamPage
Managing and Monitoring TeamPageManaging and Monitoring TeamPage
Managing and Monitoring TeamPageTraction Software
 
Cloud computing 3702
Cloud computing 3702Cloud computing 3702
Cloud computing 3702Jess Coburn
 
Caching and tuning fun for high scalability
Caching and tuning fun for high scalabilityCaching and tuning fun for high scalability
Caching and tuning fun for high scalabilityWim Godden
 
Performance Optimization using Caching | Swatantra Kumar
Performance Optimization using Caching | Swatantra KumarPerformance Optimization using Caching | Swatantra Kumar
Performance Optimization using Caching | Swatantra KumarSwatantra Kumar
 
"But It Worked In Development!" - 3 Hard SQL Server Problems
"But It Worked In Development!" - 3 Hard SQL Server Problems"But It Worked In Development!" - 3 Hard SQL Server Problems
"But It Worked In Development!" - 3 Hard SQL Server ProblemsBrent Ozar
 
Caching and tuning fun for high scalability @ FOSDEM 2012
Caching and tuning fun for high scalability @ FOSDEM 2012Caching and tuning fun for high scalability @ FOSDEM 2012
Caching and tuning fun for high scalability @ FOSDEM 2012Wim Godden
 
Performance Tuning - MuraCon 2012
Performance Tuning - MuraCon 2012Performance Tuning - MuraCon 2012
Performance Tuning - MuraCon 2012eballisty
 

Similar to How Cache Works on Tableau Server/Cloud.pdf (20)

(DAT402) Amazon RDS PostgreSQL:Lessons Learned & New Features
(DAT402) Amazon RDS PostgreSQL:Lessons Learned & New Features(DAT402) Amazon RDS PostgreSQL:Lessons Learned & New Features
(DAT402) Amazon RDS PostgreSQL:Lessons Learned & New Features
 
Oracle Enteprise Manager Cloud Control 12c - Setting Up Metrics and Monitorin...
Oracle Enteprise Manager Cloud Control 12c - Setting Up Metrics and Monitorin...Oracle Enteprise Manager Cloud Control 12c - Setting Up Metrics and Monitorin...
Oracle Enteprise Manager Cloud Control 12c - Setting Up Metrics and Monitorin...
 
Data Guard Deep Dive UKOUG 2012
Data Guard Deep Dive UKOUG 2012Data Guard Deep Dive UKOUG 2012
Data Guard Deep Dive UKOUG 2012
 
Caching
CachingCaching
Caching
 
Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion
Advanced caching techniques with ehcache, big memory, terracotta, and coldfusionAdvanced caching techniques with ehcache, big memory, terracotta, and coldfusion
Advanced caching techniques with ehcache, big memory, terracotta, and coldfusion
 
Adding Value in the Cloud with Performance Test
Adding Value in the Cloud with Performance TestAdding Value in the Cloud with Performance Test
Adding Value in the Cloud with Performance Test
 
Performance Tuning
Performance TuningPerformance Tuning
Performance Tuning
 
Caching and tuning fun for high scalability
Caching and tuning fun for high scalabilityCaching and tuning fun for high scalability
Caching and tuning fun for high scalability
 
Dynamics ax performance tuning
Dynamics ax performance tuningDynamics ax performance tuning
Dynamics ax performance tuning
 
Lonestar php scalingmagento
Lonestar php scalingmagentoLonestar php scalingmagento
Lonestar php scalingmagento
 
Oracle Database 12c features for DBA
Oracle Database 12c features for DBAOracle Database 12c features for DBA
Oracle Database 12c features for DBA
 
11g R2
11g R211g R2
11g R2
 
Managing and Monitoring TeamPage
Managing and Monitoring TeamPageManaging and Monitoring TeamPage
Managing and Monitoring TeamPage
 
Cloud computing 3702
Cloud computing 3702Cloud computing 3702
Cloud computing 3702
 
Caching and tuning fun for high scalability
Caching and tuning fun for high scalabilityCaching and tuning fun for high scalability
Caching and tuning fun for high scalability
 
AppFabric Velocity
AppFabric VelocityAppFabric Velocity
AppFabric Velocity
 
Performance Optimization using Caching | Swatantra Kumar
Performance Optimization using Caching | Swatantra KumarPerformance Optimization using Caching | Swatantra Kumar
Performance Optimization using Caching | Swatantra Kumar
 
"But It Worked In Development!" - 3 Hard SQL Server Problems
"But It Worked In Development!" - 3 Hard SQL Server Problems"But It Worked In Development!" - 3 Hard SQL Server Problems
"But It Worked In Development!" - 3 Hard SQL Server Problems
 
Caching and tuning fun for high scalability @ FOSDEM 2012
Caching and tuning fun for high scalability @ FOSDEM 2012Caching and tuning fun for high scalability @ FOSDEM 2012
Caching and tuning fun for high scalability @ FOSDEM 2012
 
Performance Tuning - MuraCon 2012
Performance Tuning - MuraCon 2012Performance Tuning - MuraCon 2012
Performance Tuning - MuraCon 2012
 

More from Mark Wu

Tableau Administrators User Group - Data Governance
Tableau Administrators User Group - Data GovernanceTableau Administrators User Group - Data Governance
Tableau Administrators User Group - Data GovernanceMark Wu
 
Run IT as Business Meetup self-service BI
Run IT as Business Meetup self-service BIRun IT as Business Meetup self-service BI
Run IT as Business Meetup self-service BIMark Wu
 
Tableau Customer Advocacy Summit March 2016
Tableau Customer Advocacy Summit March 2016Tableau Customer Advocacy Summit March 2016
Tableau Customer Advocacy Summit March 2016Mark Wu
 
ISATUG meetup Feb 9, 2016
ISATUG meetup Feb 9, 2016ISATUG meetup Feb 9, 2016
ISATUG meetup Feb 9, 2016Mark Wu
 
Enterprise TUG Webinar 9.2 Upgrade 2-15-16
Enterprise TUG Webinar 9.2 Upgrade 2-15-16Enterprise TUG Webinar 9.2 Upgrade 2-15-16
Enterprise TUG Webinar 9.2 Upgrade 2-15-16Mark Wu
 
NetApp Tableau Presentation Final
NetApp Tableau Presentation FinalNetApp Tableau Presentation Final
NetApp Tableau Presentation FinalMark Wu
 

More from Mark Wu (6)

Tableau Administrators User Group - Data Governance
Tableau Administrators User Group - Data GovernanceTableau Administrators User Group - Data Governance
Tableau Administrators User Group - Data Governance
 
Run IT as Business Meetup self-service BI
Run IT as Business Meetup self-service BIRun IT as Business Meetup self-service BI
Run IT as Business Meetup self-service BI
 
Tableau Customer Advocacy Summit March 2016
Tableau Customer Advocacy Summit March 2016Tableau Customer Advocacy Summit March 2016
Tableau Customer Advocacy Summit March 2016
 
ISATUG meetup Feb 9, 2016
ISATUG meetup Feb 9, 2016ISATUG meetup Feb 9, 2016
ISATUG meetup Feb 9, 2016
 
Enterprise TUG Webinar 9.2 Upgrade 2-15-16
Enterprise TUG Webinar 9.2 Upgrade 2-15-16Enterprise TUG Webinar 9.2 Upgrade 2-15-16
Enterprise TUG Webinar 9.2 Upgrade 2-15-16
 
NetApp Tableau Presentation Final
NetApp Tableau Presentation FinalNetApp Tableau Presentation Final
NetApp Tableau Presentation Final
 

Recently uploaded

State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!Memoori
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...marcuskenyatta275
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024Lorenzo Miniero
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceSamy Fodil
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data SciencePaolo Missier
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxjbellis
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...panagenda
 
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)Samir Dash
 
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...ScyllaDB
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentationyogeshlabana357357
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMKumar Satyam
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftshyamraj55
 
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties ReimaginedEasier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties Reimaginedpanagenda
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingScyllaDB
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)Wonjun Hwang
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptxFIDO Alliance
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Skynet Technologies
 
How to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in PakistanHow to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in Pakistandanishmna97
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityVictorSzoltysek
 

Recently uploaded (20)

State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!State of the Smart Building Startup Landscape 2024!
State of the Smart Building Startup Landscape 2024!
 
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
TEST BANK For, Information Technology Project Management 9th Edition Kathy Sc...
 
WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024WebRTC and SIP not just audio and video @ OpenSIPS 2024
WebRTC and SIP not just audio and video @ OpenSIPS 2024
 
WebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM PerformanceWebAssembly is Key to Better LLM Performance
WebAssembly is Key to Better LLM Performance
 
Design and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data ScienceDesign and Development of a Provenance Capture Platform for Data Science
Design and Development of a Provenance Capture Platform for Data Science
 
Vector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptxVector Search @ sw2con for slideshare.pptx
Vector Search @ sw2con for slideshare.pptx
 
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
Easier, Faster, and More Powerful – Alles Neu macht der Mai -Wir durchleuchte...
 
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
AI+A11Y 11MAY2024 HYDERBAD GAAD 2024 - HelloA11Y (11 May 2024)
 
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
Event-Driven Architecture Masterclass: Integrating Distributed Data Stores Ac...
 
AI mind or machine power point presentation
AI mind or machine power point presentationAI mind or machine power point presentation
AI mind or machine power point presentation
 
Introduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDMIntroduction to use of FHIR Documents in ABDM
Introduction to use of FHIR Documents in ABDM
 
Oauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoftOauth 2.0 Introduction and Flows with MuleSoft
Oauth 2.0 Introduction and Flows with MuleSoft
 
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties ReimaginedEasier, Faster, and More Powerful – Notes Document Properties Reimagined
Easier, Faster, and More Powerful – Notes Document Properties Reimagined
 
Event-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream ProcessingEvent-Driven Architecture Masterclass: Challenges in Stream Processing
Event-Driven Architecture Masterclass: Challenges in Stream Processing
 
CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)CORS (Kitworks Team Study 양다윗 발표자료 240510)
CORS (Kitworks Team Study 양다윗 발표자료 240510)
 
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
TrustArc Webinar - Unified Trust Center for Privacy, Security, Compliance, an...
 
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider  Progress from Awareness to Implementation.pptxTales from a Passkey Provider  Progress from Awareness to Implementation.pptx
Tales from a Passkey Provider Progress from Awareness to Implementation.pptx
 
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
Human Expert Website Manual WCAG 2.0 2.1 2.2 Audit - Digital Accessibility Au...
 
How to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in PakistanHow to Check GPS Location with a Live Tracker in Pakistan
How to Check GPS Location with a Live Tracker in Pakistan
 
ChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps ProductivityChatGPT and Beyond - Elevating DevOps Productivity
ChatGPT and Beyond - Elevating DevOps Productivity
 

How Cache Works on Tableau Server/Cloud.pdf

  • 1. Cache, cache, cache - Visionary’s Guide to configuring Tableau Cache Tableau Server & Cloud Admin TUG - June 2022 Mark Wu (Tableau Visionary) enterprisetableau.com markwu2000@gmail.com
  • 2. 2 What is Cache Cache stores recently used query results so that it can be quickly rendered at a later time without querying source data again
  • 3. 3 Cache always happens •access the view •run subscription •API ….
  • 4. 4 Cache happens all layers Data Query Cache Render Tile Cache Browser (client side) Cache
  • 5. Singlecachepolicy on server Live Connection Shorter cache Extract Longer cache
  • 6. DefaultCachepolicyonTableauserver Cache and reuse as long as possible DefaultCachepolicyonTableauCloud 12 hrs (not customizable) (tsm data-access caching set -r low)
  • 7. If mostly Live Connections tsmdata-accesscachingset-ralways Alwaysget the latest data : cache refreshes each time a page is reloaded. TableauServerCacheConfig
  • 8. 2 Server:MixedLive&Extract(mostcommon) 1 12+ Hrs tsmdata-access cachingset-r720 Live: Workbook level data refresh policy by owner 3 (v2021.3) Extract: View Acceleration by owner 4 (v2022.1) Recommend: Longer Cache Policy (12-24 hrs) with cache warmup
  • 9. tsm data-access caching set -r <value> ◦ low or empty string (""). Default. Aways use cached data ◦ <value>. "<value>" maximum minutes data should be cached ◦ always or 0 (zero). Always get the latest data 1. Server (not site) 2. Tableau Cloud 720 mins (12 hrs) 3. Mark’s server recommendation : 720-1440 mins 4. All workbooks (live or extract) except … a. Row Level Security b. Data fresh policy on live workbooks #1:
  • 10. 10 warm up cache only for frequently used workbooks (not all workbooks) #2: Pre-compute Query Cache After Extract
  • 11. Workbook Performance CPU and memory # of views of the workbook last 7 days —————————————————— >= 2 # of refresh in next 7 days #2: Pre-compute Query Cache After Extract By default,
  • 12. # of views of the workbook last 7 days —————————————————— >= 2 # of refresh in next 7 days #2: Pre-compute Query Cache After Extract By default, backgrounder.externalquerycachewarmup.view_threshold 2
  • 13. Implement Pre-compute Query Cache After Extract 1.Enabled by default: backgrounder.externalquerycachewarmup.enabled true 2.Config: backgrounder.externalquerycachewarmup.view_threshold 2 (# of views of the workbook last 7 days / # of refresh in next 7 days) 3.Observe &Adjust
  • 15. #3: Live - Set Data Refresh Policy by owner (v2021.3)
  • 16. #4: Extract - View Acceleration by owner (v2022.1) View Acceleration : Force pre-compute cache after extract refresh for the specific queries used in the view only
  • 17. View Acceleration vs Pre-compute Query Cache After Extract What Pre-compute Query Cache View Acceleration Who controls Admin Content Owner Trigger Extract refresh Extract refresh Scope Extract workbooks only Extract workbooks only Exclude RLS workbooks Yes No but only for thumbnail generation user All frequently used workbook Yes Selected workbook Exclude fast workbooks No Yes Exclude refresh often workbooks In context of usage Yes - 6 times or more refresh/ daily Exclude inactive owner workbook No Yes If embedded credentials expired No warm up No warm up If extract failed No warm up No warm up Site level control Enable or not site limit Admin blacklist Not possible Yes
  • 18.
  • 19. tsm data-access caching set -r <value> Workbooks do not follow the above server setting 1.Row Level Security - no cache (live or extract) 2.Data freshness policy (live only) Quiz 1:
  • 20. tsm data-access caching set -r 720(or 1440) Workbook w/o RLS, if •7 : 00am: view accessed by an user •8 : 00am: extract refreshed •9 : 00am: John clicks the view, would John get the changed data? Cache & Extract Quiz 2: if pre-compute workbooks flag set or not
  • 21. tsm data-access caching set -r 720(or 1440) Workbook w/o RLS and cache warm-up flag checked, if •7 : 00am view accessed by an user, •8 : 00am: extract refreshed •9 : 00am: John clicks the view, would John get the changed data? Cache & Extract Quiz 3: if workbook is frequently used or not
  • 22. tsm data-access caching set -r 720(or 1440) Workbook w/o RLS and cache warm-up flag checked and workbook is the frequently used, if •7 : 00am view accessed by an user, •8 : 00am: extract refreshed •9 : 00am: John clicks the view, would John get the changed data? Cache & Extract Quiz 4: possible browser cache, refresh
  • 23. tsm data-access caching set -r 720(or 1440) Workbook w/o RLS and cache warm-up flag checked, if •7 : 00am view accessed by an user, •8 : 00am: extract refreshed •9 : 00am: John clicks the view, would get the changed data? Cache & Extract Frequently used Less Frequently used Use Case: Critical workbook, get latest data always? Answers: 1.Live: Data refresh policy 2021.3 2.Extract: View Acceleration 2022.1 Quiz 5:
  • 24. 24 Re-cap: Tableau Tools Available for Cache Command Notes 1 tsm data-access caching set -r <value> Server level 2 Pre-compute recently viewed workbooks Server & Site level 3 backgrounder.subscription_image_caching Server level 4 No cache for Row Level Security workbook Workbook level 5 Data acceleration (for less frequently used workbook) Workbook level 6 Force a refresh • ?:refresh=yes • “Refresh”button • Reload browser Workbook level 7 Set workbook level policy (Live Connection only) Workbook level
  • 26. vizqlserver.querylimit-v180 3 mins 1 Shorter VizQL Session Timeout 3 mins (default 30 mins) Avoidonebadlydesignedworkbook bringingwholeservertoitsknees
  • 27. Behavior of VizQL mem 80% limit : Reclamation 95-100% limit : Restart 2 VizQL Process Memory Recycle 60% system memory (default 80%) Problem: Restart happens too late Question: How to let VizQL restart earlier enough (60% memory) before damage made
  • 28. native_api.memory_limit_enabled-vtrue-frc native_api.memory_limit_per_process_gb-vxx -frc Q: What is the value for host has 300GB with 4 VizQL w/o File Store? 2 VizQL Process Memory Recycle 60% system memory 1.No measurable impact on rolling recycle VizQL process 2.Undocumented feature but works perfect! 3.This setting applies to Hyper process as well A: 50 (50GB x 4 = 200GB is 66% of 300GB)
  • 29. •HoSS to consume 210G •Recommend 70% of system memory (for example 300GB for HoSS only) 3 Hyper Process Memory ‘hyperd’ 70% system memory Undocumented feature but works perfect! ProblemStatement:OnlyONE‘Hyperd’processon eachHoSSnodesothememory_limit_per_process_gb capHoSSnotabletoleveragesystemmemory Solution: hyper.srm_memory_limit_per_process_gb -v210-frc ServerManagementAdd-onfeature
  • 30. •hyper.session_memory_limit-v10g-frc 10 GB 4 Hyper Session Memory Timeout 5-10G (default no limit) Avoidonebadlydesignedworkbook impactsmanyotherusers
  • 31. •hyper.session_memory_limit-v10g-frc 10 GB 4 Hyper Session Memory Timeout 5-10G (default no limit)
  • 32. 2 Re-cap:MixedLive&Extract(mostcommon) 1 12+ Hrs tsmdata-access cachingset-r720 Live: Workbook level data refresh policy by owner 3 (v2021.3) Extract: View Acceleration by owner 4 (v2022.1) Recommend: Longer Cache Policy (12-24 hrs) with cache warmup