Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
1
MySQLの限界に挑戦する(?!)
MySQL Casual Talks vol.10
2017-02-01 19:10 LT
@meijik
免責事項
• 本プレゼンテーションにおいて示されている
見解は、私自身の見解であって、私の所属す
る会社・団体の見解を必ずしも反映したもの
ではありません。ご了承ください。
2
3
MySQLの限界
• よいこのみんなはこれを見てるよね!
• 限界までMySQLを使い尽くす!!(漢のコンピュータ道)
– http://nippondanji.blogspot.jp/2009/05/mysql.html
• 今回のLTは...
4
SQL文の最大長
• 限界までMySQLを使い尽くす!!(漢のコン
ピュータ道)によると。。。。
– MySQLサーバーが実行出来るSQL文の最大長
は、max_allowed_packetシステム変数で表され
る。max_allowed_...
5
無意味なクエリでトライ!
UNION ALLで一連の値を返すクエリを作成。
↓こんな感じ
select repeat(' union all select "b"',4000) into
@a;
set @a= concat('select...
6
できたクエリ
• select "a" union all select "b" union all select "b" union all select "b" union all select "b" union all select...
7
できたクエリ(一部拡大)
• select "a" union all select "b" union all select "b" union all select "b" union all select "b" union all ...
8
さて実行!
mysql> select repeat(' union all select "b"',4000)
into @a;
Query OK, 1 row affected (0.00 sec)
mysql> set @a= con...
9
スタック不足
• thread_stack
– https://dev.mysql.com/doc/refman/5.6/ja/serve
r-system-variables.html#sysvar_thread_stack
• 各スレッ...
10
じゃ2倍の512KBで
• 成功!
• じゃreapeatも二倍の8000で!
mysql> prepare stmt1 from @a;
ERROR 1064 (42000): memory exhausted
near '"b" un...
11
ERROR 1064 (42000)
• 通常クエリが文法的に間違っているときによ
くでる。
• perror 1064
– MySQL error code 1064
(ER_PARSE_ERROR): %s near ‘%-.80s’...
12
ソースをfindstr
(windowsのgrepライクなコマンド)
• mysql-5.7.17¥sql>findstr /nsi
/c:"memory exhausted" *
• sql_hints.yy.cc:1362: yyov...
13
構文解析と字句解析
• クエリの解析
– 字句解析
– 構文解析
• 構文的には正しい(無意味な)複雑なクエリを
構文解析していて、内部的に用意しておいた
メモリを使い果たしたらしい。。。
• 通常のプログラムでは構文解析にyacc, 字...
14
bisonとflex
• クエリの解析
– 字句解析: lex -> GNU版がflex
– 構文解析: yacc -> GNU版がbison
15
MySQLの場合
• クエリの解析
– 字句解析: 自前で行っている
– 構文解析: BNFで記述したソースをGNU bisonに
てコンパイルして行っている。
• BNF(バッカス・ナウア記法)
• バッカス・ナウア記法(英: Back...
16
SQL標準をBNFで
• 初期のSQLは書籍などでBNF(バッカス・ナウ
ア記法)の記述があった。
• 現在奇特な方がSQL-92, 99, 2003のBNFを
公開してなさる。(多分読みきれん)
– https://github.com...
17
まとめ
• MySQLのクエリの最大長(max_allowed_packet)
• スタックに引っかかる場合→スタックサイズを
大きくして対応
• 構文解析でメモリを使い果たした場合→簡単
な対応なし(まぁほとんど起こらない)
• MyS...
宣伝: 書籍情報
• おうちで学べるデータベースのきほん
– DBの初心者はこちらをどうぞ。
– なぜか韓国語版が発売
• 入手方法
– じゃんけんで勝つ。
– もしくは、Amazonでポチる w。
18
19
THANK YOU
• ご清聴ありがとうございました。
• [fyi] yacc/lexの参考書籍
– 日本語
• lex&yaccプログラミング (オライリージャパン)
• Cコンパイラ設計―yacc/lexの応用
• yacc/lex...
20
MySQLの限界にチョット挑戦した。
MySQL Casual Talks vol.10
2017-02-01 LT
@meijik
Prochain SlideShare
Chargement dans…5
×

5

Partager

Télécharger pour lire hors ligne

MySQLの限界に挑戦する

Télécharger pour lire hors ligne

MySQLのSQLクエリ最大長に朝鮮

MySQLの限界に挑戦する

  1. 1. 1 MySQLの限界に挑戦する(?!) MySQL Casual Talks vol.10 2017-02-01 19:10 LT @meijik
  2. 2. 免責事項 • 本プレゼンテーションにおいて示されている 見解は、私自身の見解であって、私の所属す る会社・団体の見解を必ずしも反映したもの ではありません。ご了承ください。 2
  3. 3. 3 MySQLの限界 • よいこのみんなはこれを見てるよね! • 限界までMySQLを使い尽くす!!(漢のコンピュータ道) – http://nippondanji.blogspot.jp/2009/05/mysql.html • 今回のLTはそれを順に見ていこうとしました が、結果は。。。。
  4. 4. 4 SQL文の最大長 • 限界までMySQLを使い尽くす!!(漢のコン ピュータ道)によると。。。。 – MySQLサーバーが実行出来るSQL文の最大長 は、max_allowed_packetシステム変数で表され る。max_allowed_packetの最大値は1GBであ る。max_allowed_packetの値はセッションごとに も設定可能なので、デフォルトではそこそこの値 (16MBなど)に設定しておいて、必要に応じて大 きな対を使うと良いだろう。 – デフォルト値は5.6より前は1MB, 5.6以降4MB
  5. 5. 5 無意味なクエリでトライ! UNION ALLで一連の値を返すクエリを作成。 ↓こんな感じ select repeat(' union all select "b"',4000) into @a; set @a= concat('select "a" ', @a); prepare stmt1 from @a; execute stmt1; deallocate prepare stmt1;
  6. 6. 6 できたクエリ • select "a" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union allselect "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b"
  7. 7. 7 できたクエリ(一部拡大) • select "a" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union allselect "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" union all select "b" select “a” union all select “b” union all select “b” …
  8. 8. 8 さて実行! mysql> select repeat(' union all select "b"',4000) into @a; Query OK, 1 row affected (0.00 sec) mysql> set @a= concat('select "a" ', @a); Query OK, 0 rows affected (0.00 sec) mysql> prepare stmt1 from @a; ERROR 1436 (HY000): Thread stack overrun: 246184 bytes used of a 262144 byte stack, and 16000 bytes needed. Use 'mysqld -- thread_stack=#' to specify a bigger stack.
  9. 9. 9 スタック不足 • thread_stack – https://dev.mysql.com/doc/refman/5.6/ja/serve r-system-variables.html#sysvar_thread_stack • 各スレッドのスタックサイズ。デフォルトの 256KBは、通常の操作では十分な大きさです。 スレッドスタックサイズが小さすぎるとサーバ で処理できる SQL ステートメントの複雑さ、 ストアドプロシージャーの再帰の深さなど、メ モリーを大量に消費する処理が制限されます。
  10. 10. 10 じゃ2倍の512KBで • 成功! • じゃreapeatも二倍の8000で! mysql> prepare stmt1 from @a; ERROR 1064 (42000): memory exhausted near '"b" union all select "b"' at line 1 →失敗orz
  11. 11. 11 ERROR 1064 (42000) • 通常クエリが文法的に間違っているときによ くでる。 • perror 1064 – MySQL error code 1064 (ER_PARSE_ERROR): %s near ‘%-.80s’ at line %d • ただ今回は文法的にはあっている(はず)で キーワード(%s) が “memory exhausted “
  12. 12. 12 ソースをfindstr (windowsのgrepライクなコマンド) • mysql-5.7.17¥sql>findstr /nsi /c:"memory exhausted" * • sql_hints.yy.cc:1362: yyoverflow (YY_("memory exhausted"), • sql_hints.yy.cc:2099: yyerror (thd, scanner, ret, YY_("memory exhausted")); • sql_yacc.cc:19899: yyoverflow (YY_("memory exhausted"), • sql_yacc.cc:41019: yyerror (&yylloc, YYTHD, YY_("memory exhausted"));
  13. 13. 13 構文解析と字句解析 • クエリの解析 – 字句解析 – 構文解析 • 構文的には正しい(無意味な)複雑なクエリを 構文解析していて、内部的に用意しておいた メモリを使い果たしたらしい。。。 • 通常のプログラムでは構文解析にyacc, 字句 解析にlexを利用する。 – 拡張子.yyはyaccのソースファイル
  14. 14. 14 bisonとflex • クエリの解析 – 字句解析: lex -> GNU版がflex – 構文解析: yacc -> GNU版がbison
  15. 15. 15 MySQLの場合 • クエリの解析 – 字句解析: 自前で行っている – 構文解析: BNFで記述したソースをGNU bisonに てコンパイルして行っている。 • BNF(バッカス・ナウア記法) • バッカス・ナウア記法(英: Backus-Naur form)とは、文脈自由文法を定義するのに用 いられるメタ言語のことで、一般にBNFやBN 記法と略される。(wikipediaより引用)
  16. 16. 16 SQL標準をBNFで • 初期のSQLは書籍などでBNF(バッカス・ナウ ア記法)の記述があった。 • 現在奇特な方がSQL-92, 99, 2003のBNFを 公開してなさる。(多分読みきれん) – https://github.com/ronsavage/SQL • これを使えば多分クエリの整形ツールをつ くったり(多分できる)、RDBMSを作ったりする ことができると思われます。(多分無理)
  17. 17. 17 まとめ • MySQLのクエリの最大長(max_allowed_packet) • スタックに引っかかる場合→スタックサイズを 大きくして対応 • 構文解析でメモリを使い果たした場合→簡単 な対応なし(まぁほとんど起こらない) • MySQLのビルドになぜbisonが必要なのか、 と@kumagiのツイートの意味が分かった。
  18. 18. 宣伝: 書籍情報 • おうちで学べるデータベースのきほん – DBの初心者はこちらをどうぞ。 – なぜか韓国語版が発売 • 入手方法 – じゃんけんで勝つ。 – もしくは、Amazonでポチる w。 18
  19. 19. 19 THANK YOU • ご清聴ありがとうございました。 • [fyi] yacc/lexの参考書籍 – 日本語 • lex&yaccプログラミング (オライリージャパン) • Cコンパイラ設計―yacc/lexの応用 • yacc/lex―プログラムジェネレータon UNIX – 英語 • flex&bison (O‘reilly) – 比較的新しい(2009/08)本(@meijikは未見) – SQLの例がある(らしい)
  20. 20. 20 MySQLの限界にチョット挑戦した。 MySQL Casual Talks vol.10 2017-02-01 LT @meijik
  • terumba

    May. 6, 2017
  • magicvox

    Feb. 7, 2017
  • lockcole

    Feb. 5, 2017
  • masahirohasegawa92

    Feb. 5, 2017
  • hiroaki_imai

    Feb. 5, 2017

MySQLのSQLクエリ最大長に朝鮮

Vues

Nombre de vues

5 108

Sur Slideshare

0

À partir des intégrations

0

Nombre d'intégrations

164

Actions

Téléchargements

12

Partages

0

Commentaires

0

Mentions J'aime

5

×