Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Spark on SQL Server?
meetup app Osaka@3
2018/12/22 @shinsukeoda
SQL Server 2019
最近 CTP 2.2 出てます
https://docs.microsoft.com/en-us/sql/sql-
server/sql-server-ver15-release-
notes?view=sqla...
目玉機能は何?
多くの人は
これを推すはず
Big Data Clusters
Big Data Clusters
k8s で動かすクラスタ (SQL Server /
Spark / HDFS)
データの仮想化
Mongo, HDFS, Oracle とかにアクセス
Data lake
一つの場所にデータを集約(Shar...
https://docs.microsoft.com/en-us/sql/big-data-cluster/big-data-cluster-overview?view=sqlallproducts-allversions
https://docs.microsoft.com/en-us/sql/big-data-cluster/concept-storage-
pool?view=sqlallproducts-allversions
Big Data Clusters
Spark?
Apache Spark
http://spark.apache.org/
Spark ジョブ から HDFS/SQL Server の
データにアクセス
Spark on SQL Server?
Spark on SQL Server?
Big Data Clusters 知りたい人は…
SEの雑記
https://blog.engine
er-memo.com/
Java language extension
in SQL Server 2019
external_script で Java が呼べるように
なった!
external_script = 外部スクリプト
2016 で R、2017 で P...
http://sparkjava.com/
残念なお知らせ
実行は出来たけど別プロセスからアクセ
ス出来ない
同じプロセス内からならアクセス可
public static void runSqlServer() {
// Spark 設定
get("/hello", (request, response) -> "<h1>Hello Spark!! on SQL
Server?</h1...
デモ
https://docs.microsoft.com/en-us/sql/advanced-analytics/concepts/extensibility-
framework?view=sqlallproducts-allversions
実行プロセスは?
R
RTerm.exe
Python
Python.exe
Java
Java.exe
ExtHost.exe
SQL External Runtime Host
ExtHost.exe
インストールパス
/MSSQL15.MSSQLSERVER¥MSSQL¥Bi
nn¥ExtHost.exe
情報無くて動きよー分からん。
Get-NetTCPConnection と Task Manager の照合
試した結果
localhost:4567 で ExtHost.exe が待ち
受けてるのが確認出来る。
でも外部プロセスからアクセス不可
Invoke-WebRequest –Uri
http://localhost:4567/hello で蹴...
まとめ
惜しい感じで Spark on ~ じゃない。
今のところ SQL Server で Web アプリ
は動かせるけど、アクセス出来ないから
使えない。
ちなみに 外部に HTTP アクセスは出来る
ので、どっかの API 叩くとかは出来...
Prochain SlideShare
Chargement dans…5
×

Spark on sql server?

508 vues

Publié le

2018/12/22 meetup app Odaka@3
https://meetupapp.connpass.com/event/112659/

Publié dans : Technologie
  • -- DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT -- ......................................................................................................................... ......................................................................................................................... Download FULL PDF EBOOK here { http://bit.ly/2m6jJ5M } ......................................................................................................................... (Unlimited)
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (Unlimited) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { http://bit.ly/2m77EgH } ......................................................................................................................... Download Full EPUB Ebook here { http://bit.ly/2m77EgH } ......................................................................................................................... ACCESS WEBSITE for All Ebooks ......................................................................................................................... Download Full PDF EBOOK here { http://bit.ly/2m77EgH } ......................................................................................................................... Download EPUB Ebook here { http://bit.ly/2m77EgH } ......................................................................................................................... Download doc Ebook here { http://bit.ly/2m77EgH } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (Unlimited) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... Download Full EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ACCESS WEBSITE for All Ebooks ......................................................................................................................... Download Full PDF EBOOK here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... Download EPUB Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... Download doc Ebook here { https://tinyurl.com/yyxo9sk7 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (Unlimited) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... Download Full EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ACCESS WEBSITE for All Ebooks ......................................................................................................................... Download Full PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... Download EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... Download doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • DOWNLOAD THIS BOOKS INTO AVAILABLE FORMAT (Unlimited) ......................................................................................................................... ......................................................................................................................... Download Full PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... Download Full EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ACCESS WEBSITE for All Ebooks ......................................................................................................................... Download Full PDF EBOOK here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... Download EPUB Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... Download doc Ebook here { https://tinyurl.com/y6a5rkg5 } ......................................................................................................................... ......................................................................................................................... ......................................................................................................................... .............. Browse by Genre Available eBooks ......................................................................................................................... Art, Biography, Business, Chick Lit, Children's, Christian, Classics, Comics, Contemporary, Cookbooks, Crime, Ebooks, Fantasy, Fiction, Graphic Novels, Historical Fiction, History, Horror, Humor And Comedy, Manga, Memoir, Music, Mystery, Non Fiction, Paranormal, Philosophy, Poetry, Psychology, Religion, Romance, Science, Science Fiction, Self Help, Suspense, Spirituality, Sports, Thriller, Travel, Young Adult,
       Répondre 
    Voulez-vous vraiment ?  Oui  Non
    Votre message apparaîtra ici
  • Soyez le premier à aimer ceci

Spark on sql server?

  1. 1. Spark on SQL Server? meetup app Osaka@3 2018/12/22 @shinsukeoda
  2. 2. SQL Server 2019 最近 CTP 2.2 出てます https://docs.microsoft.com/en-us/sql/sql- server/sql-server-ver15-release- notes?view=sqlallproducts-allversions
  3. 3. 目玉機能は何?
  4. 4. 多くの人は これを推すはず
  5. 5. Big Data Clusters
  6. 6. Big Data Clusters k8s で動かすクラスタ (SQL Server / Spark / HDFS) データの仮想化 Mongo, HDFS, Oracle とかにアクセス Data lake 一つの場所にデータを集約(Shared HDFS) スケールアウトデータマート Compute Plane / Data Plane
  7. 7. https://docs.microsoft.com/en-us/sql/big-data-cluster/big-data-cluster-overview?view=sqlallproducts-allversions
  8. 8. https://docs.microsoft.com/en-us/sql/big-data-cluster/concept-storage- pool?view=sqlallproducts-allversions
  9. 9. Big Data Clusters Spark? Apache Spark http://spark.apache.org/ Spark ジョブ から HDFS/SQL Server の データにアクセス
  10. 10. Spark on SQL Server?
  11. 11. Spark on SQL Server?
  12. 12. Big Data Clusters 知りたい人は… SEの雑記 https://blog.engine er-memo.com/
  13. 13. Java language extension in SQL Server 2019 external_script で Java が呼べるように なった! external_script = 外部スクリプト 2016 で R、2017 で Python 2019 で Java https://docs.microsoft.com/en- us/sql/advanced-analytics/java/extension- java?view=sqlallproducts-allversions
  14. 14. http://sparkjava.com/
  15. 15. 残念なお知らせ 実行は出来たけど別プロセスからアクセ ス出来ない 同じプロセス内からならアクセス可
  16. 16. public static void runSqlServer() { // Spark 設定 get("/hello", (request, response) -> "<h1>Hello Spark!! on SQL Server?</h1>"); // HTTP でアクセス! try { String res = run("http://localhost:4567/hello"); System.out.println(res); } catch (IOException e) { System.out.println(e); } } static OkHttpClient client = new OkHttpClient(); static String run(String url) throws IOException { Request request = new Request.Builder() .url(url) .build(); try (Response response = client.newCall(request).execute()) { return response.body().string(); } }
  17. 17. デモ
  18. 18. https://docs.microsoft.com/en-us/sql/advanced-analytics/concepts/extensibility- framework?view=sqlallproducts-allversions
  19. 19. 実行プロセスは? R RTerm.exe Python Python.exe Java Java.exe ExtHost.exe SQL External Runtime Host
  20. 20. ExtHost.exe インストールパス /MSSQL15.MSSQLSERVER¥MSSQL¥Bi nn¥ExtHost.exe 情報無くて動きよー分からん。
  21. 21. Get-NetTCPConnection と Task Manager の照合
  22. 22. 試した結果 localhost:4567 で ExtHost.exe が待ち 受けてるのが確認出来る。 でも外部プロセスからアクセス不可 Invoke-WebRequest –Uri http://localhost:4567/hello で蹴ると Firewall に阻まれてる感じのレスポンス の遅さ Firewalll 無効にしてもダメ。。
  23. 23. まとめ 惜しい感じで Spark on ~ じゃない。 今のところ SQL Server で Web アプリ は動かせるけど、アクセス出来ないから 使えない。 ちなみに 外部に HTTP アクセスは出来る ので、どっかの API 叩くとかは出来そう データの仮想化!!!!

×