Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
Taro L. Saito, Ph.D.
Arm Treasure Data
July 9th, 2019
Arm Treasure Data Tokyo Office
Airframe HTTP
Airframe Meetup #2
1
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
Today’s Agenda
● Airframe Updates 2019
● New modul...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
About Me: Taro L. Saito (Leo)
3
● Principal Softwa...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
New Release from O’Reilly Japan
● Techniques and c...
400+
Customers
Founded in
2011
Raised
$54M
Security
Acquired by Arm / Softbank
2018
Arm Treasure Data
Arm Treasure Dataの概要

Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
The Architecture of Arm Treasure Data
6
DataLogs
D...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
Module Mix-InPackaging
HTTP Requests and
Responses...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
5 Years Ago (2014)
● No Scala engineer in the comp...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
Small Investments to Scala
● For saving time of co...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
Airframe
● Lightweight Building Blocks for Scala
●...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
Our OSS Strategy Around Scala
● Gather the best pr...
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
airframe-http
● Created a simple HTTP framework
● ...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
Airframe Updates 2019
● The latest version 19.7.2 ...
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
airframe-di: Scala-Friendly DI
● Dependency inject...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
airframe-di use cases
● Configure a lot of modules
...
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
airframe-di: Constructor vs In-Trait Injections
● ...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
New: More Detailed DI Failure Messages
17
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
New: DI Tracing
● chrome://tracing
● Tracer interf...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
New: DI Coverage Stats
19
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
New: airframe-canvas
● Off-heap memory buffer
● Save...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
Update: airframe-log
● Added msec to log time stam...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
airframe-launcher (renamed from airframe-opts)
● H...
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
airframe-config: Application Configuration Flow
● YA...
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
New: airframe-config + DI Integration
24
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
sbt-pack plugin
● A sbt plugin to create standalon...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
Object-Data Mapping Pattern
● Data -> Object Mappi...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
airframe-msgpack: MessagePack as Universal Data Fo...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
● Many JSON libraries
● json4s, jackson, jawn, cir...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
airframe-codec: Schema-On-Read Pack/Unpack Interfa...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
Pre-defined Codecs in airframe-codec
● Primitive Co...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
PlazmaDB: MessagePack DBMS
● Fluentd -> MessagePac...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
Schema-On-Read Data Processing with MessagePack
● ...
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
airframe-surface: Object Shape Inspector
● Surface...
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
airframe-surface
● Reading Type Signatures From Sc...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
MessageCodec.of[A]: Combination of Codecs
Unpack
P...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
New: airframe-fluentd
● Collect Scala Application L...
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
New: airframe-control: Retry
● Everything can fail...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
New: airframe-control: Parallel
● Running tasks in...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
Airframe 19 Milestones
● Airframe 19 Milestones
● ...
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
airframe-http
● Created a simple HTTP framework
● ...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
airframe-http: Scala as IDL
● IDL: Interface Defini...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
Path Mapping
● :param
● Mapping to a function argu...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
airframe-http-client
● Read server responses as ta...
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Retry Patterns for HTTP requests
● Error handling ...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
RouteMatcher
● Router
● List of @Endpoint definitio...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
Http Request/Response Adapters
● Finagle Adapters
...
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
airframe-http-recorder
● Testing against actual we...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
[WIP] Scala.js RPC
● Scala.js
● Compiling Scala co...
Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved.
Demo: Live Coding
● Build a simple web server
● JS...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
Scala At Arm Treasure Data
● Scala is now an officia...
Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved.
Presto Conference Tokyo 2019
● July 11 (Thu), 2019...
Confidential © Arm 2017Confidential © Arm 2017Confidential © Arm 2017
Thank You!
Danke!
Merci!
谢谢!
ありがとう!
Gracias!
Kiitos!
52
Prochain SlideShare
Chargement dans…5
×

1

Partager

Télécharger pour lire hors ligne

Airframe HTTP - Airframe Meetup #2 Tokyo 2019-07-09

Télécharger pour lire hors ligne

Airframe HTTP enables using Scala as IDL for building web applications

The live coding demo project can be found from here https://github.com/xerial/airframe-http-demo

Airframe HTTP - Airframe Meetup #2 Tokyo 2019-07-09

  1. 1. Taro L. Saito, Ph.D. Arm Treasure Data July 9th, 2019 Arm Treasure Data Tokyo Office Airframe HTTP Airframe Meetup #2 1
  2. 2. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. Today’s Agenda ● Airframe Updates 2019 ● New modules, new features ● Airframe 19 Roadmap ● Airframe HTTP ● Scala as IDL ● Airframe HTTP internals ● Live Coding Demo ● Discussion & Hackathon 2 Airframe
  3. 3. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. About Me: Taro L. Saito (Leo) 3 ● Principal Software Engineer at Arm Treasure Data ● Building distributed query engine service ● Living in US for 4 years ● DBMS & Data Science Background ● Ph.D. of Computer Science ● OSS Projects Around Scala ● sbt-sonatype: used for releasing 3000+ Scala projects ● snappy-java: a compression library used in Spark, Parquet, etc.
  4. 4. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. New Release from O’Reilly Japan ● Techniques and concepts around distributed data processing systems ● Available at Amazon.co.jp and O’Reilly Japan web sites ● Will be published on July 18, 2019 4 分散データシステム入門の決定版の翻訳が来月発売

  5. 5. 400+ Customers Founded in 2011 Raised $54M Security Acquired by Arm / Softbank 2018 Arm Treasure Data Arm Treasure Dataの概要

  6. 6. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. The Architecture of Arm Treasure Data 6 DataLogs Device Data Batch Data PlazmaDB Table Schema Data Collection Cloud Storage Distributed Data Processing 2 million records / sec. 130 trillion records 1 billion rows processed / sec. Jobs Job Management SQL Editor Scheduler Workflows Machine Learning Treasure Data OSS Third Party OSS
  7. 7. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. Module Mix-InPackaging HTTP Requests and Responses Data airframe-launcher > _ airframe-log production: port: 10010 user: xxxx ... airframe-config airframe-codec sbt-pack airframe-fluentd Scala Objects Table Data (CSV, TSV) JSON airframe-jsonairframe-surface airframe-tablet airframe-jmx Monitor Runtime States Generate Mapping Codec Metrics & Log Data JDBC ResultSets airframe-jdbc airframe-http airframe-http-finagle Launch HTTP Services airframe DI Debug Logs Schema-On-Read Mapping Airframe
  8. 8. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. 5 Years Ago (2014) ● No Scala engineer in the company ● Scala in 2014: Scala 2.9.x 8 Knowledge Experiences Design Decisions Products 24/7 Services Business Values Programming Ruby, Java Outcome
  9. 9. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. Small Investments to Scala ● For saving time of code development ● If you can save 1 minute in a day, your can spend 6 hours for this improvement ■ Save 1 minute / day = 365 minutes / year = 6 hour investment ■ Save 10 minutes / week = 520 minutes / year = 8.6 hour investment ■ Save 1 hour / week = 52 hours / year = 2.2 day investment ● Time is your most valuable asset 9
  10. 10. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. Airframe ● Lightweight Building Blocks for Scala ● Collection of our investments to Scala ● Repackaged into wvlet.airframe in 2016 ● airframe-log ● airframe-launcher ● airframe-config ● airframe-surface ● airframe-di ● airframe-codec ● ... ● As of 2019, Airframe has 20 modules ● 35+ releases in 2018 ● Already had 17+ releases in 2019 ● Contributing to the Scala Community Build ● To test the latest Scala versions 102016年に各種ツールをAirframeとして統合。20のモジュール、頻繁なリリースサイクル
 Airframe
  11. 11. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. Our OSS Strategy Around Scala ● Gather the best practices of Scala into Airframe OSS ● Get the real experiences by operating 24/7 services 11 Knowledge Experiences Design Decisions Products 24/7 Services Business Values Programming OSS Outcome Airframe
  12. 12. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. airframe-http ● Created a simple HTTP framework ● Based on Airframe modules: ■ airframe-surface ■ airframe-codec ■ airframe-msgpack ■ etc. ● Blog ● Building Low-Friction Web Service Over Finagle ● Save the time for choosing a web framework: ● Many many web frameworks…. ● e.g, Finatra, Finch, akka-http, spring, RESTeasy, open-api, swagger, etc. 12Airframe資産を活用して、Webフレームワークも手軽に作成

  13. 13. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. Airframe Updates 2019 ● The latest version 19.7.2 (2019-07) ● YY.MM.patch versioning scheme ● was version 0.69 on October 2018 (Airframe Meetup #1) ● 30+ releases ● What’s New Since 0.69 ● New Airframe logo ● Scala 2.13 support ● New features (explained later) ● New modules ■ airframe-fluentd ■ airframe-launcher ■ airframe-http-client ■ airframe-http-recorder ■ airframe-canvas 13 Airframe
  14. 14. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. airframe-di: Scala-Friendly DI ● Dependency injection (DI) library for Scala ● Usage ● import wvlet.airframe._ ● Simple 3 Step DI ● bind ● design ● build 14
  15. 15. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. airframe-di use cases ● Configure a lot of modules ● Switch production/test modules ● Manage FILO lifecycle of services ● See also: ● Airframe Meetup #1 Report (2018) 15
  16. 16. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. airframe-di: Constructor vs In-Trait Injections ● Constructor Injection ● In-Trait Injection 16
  17. 17. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. New: More Detailed DI Failure Messages 17
  18. 18. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. New: DI Tracing ● chrome://tracing ● Tracer interface 18
  19. 19. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. New: DI Coverage Stats 19
  20. 20. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. New: airframe-canvas ● Off-heap memory buffer ● Save the CPU cost of zero-filing when initializing Java/Scala arrays. ● Managing memory outside JVM heap ■ Allocate a larger memory region than -Xmx (heap size) option. ● Quickly release allocated memory with Canvas.release. ● Even if you forget to release canvases, the allocated memory can be released upon GC. ● A successor of LArray ● https://github.com/xerial/larray ● Development Plan ● InfiniteCanvas ■ Automatically read/write data using disks ■ Cursor rollback support for error handling at airframe-codec 20
  21. 21. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. Update: airframe-log ● Added msec to log time stamps ● A Modern Logging Library for Scala (at Medium) ● ANSI color and source code location display ● Just add LogSupport trait to your class 21プログラムの開発をログメッセージで効率化する

  22. 22. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. airframe-launcher (renamed from airframe-opts) ● Handle complex command line options and nested commands ● Enabled annotation-based command line definitions 22複雑なコマンドラインプログラムを簡単に作成できるようにする

  23. 23. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. airframe-config: Application Configuration Flow ● YAML config (embedded into Docker) ● Override credentials, then bind to config objects YAML development: addr: api-dev.com production: addr: api.com Config Object case class ServerConfig( addr: String, port: Int = 8080, password: String ) production: addr: api.com command: -e production Credentials and Local Configurations Merge Immutable Object Default Parameters (e.g., port = 8080) Object Mapping 23アプリケーション設定のフローをライブラリ化
 airframe-launcher > _
  24. 24. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. New: airframe-config + DI Integration 24
  25. 25. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. sbt-pack plugin ● A sbt plugin to create standalone Scala packages ● A single folder package with bin and lib folders containing all dependent JARs ● Generates command-line launcher scripts ● My small investment in 2012 to save packaging time 25sbt-packでプログラムをパッケージングし、Dockerイメージを手軽に作成
 airframe-launcher airframe-config YAML config file Standalone Scala Package sbt-pack Dockerfile
  26. 26. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. Object-Data Mapping Pattern ● Data -> Object Mapping ● How many data readers and object mappers do we need? ● How can we save our time for handling such various data types? YAML JDBC ResultSet YAML Parser + Object Mapper Config Object Table Object Object-Relation Mapper JSON JSON Parser + Object Mapper Object 26入力データをScala Objectにマッピングしたいケースは多い

  27. 27. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. airframe-msgpack: MessagePack as Universal Data Format ● MessagePack (msgpack.org) ● Compact JSON-like binary format ● Describes data types and data values at the same time (self-describing) Object Unpack Pack JDBC ResultSet Pack/Unpack YAML JSON 27MessagePackを中間フォーマットとして使うと、オブジェクトマッパーの実装は1つに
 MessagePack
  28. 28. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. ● Many JSON libraries ● json4s, jackson, jawn, circe, uPickle, etc. ● Implemented in 2 days by using Jawn parser as a reference implementation ● JSON to MessagePack conversion ● Comparable performance with Jawn and Circe ● Saves time to work with other projects ● e.g., Adding a fast pass tailored to MessagePack airframe-json 28 JDBC ResultSet Object Unpack Pack YAML JSON MessagePack
  29. 29. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. airframe-codec: Schema-On-Read Pack/Unpack Interface ● Apply schema-on-read for Scala objects Input MessagePack Output Pack Unpack PackUnpack 29MessagePackを通したSchema-on-readデータ変換インターフェースをScalaに適用

  30. 30. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. Pre-defined Codecs in airframe-codec ● Primitive Codecs ● ByteCodec, CharCodec, ShortCodec, IntCodec, LongCodec ● FloatCodec, DoubleCodec ● StringCodec ● BooleanCodec ● TimeStampCodec ● Collection Codec ● ArrayCodec, SeqCodec, ListCodec, IndexSeqCodec, MapCodec, etc. ● OptionCodec ● JsonCodec (airframe-json) ● Java-specific Codec ● FileCodec, ZonedDateTimeCodec, JDBCResultSetCodec, etc. ● Adding Custom Codecs ● Implement MessageCodec[X] interface 30Scalaで必要なほぼ全てのデータ型へのマッピングをサポート

  31. 31. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. PlazmaDB: MessagePack DBMS ● Fluentd -> MessagePack -> Arm Treasure Data ● Automatically generating table schema from MessagePack data ● Apply schema–on-read for providing table data for Presto/Hive/Spark, etc. Table Schema Int Column Reader String Column Reader Update Schema Generate Reader Set Table Reader Schema-free Data 31 Data Collection Distributed Data Processing Arm Treasure DataはMessagePackベースのSchema-on-readシステム

  32. 32. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. Schema-On-Read Data Processing with MessagePack ● Users can store arbitrary typed data (No table design is required) ● Data can be read in a target type required by the application (e.g., SQL query) Int Float Boolean String Array Map Binary SQL BigInt parseInt toInt 0 or 1 IntCodec Pack Unpack Error or null “100” (string) 100 (int) 100 (int) 32 Logs データ読み込み時に、アプリケーションの要求する型に合わせる (Schema-on-Read)
 CSV command-line arguments
  33. 33. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. airframe-surface: Object Shape Inspector ● Surface.of[X] 33
  34. 34. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. airframe-surface ● Reading Type Signatures From ScalaSig ● Scala compiler embeds Scala Type Signatures (ScalaSig) to class files ● Surface.of[A] ■ returns A’s parameter names and types class A (data:List[B]) class A data: List[java.lang.Object] class A data: List[java.lang.Object] ScalaSig: data:List[B] javac scalac Surface.of[A] data: List[B] scala.reflect.runtime.universe.TypeTag Type erasure removes generic type information 34オブジェクトの型情報をScalaSigから取得する

  35. 35. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. MessageCodec.of[A]: Combination of Codecs Unpack Pack IntCodec StringCodec DoubleCodec MessagePack MessageCodec.of[A] 35オブジェクトの型に合わせてCodecを合成
 Surface.of[A]
  36. 36. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. New: airframe-fluentd ● Collect Scala Application Logs To Fluentd ● Scala Objects -> MessagePack -> Fluentd 36MessagePackを受け取るFluentdには、airframe-codeの出力を渡せる
 Collect Query Logs Analyze Query Logs Machine Learning Query Optimization Optimize System airframe-fluentd Scala Objects airframe-codec
  37. 37. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. New: airframe-control: Retry ● Everything can fail … ● Network disconnection ● Servere crash ● ... ● Retry ● Exponential backoff ■ 2x, 4x, ... ● Jittering ■ 1 sec., 2 * rand, 4 * rand, … ● Customize error type classifiers ● retryable failures ● non-retryable failures 37リトライ処理をパターン化

  38. 38. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. New: airframe-control: Parallel ● Running tasks in parallel ● Customize parallelism ● Fixes slow tail issue of parallel collection 38
  39. 39. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. Airframe 19 Milestones ● Airframe 19 Milestones ● https://github.com/wvlet/airframe/issues/528 ● airframe-http ● Under active development for production use ■ Request filters ■ Request logging ■ JMX metrics ■ OpenAPI spec generator (no code generation is planned) 39
  40. 40. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. airframe-http ● Created a simple HTTP framework ● Based on Airframe modules: ■ airframe-surface ■ airframe-codec ■ airframe-msgpack ■ etc. ● Blog ● Building Low-Friction Web Service Over Finagle ● Save the time for choosing a web framework: ● Many frameworks exist: ● e.g, Finatra, Finch, akka-http, spring, RESTeasy, open-api, swagger, etc. 40Airframe資産を活用して、Webフレームワークも手軽に作成

  41. 41. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. airframe-http: Scala as IDL ● IDL: Interface Definition Language ● e.g. ProtocolBuffers, gRPC ● Use Scala’s Standard Functionalities ● case classes ■ Define HTTP request/response data structure ● functions ■ Define HTTP endpoints ■ Use @Endpoint annotation for for URL to method mapping ● Router ● Router.add[X] ● filter andThen router (available in the next release) 41
  42. 42. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. Path Mapping ● :param ● Mapping to a function argument ● /user/:id ■ /user/1 ■ def getUser(id:Int) ○ getUser(1) ● *param ● Mapping url path tail to a function argument ● /resource/*path ■ /resource/my/key/path ■ def getPath(path:String) ○ getPath(my/key/path) ● case classes ● request/response body ● JSON <-> MessagePack <-> Object 42
  43. 43. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. airframe-http-client ● Read server responses as target object types ● Accept: application/json (default) ● Accept: application/x-msgpack (Receive data in MessagePack format) 43
  44. 44. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Retry Patterns for HTTP requests ● Error handling of HTTP requests is difficult ● 4xx, 5xx status code ● Should we retry the request? ■ IOException, EOFException ■ TimeoutException ■ InterruptedException ■ SSLException ■ InvocationTargetException ● HTTP client ● request retries ● response mapping ■ JSON, MessagePack format ● airframe-codec 44間違いやすいHTTPリクエストのエラーハンドリングをライブラリ化

  45. 45. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. RouteMatcher ● Router ● List of @Endpoint definitions ● Graph ● Node ■ Path fragment ● Action ■ Put variable mappings for :param or *param ● Edge ■ Node -> Token (path fragment) -> Node ● Input: ● URL request path fragments ● Algorithm ● Build NFA (Non-deterministic Finite Automaton) ● Convert NFA into DFA (deterministic finite automaton) ● Traverse the automaton ● Call the matched function (MethodSurface) ■ buildControllerMethodArgs (Route.scala) 45
  46. 46. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. Http Request/Response Adapters ● Finagle Adapters ● HttpRequest[finagle.Request] ● Upcoming Adapters ● jetty, okhttp 46
  47. 47. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. airframe-http-recorder ● Testing against actual web services is time consuming ● Record & Replay HTTP responses ● Reproducible results ● Runnable on small machines (e.g., Travis CI) 47HTTPリクエストをレコーディングして、Webサービスのテストを効率化する
 HTTP Request HTTP Recorder Request Real Web Service Recording Mode: Response HTTP Request HTTP Recorder Replay Mode: Request Response Recording Responses Request Recorded Responses
  48. 48. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. [WIP] Scala.js RPC ● Scala.js ● Compiling Scala code into JavaScript for Web Browsers ● airframe-codec: Passing model class data between Scala and Scala.js UserInfo MessagePack UserInfo Pack Unpack PackUnpack Scala Server Side Scala.js Client Side XML RPC 48airframe-codecはScala.js(ブラウザ側)とのデータ受け渡しにも使える

  49. 49. Copyright 1995-2018 Arm Limited (or its affiliates). All rights reserved. Demo: Live Coding ● Build a simple web server ● JSON messaging support ● build.sbt ● project/plugins.sbt ■ sbt-pack, sbt-revolver ● Command line program launcher ● airframe-launcher ● packaging ● sbt-pack ● Docker ● Add web APIs 49
  50. 50. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. Scala At Arm Treasure Data ● Scala is now an official language at Arm Treasure Data ● 0 -> 10+ engineers who can write Scala ● Use cases are growing: ● Query optimization, API, Spark, data analysis, storage systems, service operation, etc. ● We are happy to share our Scala assets through Airframe! 50 Airframe Scalaエンジニアが充実してきたArm Treasure Data。Scalaの適用範囲も広がっている

  51. 51. Copyright 1995-2019 Arm Limited (or its affiliates). All rights reserved. Presto Conference Tokyo 2019 ● July 11 (Thu), 2019, 13:30 ~ (Free) ● https://techplay.jp/event/733772 ● Inviting Presto Creators (Martin, Dain, David) ● Presto Software Foundation ● Talks from big Presto users in Japan ● Yahoo! JAPAN, LINE, Arm Treasure Data ● Presto Source Code Navigation 51 Presto Conference Tokyo 2019を7/11(木) 13:30~より開催 (参加無料)

  52. 52. Confidential © Arm 2017Confidential © Arm 2017Confidential © Arm 2017 Thank You! Danke! Merci! 谢谢! ありがとう! Gracias! Kiitos! 52
  • yoskhdia

    Jul. 10, 2019

Airframe HTTP enables using Scala as IDL for building web applications The live coding demo project can be found from here https://github.com/xerial/airframe-http-demo

Vues

Nombre de vues

676

Sur Slideshare

0

À partir des intégrations

0

Nombre d'intégrations

55

Actions

Téléchargements

0

Partages

0

Commentaires

0

Mentions J'aime

1

×