SlideShare une entreprise Scribd logo
1  sur  41
Erlang
Tightly coupled Erlang

Yatce: Yet Another TC-Erlang binding

     4th Tokyo Erlang Workshop
        in Aoyama, 2010/2/26
              @kuenishi
           http://creativecommons.org/licenses/by-sa/2.1/jp/
@kuenishi

http://www.google.com/profiles/kuenishi

           : an employee

Distributed System Developer, C/C++



   OCaml    …
Why YATCE?
Erlang
I/O
mnesia
dets
Tew4 Yatce presentation
Tew4 Yatce presentation
Tew4 Yatce presentation
TokyoCabinet
TokyoCabinet   Erlang


    Erlang
Erlang   C
C node
C node
     A process behaves like an Erlang node,

     BERT/TCP connected,

     Failure doesn’t involve the other.


main(){
   ...                 TCP           Erlang
        C node
  ...                                 node
}
Port
Driver
Port Driver
==Popen3, just an IPC,

BERT/stdin/stdout connected,

tolerant for Child process’ failure.

           fork/exec

              stdin        Erlang
any CUI
                          process
             stdout
tcerl

Pioneer in TC-Erlang!

Port driver,

“I couldn’t make it work”, - matthew,

seems not being maintained.
Linkedin
 Driver
Linked-in Driver
in-process erlang-port,

  % acts like an Erlang process

  % Message can be sent/recv’d,

less failure-separeted.
                              Erlang
                             process

                           output(..){
     ?DEMO.                 ... }
message passing
                                       to port

                            Process

             Process
                                 Process
     ! or
port_command/3      ! or
                   return


             port


          socket
Port!{self(), {command, Data}}
Process
                   or port_command/2


                   port


                    erl_drv_entry#output(...)



           ! or
          return



                     driver_output(...)
everything wrapped

OS
driver_alloc, driver_free

driver_enq, driver_deq

driver_monitor_process

erl_driver_select

erl_drv_thread_create, erl_drv_thread_join, erl_drv_mutex_lock, ...

     s/erl_drv_/p/g

driver_set_timer, driver_cancel_timer, driver_read_timer




R13B04          API
Toke

Most famous in TC-Erlang,

Linked-in driver,

Simple: just what matthew/LShift wants,

to be used in RabbitMQ?.
Yatce

Made in Japan by me,

Linked-in driver (0.10)

port_control instead of port_command

interface based on tokyocabinet.idl,

http://bitbucket.org/kuenishi/yatce
Yatce 0.10
                                                                                                                                     R13B02
                                        response time vs thruput for each key-value store (with my Dual-core SSD’d MacBook)
                             3000




                             2250
average response time (us)




                             1500




                             750




                               0
                                    0                    1750                   3500                  5250                    7000
                                                                            Thruput (qps)

                                             ets            dets            yatce             tcerl
one more
problem...
port
locking
Python   GIL


Ruby   GVL


       driver
Port!{self(), {command, Data}}
Process
                   or port_command/2


                   port


                    erl_drv_entry#output(...)



           ! or
          return



                     driver_output(...)
ERTS
                 Worker
Worker


          Worker




         tcadb
TC
driver        port   port level locking
  TC     DB      port
                                          …
R13B03
Native Interface
     Functions

Since R13B03, Beta,

Built-in code runs inside ERTS,

easy way to write Erlang module in C,

Nothing separated.
it’s just a function pointer
             calling inside ERTS


                         nif_func_in_c(...){


call_nif()->
  nif_func_in_c(..),
  hoge,
  done.

                            return term;
                        }

     Erlang                       C
ERTS




Thread   Thread   Thread




         tcadb
Yatce 0.20
                                                                                                                        R13B03
                               response time vs thruput for each key-value store (with my Dual-core SSD’d MacBook)
                     700




                     525
Response time (us)




                     350




                     175




                      0
                           0                     5000                  10000                15000               20000
                                                                 Throughput (qps)
                                                ets             dets                yatce
NIF


                R13B04
           erlang-patches        erlang-questions   GitHub   erlang/otp
          NIF    Erlang VM                                          IO
                   linkedin driver                      I/O
       pthread     driver_select                                  Yatce
0.20                                           TC
                                   TC


           NIF     Yatce                Erlang VM
Choose what fits to
        you!
Slow   C node
                          Safe

       Port driver

       Linked-in driver

         port_command/3

         port_control/3

Fast   NIF                Difficult
Any Questions?
          or
come on to the party
     and ask me!
References
ERTS: Interoperability tutorial guide
      http://www.erlang.org/doc/tutorial/overview.html

BERT/BERT-RPC
      http://github.com/blog/531-introducing-bert-and-bert-rpc

      http://bert-rpc.org/

Erlang external term format
      http://erlang.org/doc/apps/erts/erl_ext_dist.html

NIF
      http://erlang.org/doc/man/erl_nif.html

Toke, vs tcerl
      http://bit.ly/8nPhm2

Contenu connexe

Tendances

Lifting The Veil - Reading Java Bytecode
Lifting The Veil - Reading Java BytecodeLifting The Veil - Reading Java Bytecode
Lifting The Veil - Reading Java BytecodeAlexander Shopov
 
Jython: Integrating Python and Java
Jython: Integrating Python and JavaJython: Integrating Python and Java
Jython: Integrating Python and JavaCharles Anderson
 
You got ur Erlang in my Ruby
You got ur Erlang in my RubyYou got ur Erlang in my Ruby
You got ur Erlang in my Rubyelliando dias
 
A(n abridged) tour of the Rust compiler [PDX-Rust March 2014]
A(n abridged) tour of the Rust compiler [PDX-Rust March 2014]A(n abridged) tour of the Rust compiler [PDX-Rust March 2014]
A(n abridged) tour of the Rust compiler [PDX-Rust March 2014]Tom Lee
 
Parrot -- "one bytecode to rule them all"
Parrot -- "one bytecode to rule them all"Parrot -- "one bytecode to rule them all"
Parrot -- "one bytecode to rule them all"Nuno Carvalho
 
An introduction to erlang
An introduction to erlangAn introduction to erlang
An introduction to erlangMirko Bonadei
 
FregeDay: Design and Implementation of the language (Ingo Wechsung)
FregeDay: Design and Implementation of the language (Ingo Wechsung)FregeDay: Design and Implementation of the language (Ingo Wechsung)
FregeDay: Design and Implementation of the language (Ingo Wechsung)Dierk König
 
Aac boolean formulae
Aac   boolean formulaeAac   boolean formulae
Aac boolean formulaemarangburu42
 
Python coroutine
Python coroutinePython coroutine
Python coroutine경섭 심
 
The dark side of Ruby, or Learn functional programming with Ruby
The dark side of Ruby, or Learn functional programming with RubyThe dark side of Ruby, or Learn functional programming with Ruby
The dark side of Ruby, or Learn functional programming with RubyEvgeny Garlukovich
 
Static analysis for perl
Static analysis for perlStatic analysis for perl
Static analysis for perlmoznion
 
Kostis Sagonas: Cool Tools for Modern Erlang Program Developmen
Kostis Sagonas: Cool Tools for Modern Erlang Program DevelopmenKostis Sagonas: Cool Tools for Modern Erlang Program Developmen
Kostis Sagonas: Cool Tools for Modern Erlang Program DevelopmenKonstantin Sorokin
 
Mixing Source and Bytecode: A Case for Compilation By Normalization (OOPSLA 2...
Mixing Source and Bytecode: A Case for Compilation By Normalization (OOPSLA 2...Mixing Source and Bytecode: A Case for Compilation By Normalization (OOPSLA 2...
Mixing Source and Bytecode: A Case for Compilation By Normalization (OOPSLA 2...lennartkats
 
FregeDay: Roadmap for resolving differences between Haskell and Frege (Ingo W...
FregeDay: Roadmap for resolving differences between Haskell and Frege (Ingo W...FregeDay: Roadmap for resolving differences between Haskell and Frege (Ingo W...
FregeDay: Roadmap for resolving differences between Haskell and Frege (Ingo W...Dierk König
 
Programmable Logic Devices Plds
Programmable Logic Devices PldsProgrammable Logic Devices Plds
Programmable Logic Devices PldsGaditek
 

Tendances (20)

Lifting The Veil - Reading Java Bytecode
Lifting The Veil - Reading Java BytecodeLifting The Veil - Reading Java Bytecode
Lifting The Veil - Reading Java Bytecode
 
Jython: Integrating Python and Java
Jython: Integrating Python and JavaJython: Integrating Python and Java
Jython: Integrating Python and Java
 
You got ur Erlang in my Ruby
You got ur Erlang in my RubyYou got ur Erlang in my Ruby
You got ur Erlang in my Ruby
 
A(n abridged) tour of the Rust compiler [PDX-Rust March 2014]
A(n abridged) tour of the Rust compiler [PDX-Rust March 2014]A(n abridged) tour of the Rust compiler [PDX-Rust March 2014]
A(n abridged) tour of the Rust compiler [PDX-Rust March 2014]
 
Flash memory
Flash memoryFlash memory
Flash memory
 
Parrot -- "one bytecode to rule them all"
Parrot -- "one bytecode to rule them all"Parrot -- "one bytecode to rule them all"
Parrot -- "one bytecode to rule them all"
 
An introduction to erlang
An introduction to erlangAn introduction to erlang
An introduction to erlang
 
Parallel Programming
Parallel ProgrammingParallel Programming
Parallel Programming
 
FregeDay: Design and Implementation of the language (Ingo Wechsung)
FregeDay: Design and Implementation of the language (Ingo Wechsung)FregeDay: Design and Implementation of the language (Ingo Wechsung)
FregeDay: Design and Implementation of the language (Ingo Wechsung)
 
Aac boolean formulae
Aac   boolean formulaeAac   boolean formulae
Aac boolean formulae
 
Python coroutine
Python coroutinePython coroutine
Python coroutine
 
The dark side of Ruby, or Learn functional programming with Ruby
The dark side of Ruby, or Learn functional programming with RubyThe dark side of Ruby, or Learn functional programming with Ruby
The dark side of Ruby, or Learn functional programming with Ruby
 
Static analysis for perl
Static analysis for perlStatic analysis for perl
Static analysis for perl
 
Kostis Sagonas: Cool Tools for Modern Erlang Program Developmen
Kostis Sagonas: Cool Tools for Modern Erlang Program DevelopmenKostis Sagonas: Cool Tools for Modern Erlang Program Developmen
Kostis Sagonas: Cool Tools for Modern Erlang Program Developmen
 
Mixing Source and Bytecode: A Case for Compilation By Normalization (OOPSLA 2...
Mixing Source and Bytecode: A Case for Compilation By Normalization (OOPSLA 2...Mixing Source and Bytecode: A Case for Compilation By Normalization (OOPSLA 2...
Mixing Source and Bytecode: A Case for Compilation By Normalization (OOPSLA 2...
 
Whats new in Java 7
Whats new in Java 7Whats new in Java 7
Whats new in Java 7
 
FregeDay: Roadmap for resolving differences between Haskell and Frege (Ingo W...
FregeDay: Roadmap for resolving differences between Haskell and Frege (Ingo W...FregeDay: Roadmap for resolving differences between Haskell and Frege (Ingo W...
FregeDay: Roadmap for resolving differences between Haskell and Frege (Ingo W...
 
PLDs
PLDsPLDs
PLDs
 
Fpga
FpgaFpga
Fpga
 
Programmable Logic Devices Plds
Programmable Logic Devices PldsProgrammable Logic Devices Plds
Programmable Logic Devices Plds
 

En vedette

LT on Python Hackathon #2
LT on Python Hackathon #2LT on Python Hackathon #2
LT on Python Hackathon #2UENISHI Kota
 
[db tech showcase Tokyo 2015] B34:データの仮想化を具体化するIBMのロジカルデータウェアハウス by 日本アイ・ビー・エ...
[db tech showcase Tokyo 2015] B34:データの仮想化を具体化するIBMのロジカルデータウェアハウス by 日本アイ・ビー・エ...[db tech showcase Tokyo 2015] B34:データの仮想化を具体化するIBMのロジカルデータウェアハウス by 日本アイ・ビー・エ...
[db tech showcase Tokyo 2015] B34:データの仮想化を具体化するIBMのロジカルデータウェアハウス by 日本アイ・ビー・エ...Insight Technology, Inc.
 
[db tech showcase Tokyo 2015] C14:30万のユーザ部門を抱える日立、情シスの「理想と現実」 by 株式会社日立製作所 情報...
[db tech showcase Tokyo 2015] C14:30万のユーザ部門を抱える日立、情シスの「理想と現実」 by 株式会社日立製作所 情報...[db tech showcase Tokyo 2015] C14:30万のユーザ部門を抱える日立、情シスの「理想と現実」 by 株式会社日立製作所 情報...
[db tech showcase Tokyo 2015] C14:30万のユーザ部門を抱える日立、情シスの「理想と現実」 by 株式会社日立製作所 情報...Insight Technology, Inc.
 
Couchbase introduction-20150611
Couchbase introduction-20150611Couchbase introduction-20150611
Couchbase introduction-20150611Couchbase Japan KK
 
Mongodb x business
Mongodb x businessMongodb x business
Mongodb x businessemin_press
 
Db tech showcase2015
Db tech showcase2015Db tech showcase2015
Db tech showcase2015emin_press
 
[db tech showcase Tokyo 2015] D32:HPの全方位インメモリDB化に向けた取り組みとSAP HANAインメモリDB の効果を...
[db tech showcase Tokyo 2015] D32:HPの全方位インメモリDB化に向けた取り組みとSAP HANAインメモリDB の効果を...[db tech showcase Tokyo 2015] D32:HPの全方位インメモリDB化に向けた取り組みとSAP HANAインメモリDB の効果を...
[db tech showcase Tokyo 2015] D32:HPの全方位インメモリDB化に向けた取り組みとSAP HANAインメモリDB の効果を...Insight Technology, Inc.
 
Db tech show - hivemall
Db tech show - hivemallDb tech show - hivemall
Db tech show - hivemallMakoto Yui
 
[db tech showcase Tokyo 2015] E26 Couchbaseの最新情報/JBoss Data Virtualizationで仮想...
[db tech showcase Tokyo 2015] E26 Couchbaseの最新情報/JBoss Data Virtualizationで仮想...[db tech showcase Tokyo 2015] E26 Couchbaseの最新情報/JBoss Data Virtualizationで仮想...
[db tech showcase Tokyo 2015] E26 Couchbaseの最新情報/JBoss Data Virtualizationで仮想...Masahiro Tomisugi
 
[db tech showcase Tokyo 2015] D33:Superdome X 上の SQL Server 2014 OLTP 検証結果と S...
[db tech showcase Tokyo 2015] D33:Superdome X 上の SQL Server 2014 OLTP 検証結果と S...[db tech showcase Tokyo 2015] D33:Superdome X 上の SQL Server 2014 OLTP 検証結果と S...
[db tech showcase Tokyo 2015] D33:Superdome X 上の SQL Server 2014 OLTP 検証結果と S...Insight Technology, Inc.
 
[db tech showcase Tokyo 2015] D16:マイケルストーンブレーカー発の超高速データベースで実現する分析基盤の簡単構築・運用ステ...
[db tech showcase Tokyo 2015] D16:マイケルストーンブレーカー発の超高速データベースで実現する分析基盤の簡単構築・運用ステ...[db tech showcase Tokyo 2015] D16:マイケルストーンブレーカー発の超高速データベースで実現する分析基盤の簡単構築・運用ステ...
[db tech showcase Tokyo 2015] D16:マイケルストーンブレーカー発の超高速データベースで実現する分析基盤の簡単構築・運用ステ...Insight Technology, Inc.
 
[db tech showcase Tokyo 2015] C15:DevOps MySQL in カカクコム~ OSSによる可用性担保とリアルタイムパフ...
[db tech showcase Tokyo 2015] C15:DevOps MySQL in カカクコム~ OSSによる可用性担保とリアルタイムパフ...[db tech showcase Tokyo 2015] C15:DevOps MySQL in カカクコム~ OSSによる可用性担保とリアルタイムパフ...
[db tech showcase Tokyo 2015] C15:DevOps MySQL in カカクコム~ OSSによる可用性担保とリアルタイムパフ...Insight Technology, Inc.
 
DBTS2015 Tokyo DBAが知っておくべき最新テクノロジー
DBTS2015 Tokyo DBAが知っておくべき最新テクノロジーDBTS2015 Tokyo DBAが知っておくべき最新テクノロジー
DBTS2015 Tokyo DBAが知っておくべき最新テクノロジーMasaya Ishikawa
 
[DB tech showcase Tokyo 2015] B37 :オンプレミスからAWS上のSAP HANAまで高信頼DBシステム構築にHAクラスタリ...
[DB tech showcase Tokyo 2015] B37 :オンプレミスからAWS上のSAP HANAまで高信頼DBシステム構築にHAクラスタリ...[DB tech showcase Tokyo 2015] B37 :オンプレミスからAWS上のSAP HANAまで高信頼DBシステム構築にHAクラスタリ...
[DB tech showcase Tokyo 2015] B37 :オンプレミスからAWS上のSAP HANAまで高信頼DBシステム構築にHAクラスタリ...Funada Yasunobu
 
[db tech showcase Tokyo 2015] C27:楽天MySQL Backup Structure by 楽天株式会社 粟田啓介
[db tech showcase Tokyo 2015] C27:楽天MySQL Backup Structure by 楽天株式会社 粟田啓介[db tech showcase Tokyo 2015] C27:楽天MySQL Backup Structure by 楽天株式会社 粟田啓介
[db tech showcase Tokyo 2015] C27:楽天MySQL Backup Structure by 楽天株式会社 粟田啓介Insight Technology, Inc.
 
[db tech showcase Tokyo 2015] C33:ビッグデータ・IoT時代のキーテクノロジー、CEPの「今」を掴む! by 株式会社日立...
[db tech showcase Tokyo 2015] C33:ビッグデータ・IoT時代のキーテクノロジー、CEPの「今」を掴む! by 株式会社日立...[db tech showcase Tokyo 2015] C33:ビッグデータ・IoT時代のキーテクノロジー、CEPの「今」を掴む! by 株式会社日立...
[db tech showcase Tokyo 2015] C33:ビッグデータ・IoT時代のキーテクノロジー、CEPの「今」を掴む! by 株式会社日立...Insight Technology, Inc.
 
[db tech showcase Tokyo 2015] B36:Hitachi Advanced Data Binder 実践SQLチューニング方法 ...
[db tech showcase Tokyo 2015] B36:Hitachi Advanced Data Binder 実践SQLチューニング方法 ...[db tech showcase Tokyo 2015] B36:Hitachi Advanced Data Binder 実践SQLチューニング方法 ...
[db tech showcase Tokyo 2015] B36:Hitachi Advanced Data Binder 実践SQLチューニング方法 ...Insight Technology, Inc.
 
[db tech showcase Tokyo 2015] A33:Amazon DynamoDB Deep Dive by アマゾン データ サービス ...
[db tech showcase Tokyo 2015] A33:Amazon DynamoDB Deep Dive by アマゾン データ サービス ...[db tech showcase Tokyo 2015] A33:Amazon DynamoDB Deep Dive by アマゾン データ サービス ...
[db tech showcase Tokyo 2015] A33:Amazon DynamoDB Deep Dive by アマゾン データ サービス ...Insight Technology, Inc.
 
[db tech showcase Tokyo 2015] D22:インメモリープラットホームSAP HANAのご紹介と最新情報 by SAPジャパン株式...
[db tech showcase Tokyo 2015] D22:インメモリープラットホームSAP HANAのご紹介と最新情報 by SAPジャパン株式...[db tech showcase Tokyo 2015] D22:インメモリープラットホームSAP HANAのご紹介と最新情報 by SAPジャパン株式...
[db tech showcase Tokyo 2015] D22:インメモリープラットホームSAP HANAのご紹介と最新情報 by SAPジャパン株式...Insight Technology, Inc.
 

En vedette (20)

LT on Python Hackathon #2
LT on Python Hackathon #2LT on Python Hackathon #2
LT on Python Hackathon #2
 
[db tech showcase Tokyo 2015] B34:データの仮想化を具体化するIBMのロジカルデータウェアハウス by 日本アイ・ビー・エ...
[db tech showcase Tokyo 2015] B34:データの仮想化を具体化するIBMのロジカルデータウェアハウス by 日本アイ・ビー・エ...[db tech showcase Tokyo 2015] B34:データの仮想化を具体化するIBMのロジカルデータウェアハウス by 日本アイ・ビー・エ...
[db tech showcase Tokyo 2015] B34:データの仮想化を具体化するIBMのロジカルデータウェアハウス by 日本アイ・ビー・エ...
 
[db tech showcase Tokyo 2015] C14:30万のユーザ部門を抱える日立、情シスの「理想と現実」 by 株式会社日立製作所 情報...
[db tech showcase Tokyo 2015] C14:30万のユーザ部門を抱える日立、情シスの「理想と現実」 by 株式会社日立製作所 情報...[db tech showcase Tokyo 2015] C14:30万のユーザ部門を抱える日立、情シスの「理想と現実」 by 株式会社日立製作所 情報...
[db tech showcase Tokyo 2015] C14:30万のユーザ部門を抱える日立、情シスの「理想と現実」 by 株式会社日立製作所 情報...
 
Couchbase introduction-20150611
Couchbase introduction-20150611Couchbase introduction-20150611
Couchbase introduction-20150611
 
Mongodb x business
Mongodb x businessMongodb x business
Mongodb x business
 
Db tech showcase2015
Db tech showcase2015Db tech showcase2015
Db tech showcase2015
 
[db tech showcase Tokyo 2015] D32:HPの全方位インメモリDB化に向けた取り組みとSAP HANAインメモリDB の効果を...
[db tech showcase Tokyo 2015] D32:HPの全方位インメモリDB化に向けた取り組みとSAP HANAインメモリDB の効果を...[db tech showcase Tokyo 2015] D32:HPの全方位インメモリDB化に向けた取り組みとSAP HANAインメモリDB の効果を...
[db tech showcase Tokyo 2015] D32:HPの全方位インメモリDB化に向けた取り組みとSAP HANAインメモリDB の効果を...
 
Db tech show - hivemall
Db tech show - hivemallDb tech show - hivemall
Db tech show - hivemall
 
[db tech showcase Tokyo 2015] E26 Couchbaseの最新情報/JBoss Data Virtualizationで仮想...
[db tech showcase Tokyo 2015] E26 Couchbaseの最新情報/JBoss Data Virtualizationで仮想...[db tech showcase Tokyo 2015] E26 Couchbaseの最新情報/JBoss Data Virtualizationで仮想...
[db tech showcase Tokyo 2015] E26 Couchbaseの最新情報/JBoss Data Virtualizationで仮想...
 
[db tech showcase Tokyo 2015] D33:Superdome X 上の SQL Server 2014 OLTP 検証結果と S...
[db tech showcase Tokyo 2015] D33:Superdome X 上の SQL Server 2014 OLTP 検証結果と S...[db tech showcase Tokyo 2015] D33:Superdome X 上の SQL Server 2014 OLTP 検証結果と S...
[db tech showcase Tokyo 2015] D33:Superdome X 上の SQL Server 2014 OLTP 検証結果と S...
 
[db tech showcase Tokyo 2015] D16:マイケルストーンブレーカー発の超高速データベースで実現する分析基盤の簡単構築・運用ステ...
[db tech showcase Tokyo 2015] D16:マイケルストーンブレーカー発の超高速データベースで実現する分析基盤の簡単構築・運用ステ...[db tech showcase Tokyo 2015] D16:マイケルストーンブレーカー発の超高速データベースで実現する分析基盤の簡単構築・運用ステ...
[db tech showcase Tokyo 2015] D16:マイケルストーンブレーカー発の超高速データベースで実現する分析基盤の簡単構築・運用ステ...
 
[db tech showcase Tokyo 2015] C15:DevOps MySQL in カカクコム~ OSSによる可用性担保とリアルタイムパフ...
[db tech showcase Tokyo 2015] C15:DevOps MySQL in カカクコム~ OSSによる可用性担保とリアルタイムパフ...[db tech showcase Tokyo 2015] C15:DevOps MySQL in カカクコム~ OSSによる可用性担保とリアルタイムパフ...
[db tech showcase Tokyo 2015] C15:DevOps MySQL in カカクコム~ OSSによる可用性担保とリアルタイムパフ...
 
gen_paxos
gen_paxosgen_paxos
gen_paxos
 
DBTS2015 Tokyo DBAが知っておくべき最新テクノロジー
DBTS2015 Tokyo DBAが知っておくべき最新テクノロジーDBTS2015 Tokyo DBAが知っておくべき最新テクノロジー
DBTS2015 Tokyo DBAが知っておくべき最新テクノロジー
 
[DB tech showcase Tokyo 2015] B37 :オンプレミスからAWS上のSAP HANAまで高信頼DBシステム構築にHAクラスタリ...
[DB tech showcase Tokyo 2015] B37 :オンプレミスからAWS上のSAP HANAまで高信頼DBシステム構築にHAクラスタリ...[DB tech showcase Tokyo 2015] B37 :オンプレミスからAWS上のSAP HANAまで高信頼DBシステム構築にHAクラスタリ...
[DB tech showcase Tokyo 2015] B37 :オンプレミスからAWS上のSAP HANAまで高信頼DBシステム構築にHAクラスタリ...
 
[db tech showcase Tokyo 2015] C27:楽天MySQL Backup Structure by 楽天株式会社 粟田啓介
[db tech showcase Tokyo 2015] C27:楽天MySQL Backup Structure by 楽天株式会社 粟田啓介[db tech showcase Tokyo 2015] C27:楽天MySQL Backup Structure by 楽天株式会社 粟田啓介
[db tech showcase Tokyo 2015] C27:楽天MySQL Backup Structure by 楽天株式会社 粟田啓介
 
[db tech showcase Tokyo 2015] C33:ビッグデータ・IoT時代のキーテクノロジー、CEPの「今」を掴む! by 株式会社日立...
[db tech showcase Tokyo 2015] C33:ビッグデータ・IoT時代のキーテクノロジー、CEPの「今」を掴む! by 株式会社日立...[db tech showcase Tokyo 2015] C33:ビッグデータ・IoT時代のキーテクノロジー、CEPの「今」を掴む! by 株式会社日立...
[db tech showcase Tokyo 2015] C33:ビッグデータ・IoT時代のキーテクノロジー、CEPの「今」を掴む! by 株式会社日立...
 
[db tech showcase Tokyo 2015] B36:Hitachi Advanced Data Binder 実践SQLチューニング方法 ...
[db tech showcase Tokyo 2015] B36:Hitachi Advanced Data Binder 実践SQLチューニング方法 ...[db tech showcase Tokyo 2015] B36:Hitachi Advanced Data Binder 実践SQLチューニング方法 ...
[db tech showcase Tokyo 2015] B36:Hitachi Advanced Data Binder 実践SQLチューニング方法 ...
 
[db tech showcase Tokyo 2015] A33:Amazon DynamoDB Deep Dive by アマゾン データ サービス ...
[db tech showcase Tokyo 2015] A33:Amazon DynamoDB Deep Dive by アマゾン データ サービス ...[db tech showcase Tokyo 2015] A33:Amazon DynamoDB Deep Dive by アマゾン データ サービス ...
[db tech showcase Tokyo 2015] A33:Amazon DynamoDB Deep Dive by アマゾン データ サービス ...
 
[db tech showcase Tokyo 2015] D22:インメモリープラットホームSAP HANAのご紹介と最新情報 by SAPジャパン株式...
[db tech showcase Tokyo 2015] D22:インメモリープラットホームSAP HANAのご紹介と最新情報 by SAPジャパン株式...[db tech showcase Tokyo 2015] D22:インメモリープラットホームSAP HANAのご紹介と最新情報 by SAPジャパン株式...
[db tech showcase Tokyo 2015] D22:インメモリープラットホームSAP HANAのご紹介と最新情報 by SAPジャパン株式...
 

Similaire à Tew4 Yatce presentation

FPGA based 10G Performance Tester for HW OpenFlow Switch
FPGA based 10G Performance Tester for HW OpenFlow SwitchFPGA based 10G Performance Tester for HW OpenFlow Switch
FPGA based 10G Performance Tester for HW OpenFlow SwitchYutaka Yasuda
 
20081114 Friday Food iLabt Bart Joris
20081114 Friday Food iLabt Bart Joris20081114 Friday Food iLabt Bart Joris
20081114 Friday Food iLabt Bart Jorisimec.archive
 
QEMU - Binary Translation
QEMU - Binary Translation QEMU - Binary Translation
QEMU - Binary Translation Jiann-Fuh Liaw
 
Crushing Latency with Vert.x
Crushing Latency with Vert.xCrushing Latency with Vert.x
Crushing Latency with Vert.xPaulo Lopes
 
General Purpose Computing using Graphics Hardware
General Purpose Computing using Graphics HardwareGeneral Purpose Computing using Graphics Hardware
General Purpose Computing using Graphics HardwareDaniel Blezek
 
May2010 hex-core-opt
May2010 hex-core-optMay2010 hex-core-opt
May2010 hex-core-optJeff Larkin
 
Servers with Event Machine - David Troy - RailsConf 2011
Servers with Event Machine - David Troy - RailsConf 2011Servers with Event Machine - David Troy - RailsConf 2011
Servers with Event Machine - David Troy - RailsConf 2011David Troy
 
Os Worthington
Os WorthingtonOs Worthington
Os Worthingtonoscon2007
 
Unmanaged Parallelization via P/Invoke
Unmanaged Parallelization via P/InvokeUnmanaged Parallelization via P/Invoke
Unmanaged Parallelization via P/InvokeDmitri Nesteruk
 
Performance Tuning EC2 Instances
Performance Tuning EC2 InstancesPerformance Tuning EC2 Instances
Performance Tuning EC2 InstancesBrendan Gregg
 
EMBEDDED SYSTEMS 4&5
EMBEDDED SYSTEMS 4&5EMBEDDED SYSTEMS 4&5
EMBEDDED SYSTEMS 4&5PRADEEP
 
Vc4c development of opencl compiler for videocore4
Vc4c  development of opencl compiler for videocore4Vc4c  development of opencl compiler for videocore4
Vc4c development of opencl compiler for videocore4nomaddo
 
Bugs from Outer Space | while42 SF #6
Bugs from Outer Space | while42 SF #6Bugs from Outer Space | while42 SF #6
Bugs from Outer Space | while42 SF #6While42
 
DPDK & Layer 4 Packet Processing
DPDK & Layer 4 Packet ProcessingDPDK & Layer 4 Packet Processing
DPDK & Layer 4 Packet ProcessingMichelle Holley
 
Cray XT Porting, Scaling, and Optimization Best Practices
Cray XT Porting, Scaling, and Optimization Best PracticesCray XT Porting, Scaling, and Optimization Best Practices
Cray XT Porting, Scaling, and Optimization Best PracticesJeff Larkin
 
Reversing & Malware Analysis Training Part 4 - Assembly Programming Basics
Reversing & Malware Analysis Training Part 4 - Assembly Programming BasicsReversing & Malware Analysis Training Part 4 - Assembly Programming Basics
Reversing & Malware Analysis Training Part 4 - Assembly Programming Basicssecurityxploded
 
Revelation pyconuk2016
Revelation pyconuk2016Revelation pyconuk2016
Revelation pyconuk2016Sarah Mount
 
Network Programming: Data Plane Development Kit (DPDK)
Network Programming: Data Plane Development Kit (DPDK)Network Programming: Data Plane Development Kit (DPDK)
Network Programming: Data Plane Development Kit (DPDK)Andriy Berestovskyy
 
I3 multicore processor
I3 multicore processorI3 multicore processor
I3 multicore processorAmol Barewar
 

Similaire à Tew4 Yatce presentation (20)

FPGA based 10G Performance Tester for HW OpenFlow Switch
FPGA based 10G Performance Tester for HW OpenFlow SwitchFPGA based 10G Performance Tester for HW OpenFlow Switch
FPGA based 10G Performance Tester for HW OpenFlow Switch
 
20081114 Friday Food iLabt Bart Joris
20081114 Friday Food iLabt Bart Joris20081114 Friday Food iLabt Bart Joris
20081114 Friday Food iLabt Bart Joris
 
QEMU - Binary Translation
QEMU - Binary Translation QEMU - Binary Translation
QEMU - Binary Translation
 
Crushing Latency with Vert.x
Crushing Latency with Vert.xCrushing Latency with Vert.x
Crushing Latency with Vert.x
 
General Purpose Computing using Graphics Hardware
General Purpose Computing using Graphics HardwareGeneral Purpose Computing using Graphics Hardware
General Purpose Computing using Graphics Hardware
 
May2010 hex-core-opt
May2010 hex-core-optMay2010 hex-core-opt
May2010 hex-core-opt
 
Servers with Event Machine - David Troy - RailsConf 2011
Servers with Event Machine - David Troy - RailsConf 2011Servers with Event Machine - David Troy - RailsConf 2011
Servers with Event Machine - David Troy - RailsConf 2011
 
Os Worthington
Os WorthingtonOs Worthington
Os Worthington
 
Unmanaged Parallelization via P/Invoke
Unmanaged Parallelization via P/InvokeUnmanaged Parallelization via P/Invoke
Unmanaged Parallelization via P/Invoke
 
Performance Tuning EC2 Instances
Performance Tuning EC2 InstancesPerformance Tuning EC2 Instances
Performance Tuning EC2 Instances
 
EMBEDDED SYSTEMS 4&5
EMBEDDED SYSTEMS 4&5EMBEDDED SYSTEMS 4&5
EMBEDDED SYSTEMS 4&5
 
Vc4c development of opencl compiler for videocore4
Vc4c  development of opencl compiler for videocore4Vc4c  development of opencl compiler for videocore4
Vc4c development of opencl compiler for videocore4
 
Bugs from Outer Space | while42 SF #6
Bugs from Outer Space | while42 SF #6Bugs from Outer Space | while42 SF #6
Bugs from Outer Space | while42 SF #6
 
DPDK & Layer 4 Packet Processing
DPDK & Layer 4 Packet ProcessingDPDK & Layer 4 Packet Processing
DPDK & Layer 4 Packet Processing
 
Cray XT Porting, Scaling, and Optimization Best Practices
Cray XT Porting, Scaling, and Optimization Best PracticesCray XT Porting, Scaling, and Optimization Best Practices
Cray XT Porting, Scaling, and Optimization Best Practices
 
Reversing & Malware Analysis Training Part 4 - Assembly Programming Basics
Reversing & Malware Analysis Training Part 4 - Assembly Programming BasicsReversing & Malware Analysis Training Part 4 - Assembly Programming Basics
Reversing & Malware Analysis Training Part 4 - Assembly Programming Basics
 
Revelation pyconuk2016
Revelation pyconuk2016Revelation pyconuk2016
Revelation pyconuk2016
 
Network Programming: Data Plane Development Kit (DPDK)
Network Programming: Data Plane Development Kit (DPDK)Network Programming: Data Plane Development Kit (DPDK)
Network Programming: Data Plane Development Kit (DPDK)
 
I3 multicore processor
I3 multicore processorI3 multicore processor
I3 multicore processor
 
I3
I3I3
I3
 

Tew4 Yatce presentation

  • 1. Erlang Tightly coupled Erlang Yatce: Yet Another TC-Erlang binding 4th Tokyo Erlang Workshop in Aoyama, 2010/2/26 @kuenishi http://creativecommons.org/licenses/by-sa/2.1/jp/
  • 2. @kuenishi http://www.google.com/profiles/kuenishi : an employee Distributed System Developer, C/C++ OCaml …
  • 5. I/O
  • 11. TokyoCabinet Erlang Erlang
  • 12. Erlang C
  • 14. C node A process behaves like an Erlang node, BERT/TCP connected, Failure doesn’t involve the other. main(){ ... TCP Erlang C node ... node }
  • 16. Port Driver ==Popen3, just an IPC, BERT/stdin/stdout connected, tolerant for Child process’ failure. fork/exec stdin Erlang any CUI process stdout
  • 17. tcerl Pioneer in TC-Erlang! Port driver, “I couldn’t make it work”, - matthew, seems not being maintained.
  • 19. Linked-in Driver in-process erlang-port, % acts like an Erlang process % Message can be sent/recv’d, less failure-separeted. Erlang process output(..){ ?DEMO. ... }
  • 20. message passing to port Process Process Process ! or port_command/3 ! or return port socket
  • 21. Port!{self(), {command, Data}} Process or port_command/2 port erl_drv_entry#output(...) ! or return driver_output(...)
  • 22. everything wrapped OS driver_alloc, driver_free driver_enq, driver_deq driver_monitor_process erl_driver_select erl_drv_thread_create, erl_drv_thread_join, erl_drv_mutex_lock, ... s/erl_drv_/p/g driver_set_timer, driver_cancel_timer, driver_read_timer R13B04 API
  • 23. Toke Most famous in TC-Erlang, Linked-in driver, Simple: just what matthew/LShift wants, to be used in RabbitMQ?.
  • 24. Yatce Made in Japan by me, Linked-in driver (0.10) port_control instead of port_command interface based on tokyocabinet.idl, http://bitbucket.org/kuenishi/yatce
  • 25. Yatce 0.10 R13B02 response time vs thruput for each key-value store (with my Dual-core SSD’d MacBook) 3000 2250 average response time (us) 1500 750 0 0 1750 3500 5250 7000 Thruput (qps) ets dets yatce tcerl
  • 28. Python GIL Ruby GVL driver
  • 29. Port!{self(), {command, Data}} Process or port_command/2 port erl_drv_entry#output(...) ! or return driver_output(...)
  • 30. ERTS Worker Worker Worker tcadb
  • 31. TC
  • 32. driver port port level locking TC DB port …
  • 34. Native Interface Functions Since R13B03, Beta, Built-in code runs inside ERTS, easy way to write Erlang module in C, Nothing separated.
  • 35. it’s just a function pointer calling inside ERTS nif_func_in_c(...){ call_nif()-> nif_func_in_c(..), hoge, done. return term; } Erlang C
  • 36. ERTS Thread Thread Thread tcadb
  • 37. Yatce 0.20 R13B03 response time vs thruput for each key-value store (with my Dual-core SSD’d MacBook) 700 525 Response time (us) 350 175 0 0 5000 10000 15000 20000 Throughput (qps) ets dets yatce
  • 38. NIF R13B04 erlang-patches erlang-questions GitHub erlang/otp NIF Erlang VM IO linkedin driver I/O pthread driver_select Yatce 0.20 TC TC NIF Yatce Erlang VM
  • 39. Choose what fits to you! Slow C node Safe Port driver Linked-in driver port_command/3 port_control/3 Fast NIF Difficult
  • 40. Any Questions? or come on to the party and ask me!
  • 41. References ERTS: Interoperability tutorial guide http://www.erlang.org/doc/tutorial/overview.html BERT/BERT-RPC http://github.com/blog/531-introducing-bert-and-bert-rpc http://bert-rpc.org/ Erlang external term format http://erlang.org/doc/apps/erts/erl_ext_dist.html NIF http://erlang.org/doc/man/erl_nif.html Toke, vs tcerl http://bit.ly/8nPhm2

Notes de l'éditeur