Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

"fireap" - fast task runner on consul

1 999 vues

Publié le

"Fireap" is a consul-based rapid propagative task runner for large systems.
https://github.com/key-amb/fireap

Publié dans : Technologie
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

"fireap" - fast task runner on consul

  1. 1. “fireap” - fast task runner on Consul @key_amb In 渋谷.rb[:20160420] #shibuyarb
  2. 2. Who am I ? ❏ @key_amb ❏ Software Engineer ❏ Serverside, Web ❏ https://github.com/key-amb ❏ https://rubygems.org/profiles/key-amb
  3. 3. fireap
  4. 4. Overview of “fireap” ❏ https://github.com/key-amb/fireap ❏ Propagative task runner on Consul Cluster ❏ O(log N) … N = node number ❏ Consul key features used in “fireap”: ❏ Kv, Event, Watch ❏ And powered by diplomat, gem for Consul API ❏ Blog: ❏ Consul クラスタ上で動作する S3 非依存の pull 型デプロイツール "fireap" を作った
  5. 5. Related Products ❏ https://github.com/fujiwara/stretcher ❏ https://github.com/sorah/mamiya These above enable O(1) deployment. But both need storage with big I/O capacity (like S3) => difficult to use in private DC. Then, I came up with “fireap”.
  6. 6. How fast is “fireap”? ❏ Benchmark: ❏ rsync for 100 of t2.micro nodes on AWS EC2. GNU Parallel fireap real 0m46.906s 0m18.992s user 0m40.407s 0m00.527s sys 0m04.241s 0m00.046s
  7. 7. How “fireap” works?
  8. 8. Overview
  9. 9. config/fireap.toml [task.apps.bench] exec_commands = [ "rsync -az --delete <%= @remote.address %>:<%= ENV ['HOME'] %>/<%= @app %> <%= ENV['HOME'] %>/", ]
  10. 10. Run “fireap fire” on Publisher node // on publisher node % fireap fire -a bench -v 0.1.0-demo
  11. 11. Subscriber node exec commands // on subscriber node % tailf fireap.log : 2016-04-20 17:08:08 +0900 [INFO] EXEC rsync -az --delete 172.31.0.1:/home/bob/bench /home/bob/ :
  12. 12. Propagates through the whole cluster
  13. 13. Changes in v0.1 ~ v0.4 ❏ Enable to filter target nodes by Consul “service”, “tag” and “node name”. ❏ Add config validation before every command execution. ❏ Add `task` subcommand to show tasks in config file.
  14. 14. Things to be done ❏ Write more tests and do more benchmarks ❏ Plan to monitor Consul ❏ seems preferable especially in bigger systems ❏ Datastore alternatives (maybe) ❏ may be needed if Consul Kv goes slow in bigger systems ❏ Clean up codes
  15. 15. How to get started ❏ Documentation is Available ❏ https://github.com/key-amb/fireap/wiki ❏ If it doesn’t work, let me know as GitHub issue. ❏ I’m happy if you try “fireap”.
  16. 16. Thank you!
  17. 17. ❏ Serf/Consulで管理を自動化! ~実践的な 手法を紹介~:特集|gihyo.jp ❏ Consulと自作OSSを活用した100台規模の Webサービス運用 // Speaker Deck Appendix

×