[Rakuten TechConf2014] [C-6] Japan ICHIBA Daily Work - Tools & Processes
•Download as PPTX, PDF•
3 likes•2,455 views
Rakuten Technology Conference 2014 "Japan ICHIBA Daily Work - Tools & Processes" Takahiro Yamaki (Rakuten)
Recommended
Recommended
More Related Content
What's hot
What's hot (20)
Viewers also liked
Viewers also liked (10)
Similar to [Rakuten TechConf2014] [C-6] Japan ICHIBA Daily Work - Tools & Processes
Similar to [Rakuten TechConf2014] [C-6] Japan ICHIBA Daily Work - Tools & Processes (20)
More from Rakuten Group, Inc.
More from Rakuten Group, Inc. (20)
Recently uploaded
Recently uploaded (20)
[Rakuten TechConf2014] [C-6] Japan ICHIBA Daily Work - Tools & Processes
- 1. Japan ICHIBA Daily Work - Tools & Processes - Oct/25/2014 Takahiro Yamaki Japan RMS Group, Japan Ichiba Section, Rakuten Inc.
- 2. 2 About Me and Development Tools • Name: Takahiro Yamaki • 2004 ~ 2012 – An information technology services company – Front Line Team, Support Team – Development Tool Lover • Trac, Redmine, SVN, TestLink, MS TFS, etc.
- 3. 3 About Me and RMS Group • 2012 – Rakuten Ichiba Development Department • -> Japan Ichiba Section • -->> Japan RMS Group – Application Engineer • Purchase History • Shopping counter for the Super Sale, sale events • … etc.
- 4. 4 About Me and Kaizen Team • 2014~ – Kaizen Team in Japan RMS Group • For Japan RMS Group –Development Improvement –Operation Improvement –Newcomer Training – Productivity Engineer • CI-nization
- 5. 5 About RMS Group in the B2B2C model merchant shopper Rakuten Ichiba RMS MALL
- 6. 6 My Today's Goal Change your impression of Japan ICHIBA DevOps.
- 7. 7 My Today's Goal Not so Bad old-fashioned You Like! I like to work in Japan Ichiba!
- 8. 8 Japan ICHIBA DevOps Daily Work Table of contents 1. Ticket Driven DevOps 2. Automation 3. Tools Connectivity
- 9. 9 Japan ICHIBA DevOps Daily Work Table of contents 1. Ticket Driven DevOps 2. Automation 3. Tools Connectivity
- 10. 10 Development Ticket Flow (Backlog) (Execution) ${service} Merchants @Event Biz Go SYS DBA System Security Office Reporting Rakuten DevOps
- 11. 11 Ops Ticket Flow Backlog Execution Merchants Shopper Rakuten DevOps Call Center Helpdesk Member Service (Daisy) (ECHELP) Engineers Biz (misc) Inquires
- 12. 12 QUESTIONS?
- 13. 13 Japan ICHIBA DevOps Daily Work Table of contents 1.Ticket Driven DevOps 2.Automation 3.Tools Connectivity
- 14. 14 Ichiba and Atlassian Dev Tools Git repository management Git client CI Tool
- 15. 15 What I did to drive Development Automation?
- 16. Blueprint Phase 16 AMTUL Framework & Stakeholders Stakeholders Replace Manual Process Training Trial in Dev, Stg Demo Benefits Blueprint Aware Memory Trial Usage Loyalty
- 17. 17 Auto build application counts 100 90 80 70 60 50 40 30 Demonstration 20 10 0 1 giant ant project 64 maven projects YOKOTEN Oct-13 Nov-13 Dec-13 Jan-14 Feb-14 Mar-14 Apr-14 May-14 Jun-14 Jul-14 Aug-14 Sep-14 Oct-14 1st Build Success Day Build Plan Counts
- 18. 18 Build Plan, Current Basic Design DEV Stage clone build deploy STG Stage clone build deploy PROD Stage clone build deploy
- 19. 19 Demo Movie
- 20. 20 Demo : Auto Deploy GlassFish (Application Server) Clusters #1 #2 #3 #4 Manual Test Continuous System Test * (3) (2) (4) * Kotaro Ogino and Francois Picalausa “Continuous System Test”. Test Automation. http://kokotatata.hatenablog.com/entry/2014/03/14/075842 (1) CI Tool
- 21. 21 Continuous System Test (Current) Selenium Hub (Test Case & Results Management) CI tool Data RMS Selenium Nodes Script Results Check Manual Test Results
- 22. 22 <Future> Develop & Release Flow STG Conf Test QA Build DEV IT Acceptance Test Release Judge PROD Blue- Green Deploy Clone Build UT Code Analysis Deploy Conf Test Conf Test Deploy ST Code Review Metrics Release Judge Security Test ST Security Test
- 23. 23 QUESTIONS?
- 24. 24 Japan ICHIBA DevOps Daily Work Table of contents 1.Ticket Driven DevOps 2.Automation 3.Tools Connectivity
- 25. 25 Data Allocation Jira Test Cases Test Results Stash Codes maven Confluence SonarQube Code Quality Artifacts Library Issues Documents RMS Bamboo Burp Security Report Build Results TestRail
- 26. 26 Tickets Codes Tickets Codes Codes Tickets
- 27. 27 Build results Codes, Tickets Change Log (Who? What?) compared to last build Fixed or Related Issues
- 28. 28 Test Case Management tool Tickets Can create a issue from Test Failure Report Test Result Tickets Can refer related issues
- 29. 29 QUESTIONS?
- 30. 30 I have a question for you!
- 31. 31 Changed? Legacy old-fashioned You Not so Bad Like! I like to work in Japan Ichiba!
- 32. 32 Thank you
Editor's Notes
- Good evening everyone, I’m Takahiro Yamaki from Japan RMS Group, Japan Ichiba Section. Today I’d like to introduce our daily work to you from the view point of our daily development tools & processes. ---- みなさんこんにちは 日本市場課 日本RMSグループの山木隆寛です。 今日は日本市場の日々の仕事の様子を、ツールやプロセスといった観点からお話したいと思います。
- Before we begin, let me introduce myself. I started my career at an information technology service company and I worked in front line teams or support teams. The ratio was 50-50. In this period I used some development tools like Trac, Redmine, TestLink, Microsoft Team Foundation Server, and so on and became a tool lover. (強調 しり上がり) --- 私の自己紹介をさせていただきますと、情報技術サービス会社のエンジニアとして2004年からキャリアをスタートしました。 前線(front line) と サポートチームでの仕事が半々でした。 この期間に、TracやRedmine、TestLink、MS Team Foundationといったツールを使っていくうちに、段々と開発ツール愛好家になっていきました。
- I joined Rakuten in 2012 and I got assigned to Japan Ichiba section, Japan RMS Group. As an application engineer, I developed the purchase history and the shopping counter for Super Sale or monthly sale event. ---- 2012年に楽天へＪＯＩＮしました。最初は、日本市場課、日本ＲＭＳグループのR-Backofficeチームへ配属されました。 ここでは、アプリケーションエンジニアとして、購入履歴や、スーパーセールでのお買い物カウンターのバックエンド部分を開発したりしていました。
- Kaizen team was established in Japan RMS group this year. This team has 3 missions. Development improvement, operation improvement and new comer training for Japan RMS group. As a productivity engineer I am leading the CI-nization for Japan RMS Group. ---- 今年から、RMSグループ内に改善チームというものがつくられ、今はそこでProductivity Engineerとして働いています。 ここのチームの使命は、RMSグループのための、開発改善、運用改善、新メンバートレーニングです。
- And let me also introduce about RMS group in the Rakuten B2B2C business model. The center of B2B2C model is mainly separated into 2 big systems. Mall system is like shopping floor for shopping users. On the other hand RMS system is like backyard for shop owners and staffs. RMS system consists of shop configurations, item management, payment management, delivery management and so on. ---- ちなみに、市場システムでの役割分担を少しだけ説明します。 楽天はB２B2Cビジネスモデルです。まんなかのB部分は、MALLというエンドユーザがお買い物をするショッピングフロア部分と、RMSというバックヤードに大きく２つに分かれます。 そのバックヤード部分は、お店の共通設定や、商品マスタを管理するR-Storefront(RSF)というシステムと、決済・物流系のR-Backofficeというシステムからできています。
- Now here is my goal of today’s presentation. It is to change your impression of Japan ICHIBA DevOps! I’d like to explain what I mean. ---- 私の今日の目標は、日本市場の開発・運用に対するみなさんが持っているイメージを変えることです。
- Japan Ichiba system is the oldest system which was created from the beginning of Rakuten. So you may have the impression that Japan ICHIBA daily work is old-fashioned or legacy or something like that! For example, when we go to the orientation for our company’s new grads in spring or autumn, they don’t seem to have positive images for the Japan Ichiba. My goal of today’s presentation is to change your impression to … And I hope you’ll be more interested in working in Japan ICHIBA by the end of this speech. --- 日本市場は楽天創業時から続くシステムですので、みなさんの多くの方は「古臭い」というイメージをお持ちなのではないでしょうか？ 私たちが春入社、秋入社の新入社員に対して日本市場課の紹介しにいくと、あまり反応はよくないです。 今日の約20分ほどの紹介の後、そんなイメージが、(1) 悪くない、(2) いいね、(3)日本市場課で働きたい、というように変えることができるようにがんばります。
- Now let’s move on to today’s agenda. Today I want to introduce our daily work from 3 view points. 1st view point is Ticket Driven Development & Operations. 2nd is the automation. 3rd is tools connectivity. ---- アジェンダは３つあります。 1つ目は、チケット駆動開発・運用です。 2つ目は、自動化についてです。 3つ目は、開発・運用で使っているツール間の連携についてです。
- 1st topic is about Ticket Driven development & operations. ---- 1つ目のチケット駆動開発・運用です。
- This diagram is the summary for our development ticket workflow. These database icons stand for jira projects. A jira project is a collection of tickets. Merchants’ requests for new features or improvements are directly or indirectly added in the ‘Backlog’ project. We add these kind of tickets by ourselves. These backlog tickets are rearranged at fixed intervals. When a ticket get an approval to be implemented, this ticket is moved to the ‘Execution’ project. In this ‘Execution’ project there are tickets which is assigned to someone. Each team leaders report these ticket progress to managers. On the other hand for the front line members we usually use each service jira project. The reason is that it is easy to divide the tickets for development members. When we need to collaborate with infra structure team, we link our tickets to the request tickets for infra teams. As a result the history of each system is stored in these service jira projects. Furthermore there are some tickets which are directly added in the ‘Execution’ project. For example, security service team sometimes give us alerts for vulnerabilities. ----
- Let’s move on to the operation part. This diagram is the summary for our operation ticket workflow. When merchants have something to tell, they contact with Call Center Group or their EC consultants who are the 1st contacts for merchants. When shopping users have something to tell, they contact with Member Service Group who are the 1st contacts for shopping users. If 1st contacts are not able to resolve something, they escalate them to the support Group (we usually call them Helpdesk). They communicate using tickets. If helpdesk are not able to resolve something, they escalate them to us. When we are not able to fix them soon, we add them as backlog ticket or execution ticket. On the other hand business unit members have sometimes inquires for us. We made a jira project as the contact point to handle these kind of tickets. ----
- Do you have any questions or comments about this part? ---- ここまでで質問やコメントはありますか？
- Let’s move on to the next topic. 2nd topic is about automaton. ---- 次に、自動化に関する日々の仕事の紹介です。
- Ichiba began to use Atlassian tools. ---- Jira同様に、開発についてもアトラシアン製品の導入が進められました。
- What I did to drive our development automation? I looked backward to make this presentation. What I experienced may be helpful to you or somebody in the world.
- I think a marketing framework and the number of stakeholders are helpful to understand what is going on in the RMS group. AMTUL is the marketing frame work. It stands for Aware, Memory, Trial, Usage and Loyalty. After I experienced application development for 1 year, I realized that Build Automation is the most effective point in order to drive development automation. So I emphasized Build automation to our group members. These phases are Aware & Memory phases. Furthermore I think that the followers are important in order to make a movement. So I did training for someone who are interested in CI. Currently I coordinate these build automation or test automation training as one of the RMS new comer boot camp menus. That’s how we’ve been able to increase automation lovers gradually. Currently I am working on explaining our organization’s blueprint about development automation. This kind of blueprint prevents your stakeholders from misunderstanding where we go. Using this blueprint you may find what is the most effective point to drive your automation.
- This graph shows the number of build automation java projects. 1st 6th months were investigation and demonstration period for us. Next 6 months are rollout phase for other java applications. Next 3 months are expansion. The reason is that 1 giant ant project was divided into 64 maven projects. So about 90 java projects are managed by our CI tool. --- 結果、RMSグループのビルド自動化がこの1年であっというまに広まりました。最初は、試行錯誤とデモンストレーションフェーズで、そのご、他のアプリに対して横転していきました。 また、夏ごろ、ある途方もなく大きなCVS管理のアプリケーションが、64個のmavenプロジェクトに分割されて、分割を機に、自動ビルドできるようにしました。約90プロジェクトが自動ビルドできています。
- We use Atlassian Bamboo as CI tool. It is almost as same as Jenkins. This diagram shows the basic design for our java projects. There are separated stages for each environments. For examples development, staging and production environment. Each stage has some tasks, for example code clone, build and deploy it to each environment. After DEV stage finishes, next STG stage starts. There is a check point between STG and Production stages. After we push a button, the final stage starts. We don’t use deploy command for production environment. When our system come to near the blue print, automated release may be implemented. ---- 自動化のエンジンとして、Atlassian製のCIツールとしてBambooを使っています。Jenkinsと大体同じです。 この図は、ビルド計画のテンプレートです。DEV、STG、PROD環境別のステージわけをしていて、各ステージで、GitリポジトリからコードをCloneして、ビルドして、DEVやSTG環境にデプロイするようにしています。
- Today I want to show some demonstration.
- Today’s demonstration is a internal tool. It is running on the GlassFish application server. Today’s demonstration steps are (1) I make a new branch and push it (2) @Bamboo I start a build, (3) a war file is transferred to the GlassFish management server. (4) Bamboo kicks the deploy command to one of clusters. Usually we start to do manual tests or automated UI regression test to the new version. We call it “continuous system test”. Please refer this content which is written by Ogino-san about the concept, background and solution for the continuous system test. ---- これが全体の流れです。ビルドプランを選択し、ビルドが走り、Glassfish管理サーバにwarファイルを移送し、デプロイコマンドをキックします。デプロイ対象のクラス名は、先ほど入力した値を使います。 デプロイ後は、手動の結合テストや、セレニウムを使ったUIの自動テストを走らせます。 RMSグループでは、このUIの自動テストを 継続的システムテストと称し、デグレの早期発見に取り組みはじめました。 継続的システムテストの詳細（コンセプト、課題、ソリューション）はこちらのコンテンツをご参照ください。
- So let’s move on to the last automation topic. Last automation topic is about continuous system test. We have started to try it from this autumn. So this diagram is not the final version. The main technology is Selenium Hub & Nodes (The nickname is selenium2. Previously it was called Selenium Grid or selenium web driver) We prepares test scripts and test data which consists of input data and expected result. This test data format is json. Bamboo request the UI regression test to the selenium Hub, and then UI tests are executed in each Selenium Node. After the UI regression test finish, Bamboo puts the test result in our Test case and result management tool. We use TestRail as our test case and result management tool. So project leaders can see the test results using 1 tool. --- 今後の課題は、テスト作成コストや、メンテナンスコストを下げる工夫と、適切なテストケース数をこなすことです。個人的には、継続的システムテストは10分以内に終わるように維持したいです。 テストケースを厳選するのか、PCの数を増やすのかは、今後考えて調整していきます。
- This is our blueprint for our development automation. The most different point from current one is that there is only 1 build stage. It mean that 1 binary artifact is tested in several stages. 2nd different point is that automated tasks and human judgment are integrated as 1 build pipe line. ----
- These are about our development automation. It is under construction. So when I have a chance to talk about our automation season 2, I may talk about more automated tools and processes. And I hope so. Anyway do you have any questions or comments? --- ここまでで、何か質問はありますでしょうか？
- Let’s move on to the last topic. 3rd topic is about tools connectivity. ---- 最後のトピックのツール間の接続です。ここはさらっとお話して終わりです。
- This diagram describes our development data allocation. They are tickets, documents, codes, test cases, test results, artifact, builds results and code quality metrics data. These ‘Like’ icons are the points which I introduce you today. 1st is the connectivity between tickets and codes. 2nd is the connectivity between build results and tickets, codes. 3rd is the connectivity between Test Results and tickets. ---- チケット駆動開発・運用と、自動化関連で扱うデータ同士の関連はこのようになります。 Jiraから始まり、コンフルエンスというWikiみたいなドキュメントツールに仕様がまとめられ、これらが、テスト仕様書や、ソースコードに変わります。 ソースコードに対して、動的テストや静的コード解析のレポートがたまっていきます。 ソースコードはライブラリとして管理されるものものあり、これが、RMSシステムとして、世にリリースされます。 このパートでは、このLikeマーク部分のつなぎをご紹介します。
- This screen shot shows that Stash pull request and jira tickets are connected each other. So someone can trace why these codes are modified Or someone can trace what kind of codes are effected by this ticket. ---- まずは、Jira, Stash の関連です。 コミットコメントにチケットIDをいれる手間は必要ですが、プルリクエストの画面や、チケット側のViewに相互リンクがはられます。 このようにして、なぜこの変更が行われたかの変更が追いやすくなっています。
- This screen shot shows that build result and jira tickets, commit logs are linked. So when a build failed, you may easily find what modification brakes your build. ---- 次は、ビルド結果ページと、Stash、Jiiraへのリンクです。前回のビルドから、今回のビルドまでに、誰がどんな変更をしたか、サマリ表示されます。また、その変更はどのチケット起因かもリンクされます。 もしビルド失敗するようなことがあれば、犯人の特定がかなり容易です。
- This screen shot shows that when a tester inputs some comments as the test fail report, the comment and test preconditions, test steps and expected results are copied to a jira ticket. After that test result report and a jira ticket are connected each other. A coder get a bug report, the ticket has enough information to debug the problem. I like this feature! ---- それから、こちらは、マニュアルテストの失敗報告画面から、Jiraチケットを起票するところです。 1クリックで、テストの前提条件、手順、期待値、テスターのコメントを転記したjirチケットを起票できます。 当然、テスト結果レポートとこの起票されたJiraはリンクされるので、マネージャーが失敗したテストケースは対応済みかどうかがすぐに分かります。 コーダーはいちいちテスト実行手順を聞く手間を省けて、地味に便利な機能です。
- This is almost the end of today’s topics. Does anyone have any questions? ---- 以上が、簡単なツール間連携のお話でした。 他のパートも含め質問はありまうでしょうか？
- After listing to my presentation about Japan Ichiba daily work, has your impression changed? For example, Not so bad or Like! Or “I like to work in Japan Ichiba!” If you are interested in, please come to talk to me after this session. ---- これは私からの質問ですが、みなさんの日本市場課の開発運用に対するイメージは変わったでしょうか？ 変わった人は挙手をお願いします！？
- This is the end of my presentation; Thank you! ---- ご清聴ありがとうございます。