Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.

Atlassian's Mystique CLI, Minimizing the Experiment Development Cycle

78 vues

Publié le

Mystique CLI is an Atlassian developed CLI for Optimizely Web. It is a multi-phase project that is currently focusing on improving the development cycle for growth engineers. Currently, Mystique is the standard for developing web experiments at Atlassian, and is capable of a wide variety of operations utilizing Optimizely's REST API. This includes creating, updating, testing, and duplicating experiments/personalization campaigns, as well as "promoting" these entities between Optimizely projects for different environments (e.g. from QA => Prod). It has significantly reduced manual overhead and decreased development time by up to 95% for particular actions.

Publié dans : Logiciels
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Atlassian's Mystique CLI, Minimizing the Experiment Development Cycle

  1. 1. NATHAN CASPAR | SR. GROWTH ENGINEER | @NATHAN CASPAR Mystique CLI Minimizing the experiment development cycle
  2. 2. Content
 hosting Page
 targeting Audience
 definitions Experiment 
 bucketing
  3. 3. Content
 hosting Page
 targeting Audience
 definitions Experiment 
 bucketing
  4. 4. Content
 hosting Page
 targeting Audience
 definitions Experiment 
 bucketing
  5. 5. Content
 hosting Page
 targeting Audience
 definitions Experiment 
 bucketing
  6. 6. Why take the time and effort to build out a command line interface?
  7. 7. MANUALLY CREATE EXPERIMENT MANUALLY UPDATE EXPERIMENT COD MANUALLY DEPLOY EXPERIMENT MANUALLY TEST EXPERIMENT
  8. 8. IDEALISTIC WORKFLOW REALISTIC WORKFLOW
  9. 9. MANUALLY CREATE EXPERIMENT MANUALLY UPDATE EXPERIMENT COD MANUALLY DEPLOY EXPERIMENT MANUALLY TEST EXPERIMENT • Create & add audience targeting • Create & add page targeting • Create & add events • Create variations • Set traffic allocations • Develop experiment code in local • For each variation • For each page • Copy local code into web editor

 • Save experiment • Publish experiment • For each targeted page • Test page conditions • Test audience conditions • Test experiment visuals • Test for bugs
  10. 10. MANUALLY CREATE EXPERIMENT MANUALLY UPDATE EXPERIMENT COD MANUALLY DEPLOY EXPERIMENT MANUALLY TEST EXPERIMENT • Create & add audience targeting • Create & add page targeting • Create & add events • Create variations • Set traffic allocations • Develop experiment code in local • For each variation • For each page • Copy local code into web editor

 • Save experiment • Publish experiment • For each targeted page • Test page conditions • Test audience conditions • Test experiment visuals • Test for bugs
  11. 11. ANY CHANGE MEANS A LOT OF SMALL MANUAL STEPS
  12. 12. WHICH ADD UP
  13. 13. AND UP…
  14. 14. AND UP!
  15. 15. MANUAL STEPS ALSO MEAN DEFECT OPPORTUNITIES
  16. 16. WHAT WE WANT Consistency Efficiency Simplicity
  17. 17. Why not use an existing CLI?
  18. 18. W Design Customization in how we design experiments Development Customization in how we build experiments Analytics Customization in how we perform post-analysis
  19. 19. Phenomenal REST API Extremely Helpful support team
  20. 20. Be the change you seek
  21. 21. The many faces of Optimizely
  22. 22. Operations Optimizely UI performance Optimizely snippet performance Optimizely maintenance Developer Simple development cycle Keeping repo and Optimizely in sync Quickly deploying and testing updates Understanding when/where experiments ran Accessing Optimizely results and raw data Communicating all things experimentation PM / Analyst
  23. 23. Developer Simple development cycle Keeping repo and Optimizely in sync Quickly deploying and testing updates PM / Analyst Understanding when/where experiments ran Accessing Optimizely results and raw data Communicating all things experimentation Operations Optimizely UI performance Optimizely snippet performance Optimizely maintenance
  24. 24. Developer Simple development cycle Keeping repo and Optimizely in sync Quickly deploying and testing updates
  25. 25. REPETITIVE MANUAL STEPS QUICK SIMPLE COMMANDS
  26. 26. BASIC ACTIONS
  27. 27. SETUP Projects $ optx setup -pr TokensTokens  GET /projects
  28. 28. { "Project": { "key_metrics": [ 12345, 45678, 98765 ], "prefix": "www.website.com", "project_id": 3232345, "qa_audience": 2975832345, }, "QA Project": { "key_metrics": [ 53682, 48154, 70962 ], "prefix": "qa.stg.website.com", "project_id": 592823671, "qa_audience": 8275362345, }… } JSON everywhere Project config Experiment Personalization Events
  29. 29. $ optx create -p=exp.json CREATE | UPDATE | PRUNE P13n Campaigns TokensA/B Experiments  POST /experiments Events Pages Audiences
  30. 30. { "type": "ab_experiment", "jira_ticket": "BXPSCI-123", "optx_project": "QA Project", "experiment_name":"Opticon", "experiment_description": "Demo for Opticon”, "variations": ["Opticon demo"], "page_paths_ids": ["/", "/software"], } JSON everywhere Project config Experiment Personalizatio n Events
  31. 31. JSON everywhere Project config Experiment Personalizatio n Events { "type": "personalization_campaign", "jira_ticket": "BXPSCI-123", "optx_project": "QA Project", "campaign_name": "Opticon campaign", "page_paths_ids": ["/software","/try"], "experiences": [{ "exp_name": "Opticon experience 1", “variations": [ "first variation”, "second variation”] },{ "exp_name": "Opticon experience 2", "variations": [ "first variation v2", "second variation v2"]}] }
  32. 32. JSON everywhere Project config Experiment Personalizatio n Events { "type": "custom_events", "project": "QA Project", "events": [ {"key":"api_key_1", "name":"event name 1"}, {"key":"api_key_2", "name":"event name 2"}, {"key":"api_key_3", "name":"event name 3"} ] }
  33. 33. COMPLEX ACTIONS
  34. 34. PROMOTION EXP launch PROD create build/test EXP launch QA create build/test PR EXP launch PROD replicates
  35. 35. EXP launch PROD DUPLICATION EXP launch PROD create We want confirmation duplicates Time to product ionize
  36. 36. QA
  37. 37. Future Vision Automating clean up Handling project JS changes Exposing experiment metadata Working to open source

×