Ce diaporama a bien été signalé.
Nous utilisons votre profil LinkedIn et vos données d’activité pour vous proposer des publicités personnalisées et pertinentes. Vous pouvez changer vos préférences de publicités à tout moment.
PimpYour
Pipeline
Central
Configuration
Management
Jens Saade, youngculture
April 25th 2015, Iasi CodeCamp Romania
ReferencesAbout youngculture Group
Jens Saade
Head Innovation & Technology E-Commerce
Engineering Background (Retail, PIM,...
Central Configuration
Repository
Appliance in Continuous
Delivery
Automation &
Configuration
Intro youngculture
E-Commerce
Intro:
E-Commerce
@YC
youngculture ECOM team delivers highly
integrated E-Commerce Solutions based on
the Hybris Platform.
Customized B2C Storefront Showcase
B2C & B2B
Shops and
PIM Systems
Delivering good quality in a highly
complex project environment is only
possible by aligning team, processes
and infrastru...
Con2nuous	
  
Delivery	
  
Configura2on	
  
Mgmt	
  
Test	
  
Automa2on	
  
Cloud	
  
Services	
  
Automated	
  
Provisioni...
Automation
Continuous Delivery
Familiar with the Concepts?
Tried applying it in own Company?
Part of your Culture?
AUTOMATION
is hard work ...
expensive to establish...
and can get pretty complex
BUT
... reduces monkey work
... speeds up...
Release anytime, reproducible
Create
Hybris
Projects,
Extension
and Addons
Setup
Continously
Build
deployable
artifacts
Bu...
CustomersDevelopers Product
Owner
Scrum
Master
Delivery Process: Automation is Key
Test Execution Env
Jenkins
CI
Artifacto...
One essential problem left:
Configurations
Ecosystem of
Configurations
Versioned
Software
Artifact
Dev
1
Dev
2
Ops
1
Ops
2
QA
Test specifc
artifact
versions for
regre...
7
Env.properties
?
Project Specific
236Project.properties
436Advanced.properties
7
Env.properties
?
Project Specific
236Proj...
(manual)
Configuration
Central
Configuration
Management
Diversity: Support complex environment setups
!=
We need good concepts how to configure
environments nearly equivalent
but ...
„The most efficient way to manage
configuration is to have a central service
through which every application can
get the confi...
Artifact
Generic
Configuration
Machine
Java VM
Rolling out Configurations
Configurations get
packaged and rolled out
JVMs are...
Turning from distributing (and manually
tweaking) local configurations ...
... to a centralized environment
that hosts confi...
Artifact
Machine
Java VM
Rolling out Configurations, alternative
Rollout Software Artifact as
usual, but keep it generic,
j...
Hierachical Key-Value Store
Configuration
Configuration
UUID3
UUID4
Machine
Application
Client Wrapper
Central Configuration ...
CD & CCR
Applied @YC
Configuration Hierarchies
Automated space
UUID1
Shared space
UUID2
UAT Environment
UUID7
UAT node 1
UUID8
UAT Node 2
UUID9
...
Customized (env1)
UUID3
Customized (env2)
UUID4
Machine
Java VM
discoBit Java Agent (UUID4)
System Context Injection via J...
jsaade@MacBook-Pro:~/hybris/hybris-commerce-suite-5.4.0.0/hybris/bin/platform$ ./hybrisserver.sh
Launching a JVM...
Apr 20...
Key Take
Aways
Key Take Aways
•  Simple, unified approach to avoid wide-spread
configuration and their mutation
•  Language & product agnos...
Usecases
•  Configure applications in hosting enviroments,
test farms, desktops
•  Developers share configurations while cre...
Pros
•  Central administration
•  Create multi-level configuration inheritance
•  Turn from a push-based model to a consume...
Cons (yes there are)
•  Accessibilty from environment to CCR needs to
be resilient, fallbacks needed (e.g. local caching)....
Where to go next? Potential Extensions
•  Tracking changes possible (who did what and
when)
•  Add versioning by storing c...
Questions?
Thanks!
Mail: j.saade@youngculture.com
Twitter: @jenssaade
youngculture.com
Hybris.com
discoBit.com
Appendix
Continuous Delivery Hosting
QA Testing
Jenkins CI
Embedding into Continuous Delivery Flow
discoBit
Application
Instance 2
...
Continuous Delivery
QA Testing
Source (git)
Jenkins
Stacked Configurations in the Central Config Repo
Commit Stage
env.prope...
Pimp Your Pipeline - Central Configuration Management - Jens Saade
Pimp Your Pipeline - Central Configuration Management - Jens Saade
Pimp Your Pipeline - Central Configuration Management - Jens Saade
Pimp Your Pipeline - Central Configuration Management - Jens Saade
Prochain SlideShare
Chargement dans…5
×

Pimp Your Pipeline - Central Configuration Management - Jens Saade

960 vues

Publié le

Jens Saade, Head of Innovation & Technology at our E-commerce department, was speaking about Central Configuration Management @ CodeCamp Romania

Publié dans : Ingénierie
  • Soyez le premier à commenter

  • Soyez le premier à aimer ceci

Pimp Your Pipeline - Central Configuration Management - Jens Saade

  1. 1. PimpYour Pipeline Central Configuration Management Jens Saade, youngculture April 25th 2015, Iasi CodeCamp Romania
  2. 2. ReferencesAbout youngculture Group Jens Saade Head Innovation & Technology E-Commerce Engineering Background (Retail, PIM, E-Commerce) Located in Zürich, CH Dev   Process   Architecture   Innova2on   Ini2a2ves  
  3. 3. Central Configuration Repository Appliance in Continuous Delivery Automation & Configuration Intro youngculture E-Commerce
  4. 4. Intro: E-Commerce @YC
  5. 5. youngculture ECOM team delivers highly integrated E-Commerce Solutions based on the Hybris Platform.
  6. 6. Customized B2C Storefront Showcase
  7. 7. B2C & B2B Shops and PIM Systems
  8. 8. Delivering good quality in a highly complex project environment is only possible by aligning team, processes and infrastructure environments.
  9. 9. Con2nuous   Delivery   Configura2on   Mgmt   Test   Automa2on   Cloud   Services   Automated   Provisioning   Workspace   (Confluence,   JIRA)   Prac2ce   Meetups   Integrated   Communica2on   (HipChat)   System   Monitoring   HW/OS   (NewRelic,   Nagios)   Applica2on  Monitoring   Logs   (Elas2cSearch/ Kibana,  Splunk)   Performance   (Dynatrace),   Delivery Environment Environments CAM Collaborate AutomateMonitor TeamsSystems Projects
  10. 10. Automation
  11. 11. Continuous Delivery Familiar with the Concepts? Tried applying it in own Company? Part of your Culture?
  12. 12. AUTOMATION is hard work ... expensive to establish... and can get pretty complex BUT ... reduces monkey work ... speeds up the dev & delivery processes ... calms down stress level ...and raises confidence and quality.
  13. 13. Release anytime, reproducible Create Hybris Projects, Extension and Addons Setup Continously Build deployable artifacts Build Assess Software by Automated Testing Test Continuous Delivery Overview Install for testing, acceptance and production Deploy > Commit Stage > Acceptance Stage > Deployment Stage Feedback Feedback Feedback Establishing a Delivery Pipeline is a joint team effort! It involves lots of disciplines and evolves constantly.
  14. 14. CustomersDevelopers Product Owner Scrum Master Delivery Process: Automation is Key Test Execution Env Jenkins CI Artifactory Repository Git Repository Hybris Node YC Deploy Client Apache HTTPdSelenium Gatling JMeter Test Exec Hybris Test Commit Stage Acceptance Stage Deploy Stage Continuous Delivery Monitoring Node Stage/Production Hybris Node1 Hybris Cluster Apache HTTPd Varnish Apache HTTPd Varnish DB Cluster Load Balancer ElasticSeach Logstash Kibana Zabbix External Monitoring NewRelic Hybris Node 2 YC Deploy Client YC Deploy Client Hosting Environment TeamsInfrastructures Maintenance team
  15. 15. One essential problem left: Configurations
  16. 16. Ecosystem of Configurations Versioned Software Artifact Dev 1 Dev 2 Ops 1 Ops 2 QA Test specifc artifact versions for regressionQA on features Create new Conf Property Share his conf with Dev 2 Prepare Rollout Conf for tonight Enable feature for customer Req Eng Introduce new concept, define triggers A=1 B=true c=debug
  17. 17. 7 Env.properties ? Project Specific 236Project.properties 436Advanced.properties 7 Env.properties ? Project Specific 236Project.properties 436Advanced.properties Node 1 7 Env.properties ? Project Specific 236Project.properties 436Advanced.properties Node 2 7 Env.properties ? Project Specific 236Project.properties 436Advanced.properties Node 3 1 Clustered Environment Dev(s) QA Systems Production UAT/Stage ... Hybris Configuration Stack (Properties based)
  18. 18. (manual) Configuration
  19. 19. Central Configuration Management
  20. 20. Diversity: Support complex environment setups != We need good concepts how to configure environments nearly equivalent but allow flexibility for individual deviations
  21. 21. „The most efficient way to manage configuration is to have a central service through which every application can get the configuration it needs.“ Jez Humble and Dave Farley „Continuous Delivery: Reliable Software Releases through Build, Test, and Deployment Automation“, Addison-Wesley 2010
  22. 22. Artifact Generic Configuration Machine Java VM Rolling out Configurations Configurations get packaged and rolled out JVMs are bootstrapped from packaged configurations Machine Java VM Machine Java VM Machine Java VM Environment Configuration
  23. 23. Turning from distributing (and manually tweaking) local configurations ... ... to a centralized environment that hosts configurations for self-service. Direction change: Push -> Pull
  24. 24. Artifact Machine Java VM Rolling out Configurations, alternative Rollout Software Artifact as usual, but keep it generic, just provide configuration IDs Isolate configurations from the deployment package and provide it centrally from the CCR Machine Java VM Machine Java VM Machine Java VM UUID Generic Configuration CCR   Environment Configuration UUID UUID UUID UUID UUID Generic Configuration Environment Configuration
  25. 25. Hierachical Key-Value Store Configuration Configuration UUID3 UUID4 Machine Application Client Wrapper Central Configuration Repository (CCR) and Consuming Clients Agent pulls configuration properties via REST / Json Then injects properties to JVM System context MachineMachineMachine Machine Machine Machine Machine MachineMachineMachineMachine Machine Machine Machine Machine Machine REST / Json discoBit A P I Admin UI CCR  
  26. 26. CD & CCR Applied @YC
  27. 27. Configuration Hierarchies Automated space UUID1 Shared space UUID2 UAT Environment UUID7 UAT node 1 UUID8 UAT Node 2 UUID9 QA: Nightly Build Test System UUID6 Personalized Conf Developer A UUID3 Personalized Conf Developer B UUID5 Ops 1 Ops 2 QA Dev 1 Dev 2 Req Eng Continuous Delivery Cycle
  28. 28. Customized (env1) UUID3 Customized (env2) UUID4 Machine Java VM discoBit Java Agent (UUID4) System Context Injection via Java Agent discoBit Agent pulls configuration properties via REST / Json Then injects properties to JVM System context Automated space UUID1 Shared space UUID2
  29. 29. jsaade@MacBook-Pro:~/hybris/hybris-commerce-suite-5.4.0.0/hybris/bin/platform$ ./hybrisserver.sh Launching a JVM... Apr 20, 2015 1:42:33 PM com.v3rticle.oss.discobit.client.bootstrap.DiscobitSettings <init> INFORMATION: [discobit] configuring server from system: http://127.0.0.1:8089 Apr 20, 2015 1:42:33 PM com.youngculture.ecom.hybris.discobit.HybrisDiscobitAgent execute INFORMATION: Reading discobit configuration from http://127.0.0.1:8089::bb2835fb-466c-4143-8081-ffcd60c49467 Apr 20, 2015 1:42:33 PM com.v3rticle.oss.discobit.client.bootstrap.DiscobitSettings <init> INFORMATION: [discobit] configuring server from system: http://127.0.0.1:8089 Apr 20, 2015 1:42:33 PM com.v3rticle.oss.discobit.client.bootstrap.DiscobitSettings <init> INFORMATION: [discobit] configuring server from system: http://127.0.0.1:8089 Apr 20, 2015 1:42:33 PM com.v3rticle.oss.discobit.client.DiscobitConnector authenticate INFORMATION: [discobit] authentication response: HTTP/1.1 200 OK Apr 20, 2015 1:42:33 PM com.v3rticle.oss.discobit.client.DiscobitConnector authenticate INFORMATION: [discobit] authentication cookie: [[version: 0][name: JSESSIONID][value: 65452FC130FE2F87E8655BF05D5F0A90][domain: 127.0.0.1][path: /][expiry: null]] Apr 20, 2015 1:42:34 PM com.youngculture.ecom.hybris.discobit.HybrisDiscobitAgent execute INFORMATION: injected 677 properties to system context WrapperManager: Initializing... Apr 20, 2015 1:42:39 PM org.apache.tomcat.util.digester.SetPropertiesRule begin WARNUNG: [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'disableURLRewriting' to 'true' did not find a matching property. Apr 20, 2015 1:42:40 PM org.apache.tomcat.util.digester.SetPropertiesRule begin WARNUNG: [SetPropertiesRule]{Server/Service/Engine/Host/Context} Setting property 'disableURLRewriting' to 'true' did not find a matching property. Apr 20, 2015 1:42:40 PM org.apache.coyote.AbstractProtocol init INFORMATION: Initializing ProtocolHandler ["http-bio-9001"] Apr 20, 2015 1:42:40 PM org.apache.coyote.AbstractProtocol init INFORMATION: Initializing ProtocolHandler ["http-bio-9002"] Apr 20, 2015 1:42:45 PM org.apache.coyote.AbstractProtocol init INFORMATION: Initializing ProtocolHandler ["ajp-bio-8009"] Apr 20, 2015 1:42:45 PM org.apache.catalina.startup.Catalina load INFORMATION: Initialization processed in 5834 ms Apr 20, 2015 1:42:45 PM org.apache.catalina.mbeans.JmxRemoteLifecycleListener createServer INFORMATION: The JMX Remote Listener has configured the registry on port 9003 and the server on port 9004 for the Platform server Apr 20, 2015 1:42:45 PM org.apache.catalina.core.StandardService startInternal INFORMATION: Starting service Catalina
  30. 30. Key Take Aways
  31. 31. Key Take Aways •  Simple, unified approach to avoid wide-spread configuration and their mutation •  Language & product agnostic •  Enabling other teams and colleagues to take controlled influence on configs in general •  Needs a clear strategy concerning delivery cycle, security and infrastructure
  32. 32. Usecases •  Configure applications in hosting enviroments, test farms, desktops •  Developers share configurations while creating a solution •  QA configures their testing systems individually •  Dark launching of software changes •  Feature toggling for indidual usage profiles
  33. 33. Pros •  Central administration •  Create multi-level configuration inheritance •  Turn from a push-based model to a consumer pull-based self service •  Allow fine grained access to (parts) of configurations for different stakeholder roles •  Change properties without rollouts (trigger reload events instead) + + + + +
  34. 34. Cons (yes there are) •  Accessibilty from environment to CCR needs to be resilient, fallbacks needed (e.g. local caching). •  Configuring during runtime can have unforseen side effects in your application (check lifecycle). •  Administrative access to configurations needs to be well thought. -­‐   -­‐   -­‐  
  35. 35. Where to go next? Potential Extensions •  Tracking changes possible (who did what and when) •  Add versioning by storing configs in an SCM for rollbacks ore regression testing •  Stage changes for different rollout scenarios •  Add hierarchical config structures like YAML
  36. 36. Questions?
  37. 37. Thanks! Mail: j.saade@youngculture.com Twitter: @jenssaade youngculture.com Hybris.com discoBit.com
  38. 38. Appendix
  39. 39. Continuous Delivery Hosting QA Testing Jenkins CI Embedding into Continuous Delivery Flow discoBit Application Instance 2 UUID 2 Application Instance 1 UUID1 Application Instance n UUID n Config Repository REST API Artifactory Distributable Artifact Acceptance Stage Deploy Stage deploybuild configure UI 1   2   3   Commit Stage env.properties project.properties advanced.properties
  40. 40. Continuous Delivery QA Testing Source (git) Jenkins Stacked Configurations in the Central Config Repo Commit Stage env.properties discoBit Application Instance 1 UUID3 Acceptance Stage Deploy Stage deploybuild configure Automated space UUID1 Shared space <extends> UUID2 UUID1 <extends> Customized (env2) Customized (env1) UUID3 UUID4 Devs’ Machine Application Instance 3 UUID4 Application Instance 2 UUID3 1   2   project.properties advanced.properties merge

×