OpenStack Upgrade - It can be done! Koby Holzer, Liran Cohen - OpenStack Day Israel 2017

•Télécharger en tant que PPTX, PDF•

0 j'aime•235 vues

- Planning, planning and planning - 80% of the work is here - Getting ready to upgrade using automation and thinking 100% uptime in every corner. - It's time: The upgrade itself

Technologie

Openstack Upgrade
It can be done!
June 2017, Tel Aviv
Koby Holzer
Liran Cohen

About us
Liran Cohen Koby Holzer
● Director of Platform ENG
● ~5 years @ Liveperson
● Cloud ENG group leader
● ~5 years @ Liverperson

Agenda
● About Liveperson
● Openstack @ Liveperson
● Planning - Openstack upgrade.
● Testing - Openstack upgrade.
● The Upgrade...

Liveperson in one sentence:
We created a
Digital Contact Center
to allow
Transitioning from Voice to
Messaging

Creating Meaningful Connections
Between Brands and Customers
Founded in 1995,
public since 2000
(NASDAQ: LPSN)
More than 18,000
customers worldwide
More than 1000
employees
25B site visits and +300M live
engagements Yearly
7 DC’s around the globe Headquartered in NYC with
offices in Americas, EMEA
and APAC

18,000 customers around the globe, including:
Customers
8 of the Top 10 Fortune
500 companies
10 of the Top 15
Commercial Banks
(Fortune 500)
Top 4 of Top 5 Telecommunication
Companies (Fortune 500)
3 of the Top 5 Internet
Service Companies (Fortune
500)

2011 2012 2013 2014
● Started early 2012
● 100% of our services
Physical
Openstack
2015 2017
Openstack @ LivePerson
● 70K virtual cores.
● ~1K physical servers.
● 7 Data Centers.
● 7K instances.
80%

Openstack @ LivePerson
Nova Neutron Glance Horizon Keystone

Operating System
CM - Deployment
Network - Provider networks (VLAN)
Hypervisor.
Storage - non persistent using hypervisor Local disks.
Openstack @ LivePerson
Ubuntu 14.04 LTS (Trusty Tahr)

Role: Controller
nova-api
nova-cert
nova-consoleauth
nova-novncproxy
nova-objectstore
nova-scheduler
glance-registry
glance-api
keystone neutron-server
neutron-dhcp
neutron-openvswitch-agent
openvswitch-switch
MySQL
neutron-l3
neutron-metadata
rabbitMQ
Images Network ComputeIdentity
Services - Controllers

Services - Compute
Role: Compute
nova-compute
neutron-openvswitch-agent
libvirt-bin
openvswitch-switch

Web Tier
App tier
Data Tier
Tech Stack @ Liveperson
OS tier
Supporting tools

Openstack Support model On Ubuntu ● LTS support 5 years
● Non LTS support 18 months
Mitaka - OS upgrade - 16.04
Queens - OS upgrade - 18.04

Release Notes
Review the release notes to learn about new,
updated, and deprecated features.

Impact ?
Users ?Apps ? Automations ?
Network Storage Compute

Choose your upgrade path
Possible ways to upgrade:
● Big Bang (in-place) Upgrades.
● Parallel Cloud - Side by Side clusters.
● Controller Side by Side.
● Rolling upgrade (Upgrade Levels).*

The upgrade plan...
● Create an upgrade procedure.
● Automate - As Much As Possible.
● Rollback is part of the plan.

production like:
● Hardware.
● Network configuration.
● OS Version , kernel.
● repos - Minor versions.
● Tools - CM , Upgrade Automations.
Testing Environment

● Apply your upgrade plan.
● Assess and Modify.
● Rollback.
Testing Environment

The path we chose
Side By Side Controller Upgrade.
2 steps:
● New controller upgrade & DB sync
● Compute upgrades
Tests after every step

Old
Controller
Nova
Neutron
Glance
Keyston
e
Nova
Neutron
Glance
Keyston
e
● Take a snapshot of the cloud.
● Shut down OpenStack services gracefully.
● Sync the data , images and DB.
Old(Active) Controller
New
Controller

● Import DB and upgrade schema.
● Start OpenStack services.
● Do basic API checks and Review Logs.
New Controller

● Upgrade control group.
● Run tests on the new controller.
● Upgrade the next group.
● Monitor (Api , AMQP , DB etc…)
Compute

Control
Group
Compute Servers
Group 1 Group 2 Control Group
Old
Controlle
r
New
Controlle
r

Compute Servers
Group N Group 1 Control Group
Group
1Old
Controlle
r
New
Controlle
r

Compute Servers
Group N Group 1 Control Group
Group
NOld
Controlle
r
New
Controlle
r

● During the Upgrade
● After the Upgrade
Must have for every upgrade

Koby Holzer
kobyh@liveperson.com
Liran Cohen
liranc@liveperson.com

Contenu connexe

Tendances

OpenStack is a free and open-source software platform for cloud computing, mostly deployed as an infrastructure-as-a-service (IaaS). OpenDaylight is an open source project under the Linux Foundation with the goal of furthering the adoption and innovation of SDN through the creation of a common industry supported platform. In this session, I will talk about how OpenStack and OpenDaylight can be combined together to solve real world business cases and networking needs. We will cover: - What is OpenDaylight - Use cases for OpenDaylight with OpenStack - The OpenDaylight NetVirt project - How OpenDaylight interacts with OpenStack - The future of OpenDaylight, and how we see it help solving challenges in the networking industry such as NFV, container networking and physical network fabric management -- the open source way.

OpenStack and OpenDaylight: An Integrated IaaS for SDN/NFV

Cloud Native Day Tel Aviv

The primary requirements for OpenStack based clouds (public, private or hybrid) is that they must be massively scalable and highly available. There are a number of interrelated concepts which make the understanding and implementation of HA complex. The potential for not implementing HA correctly would be disastrous. This session was presented at the OpenStack Meetup in Boston Feb 2014. We discussed interrelated concepts as a basis for implementing HA and examples of HA for MySQL, Rabbit MQ and the OpenStack APIs primarily using Keepalived, VRRP and HAProxy which will reinforce the concepts and show how to connect the dots.

High Availability for OpenStack

Kamesh Pemmaraju

Neutron high availability open stack architecture openstack israel event 2015

Arthur Berezin

YouTube Recording: https://www.youtube.com/watch?v=cCdqOxD5G0M Whether you are a newbie to OpenStack looking at building your first cloud or an experienced operator with years of OpenStack success behind you, you've probably spent some time wondering what to expect from the OpenStack project over the next several releases. Will it finally support that new capability you've been waiting for? Should you plan for an upgrade in the next 6 months? While the development community is always working and planning new features, its takes a lot of time on IRC to get a complete view across the different projects. The OpenStack Product WG spent time this cycle working with the project teams and PTLs to understand their priorities for the next several OpenStack releases. Where we have always had an understanding of what's to come in the next release, we're hoping to present a long-term view of the future landscape of OpenStack. In this session, we'll present our findings across the different projects in an effort to give users a glimpse into the OpenStack roadmap

What's Next in OpenStack? A Glimpse At The Roadmap

ShamailXD

OpenDaylight is the largest open source SDN controller. OpenDaylight's Netvirt project provides an alternative mechanism and architecture for provisioning networks using Neutron. OpenDaylight Netvirt is an active project with dozens of contributors worldwide and serves as the basis of numerous commercial products and services. This talk aims to provide a technical overview of the OpenDaylight Netvirt project and its integration with Neutron. Topics will include: * Neutron/OpenDaylight architectural overview * openstack/networking_odl v2 architecture and new features * OpenDaylight Netvirt's networking deep dive * Advantages of OpenDaylight Netvirt This talk targets people interested in using or developing OpenDaylight for OpenStack.

OpenDaylight Netvirt and Neutron - Mike Kolesnik, Josh Hershberg - OpenStack ...

Cloud Native Day Tel Aviv

[OpenStack Day in Korea 2015] Keynote 2 - Leveraging OpenStack to Realize the...

OpenStack Korea Community

[OpenStack Days Korea 2016] Track1 - Red Hat enterprise Linux OpenStack Platform

OpenStack Korea Community

Audience: Advanced About: Real world lessons and war stories about Catalyst IT’s experience in rolling out an OpenStack based public cloud in New Zealand. This presentation will provide tips and advice that may save you a lot of time, money and nights of sleep if you are planning to run OpenStack in the future. It may also bring some insights to people that are already running OpenStack in production. Topics covered will include: selection of hardware for optimal costs, techniques that drive quality and service levels up, common deployment mistakes, in place upgrades, how to identify the maturity level of each project and decide what is ready for production, and much more! Speaker Bio: Bruno Lago – Entrepreneur, Catalyst IT Limited Bruno Lago is a solutions architect that has been involved with the Catalyst Cloud (New Zealand’s first public cloud based on OpenStack) from its inception. He is passionate about open source software, cloud computing and disruptive technologies. OpenStack Australia Day - Sydney 2016 https://events.aptira.com/openstack-australia-day-sydney-2016/

Things You MUST Know Before Deploying OpenStack: Bruno Lago, Catalyst IT

OpenStack

[OpenStack Days 2016] Track4 - OpenNSL으로 브로드콜 기반 네트,워크 스위치 제어하기

OpenStack Korea Community

What do you do when your usual setup or turnkey solution isn’t suited for your workload? Most of the documentation and user feedback that you can find about OpenStack is written for the use-case of running a public facing cloud serving several external customers. When you want to host a single tenant with a single application the problem is completely different, you don't want publicly exposed APIs. You want to ensure optimal resource allocation to maximize your application performance. You want to leverage the fact that you own the infrastructure layer to optimize your instance placement strategy, and to get the best latency and to avoid creating SPOFs using affinity (or anti affinity rules). This talk will focus on what we learned during a two years journey; from getting OpenStack up and running reliably, to investigating performance bottlenecks, to maximizing the performance of our private cloud.

Openstack Summit Tokyo 2015 - Building a private cloud to efficiently handle ...

Pierre GRANDIN

OpenStack practitioners who have deployed cloud at scale would frown when they hear the mention of Open Virtual Switch (OVS), which has been a bottleneck for cloud network performance and scalability. As emerging technologies such as NFV keep pushing for higher data forwarding performance across the network infrastructure, it becomes critical to improve OVS performance without compromising flexibility, network programmability, and cost. We will present a novel way to offload the entire OVS dataplane onto the embedded switch (eSwitch) implemented in the server NIC. This approach maximizes the effective bandwidth that the applications can use to communicate with each other or fetch data from storage, and enhances the efficiency of the cloud. Accelerated Switching And Packet Processing (ASAP2) Direct works seamlessly within the framework of SDN, and allow controllers to configure and update flows onto OVS the same way as before so that network programmability remains intact.

Can the Open vSwitch (OVS) bottleneck be resolved? - Erez Cohen - OpenStack D...

Cloud Native Day Tel Aviv

Cloud Networking - Leaving the Physical Behind - Omer Anson - OpenStack Day I...

Cloud Native Day Tel Aviv

[2015-11월 정기 세미나] Cloud Native Platform - Pivotal

OpenStack Korea Community

Cloudify NFV Orchestrator for Optimal Performance

Arthur Berezin

Cinder Updates - Liberty Edition

OpenStack Foundation

[OpenStack Day in Korea 2015] Track 2-3 - 오픈스택 클라우드에 최적화된 네트워크 가상화 '누아지(Nuage)'

OpenStack Korea Community

Role of sdn controllers in open stack

openstackindia

OpenStack HA

Kenneth Hui

Five years of operating a large scale globally replicated Pulsar installation...

StreamNative

High availability and fault tolerance of openstack

Deepak Mane

Tendances (20)