Beyond Horizon: Open Source Management On the Go for OpenStack and the Rest of Your Computing Infrastructure
•Download as PPTX, PDF•
2 likes•1,469 views
Mist.io's presentation in the OpenStack Summit in Paris: Using a single UI to manage and monitor your private and public cloud, Docker containers and bare metal servers.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Beyond Horizon: Open Source Management On the Go for OpenStack and the Rest of Your Computing Infrastructure
Similar to Beyond Horizon: Open Source Management On the Go for OpenStack and the Rest of Your Computing Infrastructure (20)
Recently uploaded
Recently uploaded (20)
Beyond Horizon: Open Source Management On the Go for OpenStack and the Rest of Your Computing Infrastructure
- 1. Beyond Horizon open source management for OpenStack & the rest of your computing infrastructure
- 2. About us Dimitris Moraitis CTO & co-founder dimo@mist.io Chris Loukas SRE & QA loukasc@mist.io
- 3. Agenda 1. Computing landscape today 2. Pain points & our story 3. Solution: Outline, architecture & demo 4. Advanced client use cases a. Robopolling for the US congressional elections b. Spirent Communications & OpenStack NFV testing for telcos 5. Next steps 6. Q & A
- 4. Computing landscape today Public clouds Private clouds Containers Bare metal servers Management provisioning, orchestration, monitoring, alerting, automation
- 5. Vendor or complexity lock-in?
- 7. Why u no settle with Horizon? ● No monitoring, alerting, automation ● No websockets, limited polling ● No command shell beyond clunky VNC ● One for each OpenStack install ● OpenStack only ● Desktop only
- 8. The founders Working together since 2009 Chris Psaltis CEO Dimitris Moraitis CTO Mike Muzurakis Community manager Tasos Nanos System operations Markos Gogoulos Backend engineer
- 9. Our story Strong network in USA and Europe
- 10. Advisors Allen Wirfs-Brock Research Fellow, Mozilla Project editor @ ECMAScript Language Standard David Hecht Entrepreneur, Marketing & Sales Founder @ CloudAmp & GoGrid. Techstars Cloud Mentor Geva Perry Strategy & Marketing Advisor at Heroku, Ravello, Totango etc. Board member Sauce Labs etc. Techstars mentor Andreas Kallis Business development & Strategy Startup and VC advisor, Former VP of Biz dev & Strategy EMEA @ Thomson Reuters
- 11. The other team members George Tsopanoglou Frontend Engineer Chris Loukas QA & SRE Dimitris Rozakis Backend Engineer Stephane Konstantaropoulos iOS & Android developer
- 15. API / CLI / Ansible integration >>> from mist.client import MistClient >>> client = MistClient(email='dimo@mist.io', password='***********') >>> juno = client.backends(name='Juno')[0] >>> juno.create_network(name='priv', cidr='192.168.4.0/24') >>> juno.create_machine(name='dev1',tags=['dev'],networks=['pub','priv']) >>> client.machines(tag='dev').run('apt-get update && apt-get upgrade') $ mist run 'apt-get update && apt-get upgrade' --tag=dev $ mist add rule --machine=dev1 --metric=ram --operator=gt --value=95 -- action=command:'service zope restart' $ mist play playbook.yml --backend=Linode
- 16. Architecture ... browser server Νative APIs EC2 SSH VM Ember.js jQueryMobile libcloud RackSpace VM ... collectd email, sms, graphite-M1 collectd receiver push notifications bucky mist.monitor Websocket carbon-relay graphite-M2 OpenStack mist.android mist.ansible mist.client 3rd party tools REST API paramiko mist.alert Websocket preprocessor celery + rabbitmq pyramid + uwsgi
- 17. Advanced use cases by our clients ● Progressive Change Campaign Committee o Robopolling for the US congressional elections ● Spirent Communications o NFV testing for telcos using OpenStack and Mist.io
- 18. Polling for the US mid-term elections boldprogressives.org NGO, co-founded by Aaron Swartz, campaigning for progressive candidates. > 1million members ● Candidates use pies.pccc.co to run campaigns ● PIES uses Mist.io to orchestrate the deployment of VM’s on Linode or DigitalOcean ● PCCC monitors the systems while candidates monitor the polling process with custom metrics provided by Mist.io ● When poll is complete Mist.io automates the cleanup process
- 19. SDN / NFV testing for telcos Spirent wants ● on-demand, multi-node OpenStack test environments on bare metal & cloud servers ● configure NFVs (e.g. vEPC) on multiple distros ● integration with in-house build process
- 20. SDN / NFV testing for telcos Mist.io provides: ● automated deployment/tearing-down in minutes ● automated network configuration ● health & performance monitoring of host and guests ● live monitoring of test process metrics ● auto-scaling o on the app level o on the OpenStack level adding extra compute/network nodes
- 21. Mist.io sets you free! ● democratizes cloud management ● monitors VMs, containers and bare-metals ● sends actionable alerts ● provides simple automation o execute commands o reboot/destroy/launch
- 22. Next steps ● Custom graphs & dashboards, improve UX ● Reports: usage, cost, SLA compliance ● Native Android/iOS apps ● Social devops
- 23. Try Mist.io for free https://mist.io
- 24. Thank you! Let us know about your own automation stories & desires info@mist.io
Editor's Notes
- Hey all, it’s really great to be here. This talk is titled Beyond Horizon and we’re going to show you a cool way to manage your OpenStack installations along with the rest of your infrastructure.
- A few words about us: I’m Dimitris and this is Chris. We work on mist.io which is SaaS and an open source project. We run an office in Athens, Greece, where we do most of the product development and another one in San Francisco.
- So, without further delay, here’s the agenda! We’ll take a brief look on the ways we do computing today, the pain points that this introduces and our story in coming up with Mist.io: We’ll do a quick overview of the main features, we’ll see what’s under the hood and then go for a bold live demo! Then, we’ll dive into a couple of more advanced use cases by our clients. The first one is related to the US mid-term elections that took place yesterday and the other one is about SDN & NFV testing, the next big things in networking where OpenStack is destined to play a very important role. Then we’re going to tell you about our future plans and we’ll make sure to save some time for Q&A!
- Here’s how the computing landscape looks like today. You have a rapidly growing bunch of public clouds with different advantages and pricing policies, a bunch of private cloud platforms, where OpenStack keeps expanding its mind-share. Containers are becoming increasingly important since the arrival of Docker and other technologies like CoreOS are building on this trend. Yet we still have these old school bare metal boxes, which make a lot of noise and consume power. Sometimes they’re part of some legacy app, or they may be nodes in our private OpenStack clouds. The management needs for all these are common, but each platform comes with a different toolset.
- All this power, comes with a price. We get to choose our own set of golden handcuffs. Are you going with a single vendor, using only those tools? If you do that you need to be sure that your vendor will always provide competitive features and pricing to justify living behind its walled garden. But sometimes you don’t want that, it may not be an option. So you have to use a bunch of different tools that do similar things and this poses practical restrictions of its own.
- In the OpenStack world we got Horizon, which provides an administrative dashboard, a way to launch instances, create networks, manage storage, security groups and more. So why would you need something else?
- The problem is that you don’t get to monitor your servers, you can’t get alerts or configure automation. You can’t even be sure if your machines are still running if you don’t keep hitting on the refresh button since it doesn’t get live updates through websockets or polling. Then, it’s limited to OpenStack and you need a different Horizon for each OpenStack deployment. Last but not least, it’s close to unusable from touchscreens and especially smartphones, so if you’re trying to perform management operations while on the road, well… you’re out of luck.
- All these were the problems we were facing along with my cofounders. We’ve been working together since 2009 initially on our own consulting agency.
- The idea to build mist.io emerged out of our own needs on server management and monitoring. We’ve been developing and managing information systems from clients from all around the world. The systems were hosted in different clouds and datacenters and we had to make sure they were running and when something went off we had to fix it ASAP. So, we started scratching our own itch by building a unified dashboard and we wanted to make sure it was accessible from any device in order to make it usable in cases of emergencies when there was noone in front of a laptop or desktop computer. In late 2012 we had an initial open source product and in 2013 we applied with applied to Mozilla’s accelerator, WebFWD, and got accepted. At that point we decided to focus entirely on mist.io. We traveled to the States, learned a lot and grew our network, we incorporated there and raised our first money.
- We attracted world class advisors
- And recruited a kickass team. With used all that help in order to build an awesome product. And that’s how mist.io was built.
- But what does it actually do? Mist.io provides a unified interface to manage the most popular public and private clouds, as well as bare metal servers and docker containers
- It will keep an eye on your machines at all times and you can easily configure events that trigger alerts, or automated actions. So for example, if Apache is leaking memory mist.io can automatically restart it.
- If something goes wrong, you’ll get an alert, but more importantly you can act upon it from anywhere you are. To do that mist.io provides a command shell which is optimized for touchscreens. This way it’s very simple to send a few commands and fix the problem right away.
- Lets say for instance I want to create a new machine, that will be on a public network and a new private one. To do that, I need to import the python binding, instanciate etc.
- So, before we try out the demo, let’s have a look uder the hood. On the server side Mist.io is a Python web app, build with Pyramid and served by uWSGI. It implements a RESTful API and a Websocket interface. The end user interacts with the server using either a browser and the HTML5 interface, our command line tools or the native android app which is coming soon. Both the html5 app and the native android app use websockets to stay up to date while the command line tools use the RESTful API. The mist.io server queries your cloud backends through their native APIs with the help of libcloud, a Python library that provides an abstraction layer. It can also connect through SSH to your VM’s in order to provide the command shell and to install collectd, the monitoring agent. If you don’t want to give ssh access to mist.io you can still have monitoring by installing collectd manually. Collectd adds very little overhead. It gets all the system metrics and any custom metric you have configured every 5 seconds and sends them to our monitor server. The datapoints are preprocessed there for consistency and are stored in a graphite cluster. We’ve build there some alerting logic and checks if the incoming data match the user’s rules and if they do it triggers the alerts or the automated actions.
- This is all really cool and it’s exactly what we needed when we started building mist.io. But what has really amazed us is the different and totally unexpected ways that our clients have used our set of tools
- Yesterday it was election day in the US. And we’re really proud for the role that Mist.io has played. This may have been the first time that a cloud management tool helped promote Net Neutrality! The progressive change campaign commitee, is a political organization with more than a million members. One of its cofounders was Aaron Swartz. They’re also one of our clients. They’ve built a tool called PIES which they use inhouse and just started providing to select progressive candidates. One of the thing that PIES does is to launch polling campaigns. Each time a candidate requets a new poll, PIES uses mist.io to spin up a new machine either on Linode or on DigitalOcean. That machine runs the poll with the help of Twilio, providing questions and storing answers. While the poll is running, the candidates can monitor the polling process, while the PCCC technical staff can monitor the machines that run the polls. All the monitoring metrics are provided by Mist.io, for example the number of ongoing calls and the pending ones. When the poll is complete, Mist.io automatically triggers the cleanup process, which stores the poll data to an S3 bucket and then destroys the VM. This was build within a few weeks and they switched providers
- Another client and also a technology partner of ours is Spirent communications. They’re a company founded in 1936 and their bussiness is telecommunications testing. Their clients are telcos and the new product that Spirent is offering is automated SDN and NFV testing. SDN stands for Software Defined Networking and NFV for Network Functions Virtualization. As more telcos are looking into a future where their network functions are all virtualized, they need to make sure that they don’t sacrifize anything in terms of reliability, security and performance. So, the goal of the testing environment is to perform: functional and performance tests probing for security weaknesses and vulnerabilities. Spirent performs the tests using OpenStack installations and they need to be able to easily spin up new deployments of multiple openstack distros. The main objectives for Spirent are: ease and speed of test lab setup repeatability cost minimization using commodity hardware
- We delivered all that by providing flexible, on-demand OpenStack environments, manageable via a single easy-to-use UI/CLI. These are deployed on top of either cloud or bare metal servers by Nephoscale, a cloud provider in California which is also a technology partner of ours. Nephoscale allows Spirent to provision bare metals through the API which is really handy in terms of automation and scalability. Mist.io uses the Nephoscale and the Openstack API’s in order to automatically configure the networks for the test procedures, including sophisticated configurations like setting up L2 networks & VXLANs per tenant. Once the machines are provisioned and OpenStack is deployed, Mist.io takes care of monitoring the host machines, the guest VMs and the test process itself using custom metrics. These are used to provide actionable alerts to Spirent and also to trigger auto-scaling
- As we’ve seen, mist.io helps you get free of vendor and platform lock-in You can monitor containers, bare metals and vms, both in terms of system and application metrics. It can notify the administrators or the user’s of said systems with actionable alerts And it makes it really simple to configure automations in terms of executing commands, rebooting, destroying or launching machines You can use all that to automatically upscale or downscaling on the application level, by adding VM’s that joing your distributed app And as we’ll see next you can even do autoscaling of your openstack installations by adding additional compute or network nodes where needed, as we’re going to see next
- Look how easy it is to ….. Can you do that in Horizon?
- stories, battle cases, see how we can help you