3. an open-source project supported by
Canonical that is the defacto multi-
distribution package that handles the early
initialization of a cloud instance.
cloud-init is:
Some background
5. Some background
∎ Setting up custom repositories
∎ Configuring an instance hostname
∎ Bootstrapping salt / puppet / chef / etc
∎ Managing users and SSH keys
∎ Setting up ephemeral mount points
∎ Configuring network devices
CAPABILITIES
6. Some background
User-data is a powerful part of cloud-init that
enables the end user to inject configuration
options and/or scripts into the cloud-init
process at provision time, which is used by
cloud-init to do, well, whatever you want!
USER-DATA
7. Some background
Let’s kick off a demo real quick using inithub
and user-data to show you how it works, then
circle back to the presentation
(QUICK DEMO)
8. Some background
● shell scripts
● cloud-config YAML
● url includes
● upstart-jobs
● part-handlers
● cloud-boothook
USER-DATA FORMATS
9. Some background
● simple shell script example
USER-DATA FORMATS
#!/bin/bash
echo “Hello world!” > /tmp/shellscript-test
10. Some background
● simple cloud-config (YAML) example
USER-DATA FORMATS
#cloud-config
write_files:
- content: |
Hello world!
path: /tmp/cloud-config-test
* more examples: https://cloudinit.readthedocs.io/en/latest/topics/examples.html#yaml-examples
11. Some background
● less simple cloud-config example
USER-DATA FORMATS
#cloud-config
# Add groups to the system
# The following example adds the ubuntu group with members foo and bar and
# the group cloud-users.
groups:
- ubuntu: [foo,bar]
- cloud-users
# Add users to the system. Users are added after groups are added.
users:
- default
- name: foobar
gecos: Foo B. Bar
primary-group: foobar
groups: users
selinux-user: staff_u
expiredate: 2012-09-01
ssh-import-id: foobar
lock_passwd: false
passwd: $6$j212wezy$7H/1LT4f9/N3wpgNunhsIqtMj62OKiS3nyNwuizouQc3u7MbYCarYeAHWYPYb2FT.lbioDm2RrkJPb9BZMN1O/
- name: barfoo
gecos: Bar B. Foo
sudo: ALL=(ALL) NOPASSWD:ALL
groups: users, admin
ssh-import-id: None
lock_passwd: true
ssh-authorized-keys:
- <ssh pub key 1>
- <ssh pub key 2>
- name: cloudy
gecos: Magic Cloud App Daemon User
12. Some background
Allows you to combine multiple data formats
MULTIPART-MIME
MIME-Version: 1.0
Content-Type: multipart/alternative;
boundary="_=_swift_v4_1495602312_307e131ddbcf662f952cd6a359c3ad31_=_"
--_=_swift_v4_1495602312_307e131ddbcf662f952cd6a359c3ad31_=_
Content-Type: text/cloud-config; charset=utf-8
Content-Transfer-Encoding: quoted-printable
#cloud-config
write_files:
- content: |
Hello world, I was written using cloud-config
path: /tmp/cloud-config-test
13. Some background
In the end, if you are using cloud-init to any real
degree you’ll have to decide where these pieces
of data live.
Solutions range from revisioned S3 buckets to
github gists and random files on various local
desktops.
WHERE DOES THIS LIVE?
14. Some background
Not super great, especially if you are
working in a collaborative environment.
WHERE DOES THIS LIVE?
15. Some background
Cloud-init and user-data are extremely powerful and useful
tools, but has some practical usability issues:
∎ multi-part mime format is hard to format and manage
∎ user-data can get long and unwieldy
∎ there isn’t a great place for all the user-data to live
TO SUM UP
17. So, fine.
Why does Packet care about this?
Packet has thousands of users who utilize cloud-
init and user-data every day, and when it fails or
has issues, then it becomes our problem even if
we are not the direct cause.
18. So, like many projects, a solution was born out of
frustration and laziness
20. We had several goals for inithub:
∎ extend cloud-init only, do not require any
modifications or changes for it to work
inithub.org overview
21. We had several goals for inithub:
∎ extend cloud-init only, do not require any
modifications or changes for it to work
∎ provide a dedicated repository for user-data that
is revisioned and easy to use
inithub.org overview
22. We had several initial goals for inithub:
∎ extend cloud-init only, do not require any
modifications or changes for it to work
∎ provide a dedicated repository for user-data that
is revisioned and easy to use
∎ reduce the likelyhood of user error by providing
dynamically rendered multipart user-data
accessed by simple include urls statements
inithub.org overview
23. So basically, inithub.org takes this:
inithub.org overview
#cloud-config
# Add groups to the system
# The following example adds the ubuntu group with members foo and bar and
# the group cloud-users.
groups:
- ubuntu: [foo,bar]
- cloud-users
# Add users to the system. Users are added after groups are added.
users:
- default
- name: foobar
gecos: Foo B. Bar
primary-group: foobar
groups: users
selinux-user: staff_u
expiredate: 2012-09-01
ssh-import-id: foobar
lock_passwd: false
24. and gives you this:
inithub.org overview
#include https://inithub.org/u/00ckKk
25. and gives you this:
inithub.org overview
#include https://inithub.org/u/00ckKk
27. So it’s basically a url shortener. Yippee.
Fundamentally yes, but wait, there’s more!
inithub.org overview
28. So it’s basically a url shortener. Yippee.
Fundamentally yes, but wait, there’s more!
inithub allows you to easily combine scripts,
includes, and cloud-config yaml into multipart mime,
using the same simple include format
inithub.org overview
29. So it’s basically a fancy, dynamic url shortener.
inithub.org overview
30. So it’s basically a fancy, dynamic url shortener.
Yep!
inithub aims to solve a few usability issues with
cloud-init, that cloud-init can’t easily solve without a
central platform.
inithub.org overview
35. Inithub.org is currently what we consider a first
release. It is a simple, functional tool that provides
the fundamental value we desired.
We would love to continue to develop the tool
given community interest and contributions.
FUTURE ROADMAP
In Conclusion
36. You can sign up today at inithub.org
We also are on slack at slack.inithub.org
We’re at the Packet booth for the rest of the
conference if you have questions or feedback
Thank you!
JOIN US
In Conclusion
Notes de l'éditeur
Please raise your hands if you are familiar with or have used cloud-init
Fundamentally, cloud-init is software that runs after a cloud instance has been created which can do a wide variety of operations to configure the instance. Both the cloud provider and the end user can use it to do anything from configuring RAID, setting up network, installing other software, or anything else that may be required to bring the instance online and into the desired state after boot.
Cloud-init is supported It is currently installed in the Ubuntu Cloud Images and also in the official Ubuntu images available on Packet, AWS, Azure, GCE and many other clouds.
Versions for other systems can be (or have been) created for many of the most common distributions you use and love including Ubuntu, Fedora, Debian, RHEL, CentOS, Free/Open BSD, CoreOs’ ContainerOS, RancherOs, etc. We even have cloud-init support on windows at Packet.
Some more examples of common use cases
User-data is a powerful part of cloud-init that enables the end user to inject configuration options and/or scripts into the cloud-init process at provision time, which is used by cloud-init to do, well, whatever you want!
User-data is a powerful part of cloud-init that enables the end user to inject configuration options and/or scripts into the cloud-init process at provision time, which is used by cloud-init to do, well, whatever you want!
shell scripts - anything with a she-bang will get run as a plain old shell script, bash, python, etc
cloud-config - this is the simplest way to do many things via user-data. cloud-config syntax uses YAML, so is “human friendly” and can do a lot of different things out of the box on boot, including package updates / upgrades, setting up different software repositories or mirrors on the system, importing ssh keys and configuring groups and users, writing out arbitrary files, bootstrapping distributed management agents like puppet / chef / etc, configuring resolv.conf - you get the idea
url include - The file contains a list of urls, one per line. Each of the URLs will be read, and their content will be passed through this same set of rules. Ie, the content read from the URL can be gzipped, mime-multi-part, or plain text.
upstart job - content is placed into a file in /etc/init, and will be consumed by upstart as any other upstart job.
part-handlers - custom code for either supporting new mime-types in multi-part user data, or overriding the existing handlers for supported mime-types. Must be python code and is generally used to extend or modify the behaviour of clout-init itself
cloud-boothook - this is the earliest executed hook in cloud-init, and is executed every time the system boots or cloud-init runs. I’m not incredibly familiar with use-cases of this format.
This is a somewhat simple example even, and you can see it is getting relatively verbose, so can be unweidly to cut and paste, which is often how user-data is pushed to a cloud provider. This is prone to errors and costly troubleshooting that is a pain in the ass.
Multipart mime user-data is really powerful, it let’s you combine YAML for tasks that are supported by cloud-init out of the box, with scripts or other includes that may require more custom or advanced operations.
The problem is, in practice it requires you to use a helper script or something else to format the multipart mime bits, which can be pretty cumbersome, especially if the parts change with any frequency.
We have people who bootstrap everything from network storage nodes to gaming nodes using cloudinit, and of course many just use it to do simple bootstrapping to the distributed management system of choice. Those who are doing more complicated things with container oriented operating systems like coreos’ container os utilize it even more heavily to boot into swarm / docker engine / k8s, etc.
We also heavily utilize cloud-init for bootstrapping nodes ourselves as part of our provisioning process - including initial network and disk setup. I personally use it for setting up demonstrations on the platform, and trying out / testing other software that our end users commonly use, so I started to feel the pain myself directly in terms of ease-of-use and reusability / mangement of the various user-data I was working with. That’s when I came up with an idea...