Charming 2.0 – Now with 100% more awesome Pt.2
by Charles Butler on 26 October 2015
Continuing from my blog last week, lets rewind to looking at a basic bash based historical charm.
├── config.yaml
├── hooks
│ ├── config-changed
│ ├── install
│ ├── relation-name-relation-broken
│ ├── relation-name-relation-changed
│ ├── relation-name-relation-departed
│ ├── relation-name-relation-joined
│ ├── start
│ ├── stop
│ └── upgrade-charm
├── icon.svg
├── metadata.yaml
├── README.ex
└── revision
Fig 2.1 – Traditional Charm Heirarchy
The majority of the charm logic went into the respective hook, and any duplication of logic/code was extrapolated into charm-helpers or a manual library distributed among different charms. This lent itself to version mismatches, difficulty in discerning where components came from, and a dependency on a core understanding of how the juju hooks/events were occuring in the environment.
Now, with a reactive approach, the charms directory structure gets much bigger due in part to some nice things that the charm builder exposes to us, and embeds in the basic layer. These bits are covered above.
.
├── actions
│ ├── clean-containers
│ └── clean-images
├── actions.yaml
├── composer.yaml
├── copyright
├── hooks
│ ├── config-changed
│ ├── install
│ ├── start
│ └── stop
├── icon.svg
├── lib
│ ├── charmhelpers
│ ├── charmhelpers-0.5.0.egg-info
│ ├── charms
│ ├── charms.reactive-0.3.4.egg-info
│ ├── jinja2
│ ├── Jinja2-2.8.dist-info
│ ├── markupsafe
│ ├── MarkupSafe-0.23.egg-info
│ ├── netaddr
│ ├── netaddr-0.7.18.dist-info
│ ├── pyaml
│ ├── pyaml-15.8.2.egg-info
│ ├── PyYAML-3.11.egg-info
│ ├── six-1.10.0.dist-info
│ ├── six.py
│ ├── six.pyc
│ ├── tempita
│ ├── Tempita-0.5.2.egg-info
│ └── yaml
├── Makefile
├── metadata.yaml
├── reactive
│ ├── docker.py
│ └── __init__.py
├── README.md
├── requirements.txt
├── scripts
│ └── install_docker.sh
├── tests
│ ├── 00-setup
│ ├── 10-deploy-test
│ ├── notes.md
│ └── tests.yaml
└── tox.ini
Fig 2.2 – An assembled charm in the Reactive Pattern
Looking over this directory tree, we see many new additions. The lib/ directory includes as outlined in the charm build process… but we also see a new directory:
/reactive
To grok whats happening here, lets inspect any of the hooks included in the charm.
#!/usr/bin/env python
# Load modules from $CHARM_DIR/lib
import sys
sys.path.append('lib')
# This will load and run the appropriate @hook and other decorated
# handlers from $CHARM_DIR/reactive, $CHARM_DIR/hooks/reactive,
# and $CHARM_DIR/hooks/relations.
#
# See https://jujucharms.com/docs/stable/getting-started-with-charms-reactive
# for more information on this pattern.
from charms.reactive import main
main()
Fig 2.3 – hooks/config-changed.py
As we can see, our hook code has become extremely skinny, and invokes something called the “reactive main” method. This leads us back to the reactive directory we found in the assembled charm.
import os
from subprocess import check_call
from path import path
from charmhelpers.core import hookenv
from charms import reactive
from charms.reactive import hook
@hook('install')
def install():
hookenv.status_set('maintenance', 'Installing Docker and AUFS')
charm_path = path(os.environ['CHARM_DIR'])
install_script_path = charm_path/'scripts/install_docker.sh'
check_call([install_script_path])
hookenv.status_set('active', 'Docker Installed')
reactive.set_state('docker.available')
Fig 2.4 – reactive/docker.py
Any module found in the reactive directory will be included in the hook execution, and as events are set, and subscribed to, they will be executed across any of the modules found.
In Fig 2.4, we see some familiar things and some not so obvious things. We’re defining @hook('install') which tells me this will run anytime we are in the context of the install charm deployment. Nice and familiar. But what about this reactive.set_state bit?
Rejoyce! User defineable/subscribable persistent states
Reactive’s core pattern is to allow You the charm author to define persistent states in the charm. Gone are the days of touching sentinel files, or manually
writing data to the keystore on the unit. Reactive allows you to define a meaningful state, and then subscribe to that across any reactive module running
on the unit.
How would you use this you say? Lets take a look at a snippet from the nginx-docker-layer
@when('docker.available')
@when_not('nginx.available')
def install_nginx():
'''
Default to only pulling the image once. A forced upgrade of the image is
planned later. Updating on every run may not be desireable as it can leave
the service in an inconsistent state.
'''
copy_assets()
hookenv.status_set('maintenance', 'Pulling Nginx image')
check_call(['docker', 'pull', 'nginx'])
reactive.set_state('nginx.available')
Fig 2.5 – subscribed reactive event
When the docker.available state is set, this method will be executed, and not until. This allows us to describe meaningful, domain specific events in our deployment such as storage.attached, database.available, backup.completed and anything your mind comes up with. Just be sure to clearly document them in your layer so anyone building on top knows what events are surfaced during the deployment.
Did you notice you were able to combine @when and @when_not? These two decorators were intended to work hand in hand. Set states that define when its OK to take an action, and make it idempotent with states!
Known Caveats
Before we setoff fireworks, there are a few things to be made aware of that may trip you up on your first ventures through charming with reactive:
- Do not mix @hook and the other decorators
- Event execution order is not garanteed, idempotence is critical on this path
Examples of Donts
@hook('config-changed')
@when_not('nginx.available')
def launch_nginx_container():
...
Fig 2.6 – Example of broken decorator pattern
@hook('config-changed')
def launch_nginx_container():
...
Fig 2.7 – example of racey ‘reactive/nginx.py’
@hook('config-changed')
def something_dependent_on_nginx():
...
Fig 2.8 – example of racey ‘reactive/nginx-monitor.py’
These methods will be racey and as the method declaration states: Fig 2.8 is dependent on 2.7 completing successfully in this scenario. You’re not garanteed
order in this pattern, so its best to surface and subscribe to events rather than depend on a racey hook condition among layers.
Interface Layers (Formerly known as stubs)
TL;DR – Interface Layers are a python class that define the communication between related services on a given interface. If you are familiar with how interface based programming works in a statically compiled language, you will be familiar with how interface layers work in Juju Charms.
Charm building extends even to interfaces and relationships. Historically it could be rather painful to really understand what was being sent over the wire of a relationship. It required either reading the code of the relationship hooks to determine what data was being sent, or to trap the hook execution of the relationship and inspect what was actually sent on the wire.
This was a growing problem of complexity as there were several charms that consumed/provided an interface, however they were slightly different implementations and as there is no difinitive contract between the services – nothing enforced a change in this pattern of growing complexity.
Interface Layers remove these problems.
from charmhelpers.core import hookenv
from charms.reactive import hook
from charms.reactive import RelationBase
from charms.reactive import scopes
class HttpProvides(RelationBase):
scope = scopes.GLOBAL
@hook('{provides:http}-relation-{joined,changed}')
def changed(self):
self.set_state('{relation_name}.available')
@hook('{provides:http}-relation-{broken,departed}')
def broken(self):
self.remove_state('{relation_name}.available')
def configure(self, port):
relation_info = {
'hostname': hookenv.unit_get('private-address'),
'port': port,
}
self.set_remote(**relation_info)
Fig 1.4 – http/provides.py
Looking at this interface layer, we see 2 things happening, and it overlaps with
the Reactive Pattern – outlined below.
We inherit from a common base class RelationBase and this takes care of describing that this is an interface layer.
scope = scopes.GLOBAL
This single line is particularly interesting, as it defines the conversation scope for the interface. If any unit connecting to the service will get the same data, or if each unit in the service needs to respond, or if a service level response is required. These are outlined further in the charm.reactive docs
Further down, we also see some templating logic at play here, to define multiple hooks/states in the method decorator
@hook('{provides:http}-relation-{joined,changed}')
This defines that any relation, providing the http interface (it gets substituted by what is named in metadata.yaml as the relation) will invoke this method on
the relationship action.
To consume the data sent over the wire defined by the interface layer, you gain an object when decorating a relationship event. Illustrated as follows:
@when('website.available')
def configure_website(website):
config = hookenv.config()
website.configure(config['port'])
Fig 1.5 – reactive/apache.py
The above example shows that the website.available state determines when this runs, and makes itself a paramater to the decorated method. This allows
us to push data into the proper relationship context, consistently across implementations in charms. Here we are simply setting the port in which apache is running.
Editors Note: Be sure to look at the writeup of the Vanilla PHP BB charm in Reactive. Cory Johns did a great job of explaining intent, and how it was implemented in a digestible format. Read it here
Lots of new things!
With all of these base concepts in your mind, you’re ready to take a deep dive into charming with reactive, and reducing complexity by extrapolating common
concerns into layers, and re-useable interface layers. Together we will build a bright future for modeling your workload that can run in any cloud, under the guidance of Juju.
Stay tuned for a video overview of charming with layers. And if you’ve got any questions be sure to send them to the mailing list or to drop by on IRC in #juju on irc.freenode.net
Happy Hacking!