Web application development with Juju charms: an interview with Marc André Audet from Absolunet
by Tim McNamara on 5 December 2019
Targeting the web platform is increasingly complex. Tim McNamara, Developer Advocate in the Juju team at Canonical, recently interviewed Marc André Audet, Security Expert at Absolunet to discuss how Juju charms can be used for web application development. In the interview, you’ll learn about how to use Juju for web apps.
Tim: How do you find working with Juju for working the web? Would you recommend it for other agencies?
Marc: It might be overkill to develop custom charms for a single website. But if you are a business that makes websites, Juju is great.
I’ve done it for our client’s websites that all have the exact same structure and it’s worked very well. We can spin up a custom proof of concept for a sales pitch with almost no effort.
The best thing about Juju is that it is very flexible. It is easy to look for existing charms in the charm store and you adapt them to your needs.
That’s great to hear. How big is your team?
I’m alone in my team that works on Juju charms. I’ve built about 15 custom charms in order to make everything work. Magento sites represent at least half of our eCommerce projects and integrations, though I have also adapted the code to support WordPress and others.
Has your work paid off for you and your team?
Absolutely. Right now we have 2 clients in production using Juju, but we have spun up many sites for development, testing and sales purposes.
I’ve automated everything so much that we only have to deploy a bundle and we get a ready-to-use environment from scratch in under 20 minutes on the AWS cloud. And for any version of Magento. As long as Magento retains backwards compatibility, no changes are needed.
In the near future, we have plans to make it possible for anyone to spin up a new site with a single click, regardless of the intended use. With this, we expect to see an important increase in Juju usage and adoption at Absolunet.
Your colleagues must think that you have superpowers.
The developers are mostly enthusiastic about the project, other times I get “it should be faster”. The sales team is very happy with it though, because I’m able to setup a demonstration sandbox for every prospect now, rather than have two or three generic demo sites that they share across the team.
I don’t remember seeing Magento in the charm store. Did you write the charm yourself?
Yes, but I have actually made charms for several upstream applications.
The web frameworks we support are all charmed: Magento 1, Magento 2 and WordPress. They all operate behind NGINX as a base layer.
Databases, including MariaDB and Redis. My MariaDB charm includes the ability to create a Galera Cluster when needed. We can also deploy Redis Sentinel behind HAproxy with no effort.
HAproxy with support for HTTP & HTTPS, MySQL and Redis Sentinel. The HAproxy charm has 2 failover modes. The first mode connect to the nearest backend server and uses the others as a round-robin backup. The second failover mode is simply a round-robin—or least-connections—load balancer depending on which integration we want to support. For example, Redis doesn’t offer a choice because it’s a single-master setup.
We have a subordinate charm that can install PHP 5.6, 7.0, 7.1, 7.2, or 7.3 when needed. I support the use of multiple PHP versions for each of the web charms. We can have Magento instances on any PHP version, or even multiple PHP versions. With Juju, we have complete flexibility about which PHP version to use.
We manage our GlusterFS cluster with charms. We use Juju actions heavily here.
To connect our web charms to the file system, we use a subordinate charm as a “mount point”. The web charms all create their structures automatically based on the directory communicated from the subordinate charm. It sets up the mounts that are required and sends the mounted directory to its principal. The principal charm is then able to create its directory structure as required.
Interesting. Very impressive! So, that’s quite a lot of custom work.
Thanks. The reason why I used custom charms all the way is because there were some slight gaps in the public charms at the time. Plus I really wanted to learn how things work.
I’ve written all that in a bit less than a year—part time, I technically work in cybersecurity—what took most of my time was reducing complexity.
Now I can create a fully-featured charm in less than half a day if I use the “framework” I’ve built (over the reactive charming one). The only exceptions being clustered charms where it might take me about a week depending on the complexity.
Tell me more about how have you extended reactive.
Well, I’ve been using reactive for some time. I have added a layer on top of it to simplify it and clean the resulting code a bit by allowing maximum re-usability.
Basically, I have separated the logic parts into “actions” that are being called from the reactive parts. Therefore I can create lists of actions to execute for particular events which allows me to simplify the reactive parts.
Let me take you back to the idea of using subordinate charms for multiple versions of PHP. I haven’t heard of that before. Did you think of it yourself?
I thought of it myself. Most other charms that I saw integrated PHP directly in the charm, but I was required to work in an environment that uses many different versions of PHP.
I thought about subordinates because it would give me a lot more flexibility in terms of maintenance and would allow me to completely separate the automation code and provide a better service-level isolation.
The fact that we can install multiple versions at the same time is a consequence of this isolation and was actually a requirement for more legacy systems, so it ended up being extremely useful.
In terms of GlusterFS, it’s in order to manage the charm-local part of the mount and allows me to write more connectors in the future. Even if it could be integrated within the charms, I prefer to support services separately.
That all makes sense. It sounds like the Juju architecture has enabled you to tailor Juju to do exactly what you want.
Yes, more or less. I usually will create a base layer that includes all the actions and one or more charm layers that will use that base layer that will override some actions to change how it works.
I have a three layer structure.
Base layers. They all include one “master” base layer and provide the basis for MariaDB, Redis, NGINX, PHP-based web platforms and Node.js-based web platforms.
In the middle are “Integration Layers”. They provide tools to integrate other charms or to add new features. For example I have a Sendmail integration layer that automatically installs sendmail and configures it for Amazon SES if the configuration enables it. I have another one for periodic task management. It manages cron and the /etc/cron.d directory. The charm can accept files from the user via the attach-resource command. When a new file is detected, the integration charm installs to /etc/cron.d.
Charm layers sit on top. They pull everything together to complete the application.
This three layer structure means that I have less maintenance to do for charms like MariaDB and Galera MariaDB or Redis and Redis with Sentinel. The structure allows me to quickly create variants and reuse most of the code.
It also keeps the code a lot cleaner since there are only the Galera or Sentinel-specific code for the charm layers. I also have a few “global” actions that I usually use everywhere, so I only have to change it once in a base layer for it to apply to all my charms.
How do you recommend other developers get started with charming?
I would suggest getting started with a very simple charm written without frameworks and then move on to something like the reactive framework.
Writing a simple charm that responds to hooks makes it much easier to learn what Juju is doing. The documentation for getting started is very complete. Once you have written a few charms, you’ll find patterns.
I extended reactive mainly because I have logical paths that cross. It became very hard to manage using flags only. Doable of course, but not that clean.
What comes next?
A common problem to solve is managing configuration settings. Juju allows users to update configuration dynamically. So your charms will need to support this.
I use a “only change what you have to change” model by verifying the current state at all times before doing an action and correcting the situation. I usually end up just deleting whole configuration directories and rewriting them using templates and files. My charms make use of Jinja2 templates.
Relations are a very important concept to learn. Information comes from other applications. That information can be used to update your application’s current settings. So in effect, the whole system can configure itself.
I have also made good use of Juju’s attach-resource feature to increase automation. I use it as the entry point for artifacts like source code. If the charm detects a change in the “code” resource, then we run a synchronized deployment. I have also made good use of Juju’s attach-resource feature to increase automation. I use it as the entry point for artifacts like source code. If the charm detects a change in the “code” resource, then we run a synchronized deployment.
Applications with multiple units—what I call “clustered charms”—are complicated. I’ve introduced the idea of a “cluster state” into my charms. Deployments are synchronized by using cluster states—managed under the hood by Juju interfaces—and can only switch to the next state if every cluster members are at their state or higher. This way all units can all go into maintenance mode, apply database modifications and remove maintenance mode all at the same time.
Database modifications are only run on the leader unit, the other units only wait on the leader. I also have error detection where if the leader breaks and changes to a new node during deployment, then the new leader will execute the code, but not the previous leader.
These are great lessons. It sounds like you have really enjoyed writing your framework.
It’s been fun. I’m planning on adding new features next year and start doing more production environments. I see Juju as an opportunity to remove root access to the servers. I won’t sleep well at night until the day no devs have root access to production!
Speaking seriously though – there should be no need to provide root access to production servers. Charms actions for common operations. If needed, tools like “juju scp” and “juju run” can be used so that developers can still run arbitrary commands if they need to.
I’m also planning on open-sourcing some parts of it and sending patches to add more features to currently existing interfaces next year. As time allows, I’ll try to integrate with the ecosystem a bit more. It will depend a lot on my planning, but I’m getting a devops engineer in my team, we’ll see how that goes.
Speaking from a security perspective, are you happy with Juju? Can you see anything that needs hardening from its defaults?
The thing I would personally love to see would be SSO integration directly on the controller. I imitate that through a synchronisation script that I have created. The script works with Okta and Juju to take public keys from Okta and install them on the right models. It also creates users with a register command and sends them an email so that they can get started. Anyway I think that SSO would be something most companies like to see support for.
That’s a great recommendation. And well done! You’ve answered all the questions I have. Thanks for your time, Marc.
No trouble at all. Thank you, Tim. And thanks to the Juju team!
Marc André Audet is an Information Security Expert at Absolunet, where he uses DevOps tools to help protect information assets. He’s been coding since age 11 and knows more about GDPR than he ever thought he would.
Absolunet is a North-American eCommerce agency who works with retailers, brands and distributors on their digital transformations – helping them bridge the gap between how they sell and how their customers buy and expect to interact. Their back of house IT operations make use of Juju.
Juju is an open source application modelling tool, developed and maintained by Canonical. Its primary focus is to simplify deployments and operations of complex software stacks by providing a model-driven, declarative framework that cooperates nicely with other devops tools. Juju is an open source application modelling tool, developed and maintained by Canonical. Its primary focus is to simplify deployments and operations of complex software stacks by providing a model-driven, declarative framework.
To learn more about Juju, visit its website.