Configuration management does many things, and it does most of those quite poorly. I could write a long essay on how by solving software problems with more software we’re simply creating more software problems, however I will attempt to resist that urge and instead focus on how Docker and the Docker ecosystem can help make configuration management less sucky.
Ignoring volume mounts (which are an abomination for which I hold @gabrtv wholly responsible for), Docker has two main ways to configure your application: firstly by creating the dockerfile in which you explicitly declare your dependencies and insert any configuration files, and secondly at run time where you pass commands and environment variables to be used inside the container to start your application.
We’re going to ignore the dockerfile here and assume that you have at least a passing familiarity with them; instead we’re going to focus on how to configure your application at run time.
A true Docker Native app would have a very small config file of which some or all settings could be overridden by environment variables or CLI options that can be set at run time to modify the appropriate configuration option (say, pointing it at a MySQL server at 10.2.2.55 ).
Very few applications are written in this way, and unless you’re starting from scratch or are willing to heavily re-factor your existing applications you’ll find that building and configuring your applications to run in “the docker way” is not always an easy or particularly pleasant thing to have to do. Thankfully there are ways to fake it.
To save writing out a bunch of CLI arguments the cleanest ( in my opinion ) way to pass values into docker containers is via environment variables like so:
$ docker run -ti --rm -e hello=world busybox env PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin HOSTNAME=8cb5546f1ec4 TERM=xterm hello=world HOME=/root
We can then write an application to read that environment variable and use it as a configuration directive like so:
#!/bin/sh echo hello $hello
No prizes for guessing our output, we run a docker container from the image containing this script:
$ docker run -ti --rm -e hello=world helloworld hello world
Now this was a pretty asinine demo, and apart from showing how passing environment variables into a docker container works doesn’t really do anything useful. Let’s look at a slightly more realistic application. Take a python app that reads a configuration file and when asked renders a web page using the contents of that configuration file:
note: these examples are abbreviated sections of the example factorish app.
example.py
import ConfigParser
import os
from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello():
Config = ConfigParser.ConfigParser()
Config.read("example.conf")
return 'Luke, I am your {}'.format(
Config.get("example", "text"))
if __name__ == '__main__':
app.run(host='0.0.0.0', port=80)
example.conf
[example] text: father
Now when we run this application we get the following:
$ docker run -d -p 8080:8080 -e text=mother example $ curl localhost:8080 Luke, I am your father
Obviously the application reads from the config file and thus passing in the environment variable `text` is meaningless. We need a way to take that environment variable and embed it in the config file before running the actual `example.py` application. Chances are the first thing that popped into your head would be to use `sed` or a similar linux tool to rewrite the config file like so:
run.sh
#!/bin/bash
sed -i "s/^text:.*$/text: ${text}" example.conf
exec gunicorn -b 0.0.0.0:8080 app:app
Now we can run it again with run.sh set as the starting command and the config should be rewritten.
$ docker run -d -p 8080:8080 -e text=mother example ./run.sh $ curl localhost:8080 Luke, I am your mother
This might be fine for a really simple application like this example, however for a complicated app with many configuration options it becomes quite cumbersome and offers plenty of opportunity for human error to slip in. Fortunately there are now several good tools written specifically for templating files in the docker ecosystem, my favourite being confd by Kelsey Hightower which is a slick tool written in golang that can take key-pairs from various sources ( the simplest being environment variables ) and render templates with them.
Using confd we would write out a template file using the `getv` directive which simply retrieves the value of a key. You’ll notice that the key itself is lowercase, this is because confd also supports retrieving key-pairs from tools such as etcd and confd which use this format. When set to use environment variables it is translated into reading the variable “SERVICES_EXAMPLE_TEXT”.
example.conf
[example]
text: {{ getenv "/services/example/text" }}
We would accompany this with a metadata file that tells confd how to handle that template:
example.conf.toml
[template]
src = "example.conf"
dest = "/app/example/example.conf"
owner = "app"
group = "app"
mode = "0644"
keys = [
"/services/example",
]
check_cmd = "/app/bin/check {{ .src }}"
reload_cmd = "service restart example"
The last piece of this puzzle is a executable command in the form of a shell script that docker will run which will call confd to render the template and then start the python application:
boot.sh
#!/bin/bash
# read 'text' env var and export it as confd expected value
# set it to 'father' if it does not exist
export SERVICES_EXAMPLE_TEXT=${SERVICES_EXAMPLE_TEXT:-"father"}
# run confd to render out the config
confd -onetime -backend env
# run app
exec gunicorn -b 0.0.0.0:8080 app:app
Now let’s run it, first without any environment variables:
$ docker run -d -p 8080:8080 --name example factorish/example $ curl localhost:8080 Luke, I am your father $ docker exec example cat /app/example/example.conf [example] text: father
As you can see the server is responding using the default value of `father` that we set in the export command above. Let’s run it again but set the variable in the docker run command:
$ docker run -d -e SERVICES_EXAMPLE_TEXT=mother -p 8080:8080 --name example factorish/example $ curl localhost:8080 Luke, I am your mother $ docker exec example cat /app/example/example.conf [example] text: mother
We see that because we set the environment variable it is be available to `confd` which renders it out into the config file.
Now if you go and look at the full example app you’ll see there a bunch of extra stuff going on. Let’s see some more advanced usage of confd by starting a coreos cluster running etcd in Vagrant. etcd is a distributed key-value store that can be used to externalize application configuration and retrieve it as a service.
$ git clone https://github.com/factorish/factorish.git $ cd factorish $ vagrant up
This will take a few minutes as the servers come online and build/run the application. Once they’re up we can log into one and play with our application:
$ vagrant ssh core-01 core@core-01 ~ $ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES ee80f89d2565 registry "docker-registry" 25 seconds ago Up 25 seconds 0.0.0.0:5000->5000/tcp factorish-registry c763ed34b182 factorish/example "/app/bin/boot" 52 seconds ago Up 51 seconds 0.0.0.0:8080->8080/tcp factorish-example core@core-01 ~ $ docker logs factorish-example ==> ETCD_HOST set. starting example etcd support. 2015-11-08T21:35:16Z c763ed34b182 confd[23]: INFO Target config /app/example/example.conf out of sync 2015-11-08T21:35:16Z c763ed34b182 confd[23]: INFO Target config /etc/service/confd/run out of sync 2015-11-08T21:35:16Z c763ed34b182 confd[23]: INFO Target config /etc/service/confd/run has been updated 2015-11-08T21:35:16Z c763ed34b182 confd[23]: INFO Target config /etc/service/example/run out of sync 2015-11-08T21:35:16Z c763ed34b182 confd[23]: INFO Target config /etc/service/example/run has been updated 2015-11-08T21:35:16Z c763ed34b182 confd[23]: INFO Target config /etc/service/healthcheck/run out of sync 2015-11-08T21:35:16Z c763ed34b182 confd[23]: INFO Target config /etc/service/healthcheck/run has been updated echo ==> example: waiting for confd to write initial templates... 2015-11-08T21:35:16Z c763ed34b182 confd[23]: ERROR exit status 1 Starting example *** Booting runit daemon... *** Runit started as PID 51 2015-11-08 21:35:22 [56] [INFO] Starting gunicorn 0.17.2 2015-11-08 21:35:22 [56] [INFO] Listening at: http://0.0.0.0:8080 (56) 2015-11-08 21:35:22 [56] [INFO] Using worker: sync 2015-11-08 21:35:22 [67] [INFO] Booting worker with pid: 67 core@core-01 ~ $ curl localhost:8080 Luke, I am your father
You can see here that we’ve started the example app, but notice at the top where it says “starting example etcd support”. This is because we’ve actually started it with some environment variables that makes it aware that etcd exists. It uses these to configure `confd` to run in the background and watch an etcd key for templated config value.
We can see this and modify the config setting using etcd commands:
core@core-01 ~ $ etcdctl get /services/example/text father core@core-01 ~ $ etcdctl set /services/example/text mother mother core@core-01 ~ $ curl localhost:8080 Luke, I am your mother core@core-01 ~ $ exit $ vagrant ssh core-02 core@core-02 ~ $ curl localhost:8080 Luke, I am your mother
With confd aware of etcd it is able to notice values being changed and react accordingly, in this case it rewrites the templated config file and then restarts the example application. If you look at the template’s metadata from earlier you’ll see it is instructed to watch a certain key and rewrite the template if it changes. It also has two directives `check_cmd` which is used to ensure the created template would be syntactically correct and `reload_cmd` which it runs any time the template is successfully written, in this case to reload our application.
You’ll also notice that we were able to connect to the other coreos nodes each of which was also running the example application and because etcd was clustered across the three nodes all three applications registered the changed and updated themselves.
So now, not only do we have good clean templating in our container, we also even have the ability to change some of those config settings on the fly by connecting it to etcd.
From this very simple building block we are only a short hop away from being able to automatically configure complicated stacks that react to changes in the infrastructure instantaneously.
Pretty cool huh?
This article is part of our Docker and the Future of Configuration Management blog roundup running this November. If you have an opinion or experience on the topic you can contribute as well!
