Recently, Team Carma’s Teresa Gaynor challenged everyone, both here at Carma HQ, and our hard working colleagues out in the field, to write a blog. It didn’t matter if you were an engineer, a code monkey, a marketeer, or an accountant, just write.
While not everybody, cough*Gearoid*cough, managed to meet the challenge, we got a number of great submissions, some written in good ol’ Word, some that could only be opened with a code editor, some wrote poetry, and some wrote gibberish for which the non-coders couldn’t find the keys for on the keyboard. So for the next little while, we’ll be pushing out some of our submissions form the great Carma Blog-off.
—
As I have been tasked with the actual posting of the blogs and the pretty-fi-ing of the content, it is unfortunate that I understand very little of what I am reading below. Taking a quick poll around the marketing team as to what all this could be about, I’ve decided that robots are a pretty good guess.
– Nathan Richardson
### Overgeneralization is bad for playbooks
So I can write Ansible playbooks now, nice.
Now I want to write good Ansible playbooks.
One element of that is to avoid generalization. Let’s see why.
### The setting
Say we need to run a webapp with a mongodb backend. The container and mongodb run with the same user.
We want a playbook that installs dependencies, unrolls the webapp tarball, configures the webapp, will restore data files from a backup and can be used for verification of the environment. Ideally, the app will handle rolling upgrades of the webapp. A good playbook will do that.
Everything is pretty straightforward with the container and the webapp itself: setup once, the contents will only change when we unroll a new webapp version or upgrade the environment. Perfect fit for tight access controls. Yay.
Now that’s not the case for the variable content: running webapp, workdir, tmpdir, and database files etc.
The directories used by mongodb look something like this:
db files:
{{ webapp_data_dir }}
{{ webapp_data_dir }}/db # <- mongodb data goes here
{{ webapp_data_dir }}/otherstuff
logs:
“{{ webapp_logs_dir }}”
“{{ webapp_logs_dir }}”/mongod.log
“{{ webapp_logs_dir }}”/server.log
### Doing it in one step – poorly
Procedural thinking will get you something like this: “change owner, change ownership, and do it recursively”
– name: webapp data directory exists with proper permissions
file: state=directory
path={{ item }}
owner={{ webapp_user }} group={{ webapp_group }}
mode=0750
recurse=yes
with_items:
– “{{ webapp_data_dir }}”
– “{{ webapp_logs_dir }}”
This is concise, but wrong. This is just a replacement for an installation script (part of, anyway).
### One problem: overgeneralization
The “with_items” is fine, but it is overused here. The grouping of configuration items should be done only if they share exactly the same configuration, and are part of the same configuration items. Database files and logging? Not the same thing.
An example: it turns out we need to run the webapp on a SELinux enabled machine. OK, I’ll just set the proper selinux context for the directory. But for this, I need “setype=mongod_var_lib_t” for data and “setype=mongod_log_t” for logs. Now I need to split that rule. And I will still apply selinux labels for files that don’t need those labels.
All because I lumped things together that are not the same.
## Another problem: idempotent module != idempotent task
Let’s start up the webapp at least once. After that, the playbook will happily change the permission bits to the proper “0750” for every file that mongodb created. That itself may not break the app this time, but will certainly break verification – by running “ansible-playbook –check”, we will see that this item is changed.
A false positive for verification – not nice.
Coincidentally changing the system state – worse.
### Doing it in many steps – better
– name: webapp data directory base exists with proper permission bits
file: state=directory
path={{ webapp_data_dir }}
owner={{ webapp_user }} group={{ webapp_group }}
mode=0750
– name: webapp data directory contents are owned by the webapp daemon user
file: state=directory
path={{ webapp_data_dir }}
owner={{ webapp_user }} group={{ webapp_group }}
recurse=yes
– name: webapp mongodb directory base exists with proper permission bits
file: state=directory
path={{ webapp_data_dir }}/db
owner={{ webapp_user }} group={{ webapp_group }}
mode=0750
– name: webapp mongodb directory contents are owned by the webapp daemon user
file: state=directory
path={{ webapp_data_dir }}/db
owner={{ webapp_user }} group={{ webapp_group }}
recurse=yes
– name: webapp mongodb dir has proper SELinux security context set
file: state=directory
path={{ webapp_data_dir }}/db
seuser=system_u
serole=object_r
setype=mongod_var_lib_t
recurse=yes
– name: webapp logs dir base exists with proper security permissions
file: state=directory
path={{ webapp_logs_dir }}
seuser=system_u
serole=object_r
setype=mongod_log_t
owner={{ webapp_user }} group={{ webapp_group }} mode=0750
No overgeneralization: now I am only setting the needed configuration on the needed items.
The play is idempotent now. It is enforcing only the needed configuration, doesn’t go around setting unneded things.
### My DOs and DON’Ts
The playbook describes the end result. If I’m too much concerned about HOW TO get there, I’m doing something wrong. I’m writing a script, whereas I should be writing a blueprint.
Though, inevitably, there will be procedural steps in my playbooks – I must make sure they run only when needed.
Moving as many procedural steps as possible will generally make more sense.
If I cannot move a procedural step to a handler – I write a proper script that will make for an idempotent task. It gives proper output to the stdout, and the task uses that output: it registers a variable, and check with “change_when” for changes.
I try not to overgeneralize: when generalizing (using “with_items” is a good marker), I ask myself the question: “are these the same configuration items?” If the answer for the question is “yeah, they are pretty much alike” – I break the task into several tasks. Will save me a lot of headache later on.
I am wary about tasks with many lines: am I overgeneralizing again, trying to shoehorn different configuration items into the same task?
To myself: don’t optimize! Optimization is error prone and hard to debug. It is completely uneccessarily.
Don’t shoehorn functionally different things into one task.
### Accidentally, I can backup/restore/migrate data properly now!
After doing this properly, we can even unroll a backed up state of the data directory, and everything will work nicely after that, because the steps above ensure proper permissions. Oh yes.
Just inserting this step before the permission setup tasks will do the trick.
– name: webapp database backup is unrolled
unarchive: src={{ webapp_database_backup_file }}
dest={{ webapp_base_dir }}
copy=yes
creates={{ webapp_data_dir }}
when: webapp_database_backup_file is defined
Now this one is extremey useful for many things:
– migrating the server
– testing rolling updates (preseed test server with mongodb, etc)
– disaster recovery
– actually restoring of backups (duh)