Embracing constraints

The shell environment is usually regarded as nothing more than a simple prompt for passing our keyboard input to system services that do the heavy-lifting. Peel back some layers and you will discover a functional programming language with the most pure event loop. Every key press is an event waiting to happen, wanting to make your command-line usage more efficient, more helpful.

Simple primitives - building blocks - that you can combine in endless ways using the pipe operator open this environment to endless combinations. Every command has 3 streams: stdin or fd0, stdout or fd1 and stderr fd2. The pipe operator is able to connect these data streams between utilities and give you the power to combine, slice and filter the data that passes through them to your exact taste.

Do you need to upcase a string?

echo "upcase me" | tr [a-z] [A-Z]

How about stopping all redis-server processes?

ps -ef pid,command | awk '/redis-server/ && ! /awk/ { system("kill " $1) }'

Opening a remote script inside vim’s view mode for inspection before trusting it to run on your machine, as yourself?

curl -o- https://raw.github.com/mxcl/homebrew/go | view -

Your shell is able to automate every command or set of commands and automate all repetitive operations. The utilities are there, all dusty, treasures waiting to be discovered. When you forgot the crontab file syntax, your shell has the answer: man 5 crontab. And if you want to search all local documentation for cron, don’t run to Google, just run man -k cron in your shell.

What about more complex utilities, like a fully functional deployment tool? All programmers have applications to deploy ranging from static websites to PHP, to Ruby, to Python and even node.js. How many deployment tools do you think are out there? Hundreds! They are built in almost every possible programming language. Why?

Code deployment is orthogonal to programming, yet programmers will keep writing their own deployment tool, in their favourite programming language, and force it across all projects they encounter. node.js app? Deploy it with Capistrano! Clojure? Deploy it with Fabric! Ruby app? Deploy it with Chef or Puppet! The order is random, as are the deployment tools, but this illustrates the madness of the landscape. Wouldn’t it be nice if there was a single, generic tool with no dependencies other than your shell environment that could get any app, developed in any language, into your favourite *nix production environment with no other requirements other than your shell and an ssh connection? Well, now there is: deliver

How about a pure bash Continuous Integration server? Jenkins is the darling, yes, but have you noticed how it hogs memory? As for the worst part, nothing beats the UI. Why do we accept this? Is that the best we can do? Travis? Come on, have you tried setting it up in your own environment? The hosted version won’t cut it as the vast majority of apps aren’t open source. Private builds you say? Maybe, but how do you do Continuous Delivery in a private cloud? Can you build Docker containers part of your CD cycle? With enough imagination, you can do anything, but at what cost?

Leaving CI tools aside, have you seen the craze that configuration management and system provisioning has become? Learning curves are bell-shaped, simple tasks have become week-long fights in the name of flexibility and adaptability. This is coming from someone that has run a few Chef-managed infrastructures over the years and cannot believe the mental strength required to juggle all components, in particular secrets and service configuration. When did simplicity become a second-class citizen? When did the one-step, the now, become the many steps and someday?

Don’t be afraid to embrace constraints, push tools to their limit, favour simplicity over anything else and believe that today will always be more important than tomorrow.


Now read this

Ansible & Docker - The Path to Continuous Delivery I

If I had a Rails application requiring MySQL and Redis that I wanted to host myself, this is the quickest and most simple approach. There are just 2 dependencies: Ansible & Docker. To make the introductions: Meet Ansible, a system... Continue →