While Config Management isn’t new as a concept, it is new for a lot of sysadmins. Many have started using it in the last year or two and have invested in workflows, modules, re-usability, … to keep their systems up-and-running. Yet, at the same time, there’s a disturbing trend I see emerging of automating something you don’t fully understand.
I see this in 2 variants: trying to automate a service someone doesn’t fully understand, and blindly copying modules and code, implementing it on systems and not understanding what goes on under the hood.
Automation = knowledge + insight + a goal
Whenever you write a Puppet module or a Chef Cookbook, you’re essentially writing in code what you think is the best way of implementing, managing and securing a service or role. You’re pouring your years of experience, your knowledge on the subject, your insights into the technology and your long-term goal into that module. Your module / cookbook / … should reflect your view of how that service or role should work.
That’s only possible when you truly master said technology. You can automate a piece of technology you do not understand, but you shouldn’t. It creates technical debt from day one and provides the foundation for more bad practices. It’s a downward spiral that’s hard to get out of, because chances are you’re using that piece of code in production.
It can happen for a variety of reasons, deadlines and management pressure would be the most common. You make the choice of getting the job done and you make it work. Short-term, the automation pays of: you’ve got something up-and-running. Long-term, you’re stuck with a badly implemented technology and a dependency on the code you’ve just written, with no time to undo your quick hacks. Classical technical debt.
The reality is, automation takes time. A lot of time, if you want it done right. Not the automation in itself, but understanding the technical details, the shortcomings, the workarounds and the best practices of a particular technology. And you can not write a module or cookbook for any piece of technology, if you do not master it.
Using unknown automation
So if you lack time to master technology, surely you can just grab a module or cookbook from the forge or the Supermarket and be done with it, right? Let other smart people write the modules with their years of experience and knowledge. You just implement it and benefit from their work.
This work, up until a certain point. If you just take an already written module and use it, do you understand what that module is managing? Do you know the internals and how to debug it? How will you react when your service is slow to respond or offline? Do you know where to look and where to apply the fix?
Implementing modules that others have written, for technology you do not master, is like following a guide on the internet with step-by-step commands. It’ll get the job done, but you won’t know why certain steps are taken. You won’t know why things are done the way they’re done. And when you really need it – when your service crashes – you won’t know where to look. Because the internals are a black box.
It takes commitment
And I don’t mean a git commit. Building a config management infrastructure, writing and managing modules and implementing them in a correct way takes serious commitment. It takes time, effort, knowledge and motivation. It means caring about code quality.
While it’s a gut feeling, I dare you say you shouldn’t try to automate a service if you haven’t used for a few years “manually”. If you’ve not had to debug it yet, have experienced the shortcomings of a typical “first time deployment” and not had your lessons learned, it’s better not to automate it in the first place. It won’t bring you any benefits (do you need repeatability of a bad setup?) and will only slow things down (how do I migrate my legacy module to this new and improved way of managing I just came up with?).
Truth be told, config management isn’t easy.