The deployment-sized hole between Continuous Integration, Continuous Delivery and Provisioning tools
As a developer and "honorary" (nobody else wanted to do it) build engineer, I was always dissatisfied with the amount of glue I had to apply to stick our Continuous Integration or Continuous Delivery tool, whether Jenkins, Bamboo or Go (XL Release wasn't around at the time ;-) ) to our provisioning tool (Puppet, Chef or the like), which we also used for the application tier. There was simply a fundamental impedance mismatch between the two types of tools:
- We needed an application-oriented model for our services, which consisted of multiple components ("traditional" or micro-service style) spread across various machines. The provisioning tools' domain models all felt too machine-centric.
- We were looking for clean and easy integrations with our build, packaging and Continuous Integration/Continuous Delivery tool of choice. Since there often weren't any, we ended up with a bunch of custom tasks and script steps.
- Our pipelines were - and still are - based on "directive automation", i.e. we invoked whatever tool(s) were needed for a step, kept track of the process and status and continued (or not) once it had completed. This did not match the "eventually consistent" style of the server mode of tools like Puppet or Chef at all well. That mode was great from an operational perspective ("Make sure all my x-thousand machines have the right configuration - I don't need to watch it happen, just get it done!"), but not really compatible with our delivery pipeline. We sometimes used a type of polling approach (Urgh! And brittle, too), but often ended using the single-node model (chef-solo or standalone puppet) for our pipeline. That worked, but didn't seem very much like the way the tools were ideally supposed to be used.
- For the deployment steps in our pipeline, we needed to orchestrate actions across multiple nodes, respecting dependencies but also parallelizing where possible for efficiency. This resulted in a lot of glue scripting, both to describe which kind of orchestration order was required, as well as to actually carry it out.