This post is from the XebiaLabs blog and has not been updated since the original publish date.
10 Tips for Integrating Security into DevOps
Imagine a world where product owners, Development, QA, IT Operations, and Infosec work together, not only to help each other, but also to ensure that the overall organization succeeds. By working toward a common goal, they enable the fast flow of planned work into production (e.g., performing tens, hundreds, or even thousands of code deploys per day), while achieving world-class stability, reliability, availability, and security.
The Need for Force MultiplicationOne interpretation of DevOps is that it came from the need to enable developers' productivity, because, as the number of developers grew, there weren’t enough Ops people to handle all the resulting deployment work. This shortage is even worse in Infosec — James Wickett described vividly why Infosec needs DevOps:
The ratio of engineers in Development, Operations, and Infosec in a typical technology organization is 100:10:1. When Infosec is that outnumbered, without automation and integrating information security into the daily work of Dev and Ops, Infosec can only do compliance checking, which is the opposite of security engineering—and besides, it also makes everyone hate us.
1. Integrate security into development iteration demonstrations.Here’s an easy way to prevent Infosec from being a blocker at the end of the project: invite Infosec into product demonstrations at the end of each development interval. This helps everyone understand team goals as they relate to organizational goals, see their implementations during the build process, and gives them the chance to offer input into what’s needed to meet security and compliance objectives while there’s still ample time to make corrections.
2. Ensure security work is in our Dev and Ops work tracking systems.Infosec work should be as visible as all other work in the value stream. We can do this by tracking it in the same work tracking system that Development and Operations use daily so it can be prioritized alongside everything else.
3. Integrate Infosec into blameless post-mortem processes.Also consider doing a post-mortem after every security issue to prevent a repeat of the same problem. In a presentation at the 2012 Austin DevOpsDays, Nick Galbreath, who headed up Information Security at Etsy for many years, describes how they treated security issues, “We put all security issues into JIRA, which all engineers use in their daily work, and they were either ‘P1’ or ‘P2,’ meaning that they had to be fixed immediately or by the end of the week, even if the issue is only an internally-facing application.
4. Integrate preventive security controls into shared source code repositories and shared services.Shared source code repositories are a fantastic way to enable anyone to discover and reuse the collective knowledge of the organization, not only for code, but also for toolchains, deployment pipeline, standards—and security. Security information should include any mechanisms or tools for safeguarding applications and environments, such as libraries pre-blessed by security to fulfill their specific objectives. Also, putting security artifacts into the version control system that Dev and Ops use daily keeps security needs on their radar.
5. Integrate security into the deployment pipeline.To keep Infosec issues top of mind of Dev and Ops, we want to continually give those teams fast feedback about potential risks associated with their code. Integrating security into the pipeline involves automating as many security tests as possible so that they run alongside all other automated tests. Ideally, these tests should be performed on every code commit by Dev or Ops, and even in the earliest stages of a software project.
6. Protect the deployment pipeline from malicious code.Unfortunately, malicious code can be injected into the infrastructures that support CI/CD. A good place to hide that code is in unit tests because no one looks at them and because they’re run every time someone commits code to the repo. We can (and must) protect deployment pipelines through steps such as:
- Hardening continuous build and integration servers so we can reproduce them in an automated manner
- Reviewing all changes introduced into version control to prevent continuous integration servers from running uncontrolled code
- Instrumenting the repository to detect when test code contains suspicious API calls