Last Updated Feb 08, 2017 — DevOps Expert
Building a Winning DevOps Team
DevOps
DevOps, and Ops in general, is one of the most important components of a company's production machine. Investing in building a winning DevOps team should be a priority for every company and will pay off quickly.
As organizations and operations grow (products, features, number of customers, etc.), the scale of traffic, data, OS types and so on, increase rapidly. More products to support, new features to release and a lot more in-house cycles (e.g., QA, CI\CD, SME, SRE) need to be managed. A DevOps team's first mission is to build, maintain and support the company’s technology stack. Whether it’s a PaaS, SaaS or IaaS environment, microservices or monolithic, docker or K8S, it does not matter--DevOps engineers need to be able to support any environment the company chooses at any time and anywhere (On Premises, Data Centers, Cloud). Here are the subject areas any organization and it’s DevOps engineers have to deal with:The list goes on, and every item is its own area of knowledge. It's “a small world” of expertise, and any DevOps engineer is expected to master it all (or at least a large subset of it ¯\_(ツ)_/¯). For example, let's say your company is building the next edge, top of the line API serving tool. The tool is web based and expected to serve TB of daily transactions. TB of events pounding your chosen big data tools, logging everything for sanity, monitoring everything so you’ll be able to wake anyone up when necessary, profiling, ingesting, segmenting…It’s growing and doing so exponentially. Recruiting and finding DevOps engineers to support this environment is challenging. Finding someone who understands all the aforementioned subjects is rare, and if you do find that person, they'll cost you accordingly. (Let's assume money is not an obstacle right now and focus on finding the right engineer to fulfill, create and maintain your environment.)A Six Step Process
My strategy is divide and conquer. Let's break down and define what we're looking for. Defining the interview process is crucial for successful hiring. A good process will result in your finding a quality candidate (technical, personal and cultural) and will help maximize the candidate’s compatibility with the company and role. Throughout my years of experience, I’ve defined a 6 step process that helps me form and create winning engineering teams:- Step 1 - Define the role, and sync with the company recruiter.
- Step 2 - Phone screen.
- Step 3 - Break down the skills to interview modules.
- Step 4 - Prepare questionnaire for each module.
- Step 5 - Form your interviewer squad and begin interviewing.
- Step 6 - Closer. All interviewers meet and decide.
Step 1. Define the role, and sync with the company recruiter.
The company recruiter will be your liaison to the candidate and will have the first interaction with them. It’s important to follow these steps:- The recruiting manager must have a conversation with the recruiter and make sure they understand the job description and the role's technical requirements. Talk with them, explain yourself, explain the role and expectations. Don't skip this stage. If you do, you'll end up wasting everyone's time in the long run.
- The recruiter should have a phone conversation with the candidate. The purpose of this conversation is to understand if the candidate is a good fit for the company and how the position fits within the company culture, the candidate's career development, and so on. This call also involves verifying the content of the candidate's CV.
Step 2. Conduct a phone screen interview.
Phone screen interviews are very helpful and significant time savers. Since the process is time consuming and involves numerous engineers interviewing the candidate, it’s important your time is well spent. You should respect the candidate’s time as well. After posting\advertising the job, people will start applying. Your time is precious so treat it accordingly! Even if the CV is appealing and it seems like the candidate is a perfect match, you still have to set up two phone calls, each being no more than 15-20 minutes:- Recruiting Manager Interview - The recruiting manager needs to talk with the candidate to understand if the person is qualified and is a good fit for the team and company culture. They should also use the time to find out whether the candidate thinks the role meets their career aspirations, as well as validate the candidate's CV.
- Technical Interviewer - This can be any existing DevOps engineer or software engineer who can ask a series of easy, medium or hard questions related to the posted position and candidate's background.
Step 3. Break down the skills to interview modules.
Interview modules represent a subject to discuss during the interview. For example:- System + Networking
- Methodologies\Processes - Agile, CI\CD, etc.
- Software Engineering, Architecture Design
- Problem solving, debugging, monitoring
- Culture fit
- Practical\Code challenge
Step 4. Prepare a questionnaire for each module.
For each module, prepare a questionnaire that applies to different skill levels (junior, intermediate, senior and expert). This is best organized in a question and answer format as the answer section will be helpful in situations where the answer differs from what is expected.System + Networking module example:These questions will assess a candidate's skills and knowledge in areas of your specific flavor of operating systems and scalable networking. It is very important that these areas are well understood to allow for effective troubleshooting and competency in a technical environment. Lacking this basic knowledge can result in a long learning curve.- How does traceroute command work?
- What is ARP?
- What is jitter/latency/packet-loss-ratio?
- What is proxy ARP?
- How do you set up a TCP connection?
- Etc.,
You are to architect a new email system capable of supporting an expanding user base with low delivery times. The incoming mail throughput is expected to be around a million emails per minute.
- Describe in detail or draw the mail system you would implement.
- What software choices would you make for each layer and why?
- What are your kernel, networking, storage and monitoring considerations?