Hire the Top 3% of Freelance DevOps Engineers

What is a DevOps engineer? Clearly, there’s no one-size-fits-all answer to the question: Skimming DevOps engineer vacancies from just a few companies is enough to show how diverse the role’s requirements can be.

A DevOps specialist should be able to mold their skills into the unique mechanics of a company’s IT processes. A candidate is usually expected to be fluent in deployment automation and networks, understand software licensing, be wise in the ways of hardware and software, work with multiple operating systems, and write scripts in various languages to build reports and automate tasks. They will have to continuously communicate with teammates in order to identify and prioritize tasks and excel at documenting and demonstrating the tools they create.

Assessing so many areas of technical knowledge and communication skills in a single person is tricky. That is where this guide comes in: We’ll tell you what you need to know in order to successfully hire a DevOps specialist, give you example DevOps interview questions, and highlight the skills a candidate should be expected to demonstrate.

Let’s start with a fundamental concept that every DevOps specialist should be fluent in nowadays.

Virtualization and Sandboxing

Understanding virtualization is essential for all DevOps candidates. Many contemporary ideas and strategies in the DevOps profession were built on top of it, and are at work in many companies no matter their structure or purpose.

The idea behind virtualization is to abstract away the underlying hardware or software and make things less dependent and more isolated. It’s actually a broad concept and applies to servers, networks, storage, desktops, and even applications.

For example, a server operating system (OS) with all services and tools installed can be configured the same way as on a company’s real, bare-metal server, but instead be put in a virtual machine, or VM. Then VMs can be launched on developer laptops to mimic the same real structure and relationship between services and tools as in the server environment, even if the developers use different laptops for their work. This has become especially easy today with virtualization tools like Vagrant and lightweight container management systems like Docker.

Another example is how a complex network with multiple resources can be hidden behind a few virtual layers, then partitioned into manageable parts.

The same goes for storage virtualization: Multiple physical storage devices are virtually “joined” and appear as a single storage unit, and then they can be partitioned further as the company needs.

Finally, application virtualization will let the same applications run encapsulated on multiple operating systems.

It is important that the candidate can explain the key benefits of using virtualization:

• Greater control of IT resources
• Faster setup and configuration of IT infrastructure
• Easier sandboxing
• Cheaper experimenting
• Faster release and recovery times
• Better isolation, thus higher security
• Lower risks brought by changes
• Better testability of software and services

While it might not be required for the candidate to list all of the points, understanding why to use virtualization and where it can be implemented is essential. Without it, a DevOps engineer will be very limited in their possibilities and won’t be able to propose efficient solutions.

Besides the why, the how component is what will allow you to distill whether they’re a truly skilled person—make sure some of the points are demonstrable by asking the candidate about how they actually saw these benefits in an implementation they’ve done themselves.

With virtualization comes cheap and efficient sandboxing. To sandbox means to put a piece of software, like a running program, in an isolated box, and control what resources it can access and when.

Sandboxing is very useful for both security and testing. When it comes to security, this process can stop the spread of software vulnerabilities or system failures. In testing, it is a great way to test software before shipping it to live servers. In a sandbox, you can emulate various corner cases and test various user input, see how a program performs when overloaded with multiple simultaneous requests, automate Quality Assurance testing, and so on.

At the end of the day, virtualization and sandboxing allow you to develop faster, roll out new features more often and more seamlessly, and adapt to the constantly changing company course. Eventually they save money by reducing many unavoidable risks, and by automating plenty of routine processes.

Useful terminology includes virtual machine, host and guest operating system, container, and hypervisor. The candidate should be able to explain these.

Example interview questions:

• What tools exist to implement virtualization? What are key players in the market today?
• What would be impossible without using virtualization? What are the consequences?
• What is the difference between VMWare and Docker?
• How can you standardize the environment for developers so that they work with software as if it were on your live servers?
• How can you release more often and be able to roll back in minutes, not in hours?
• How do version control systems (VCSes, e.g. Git) work together with virtualization?

The last question from the list above is especially important. The idea here is that the VM configuration should be tracked in a VCS together with the project’s code. This way the correct configuration for a virtual machine can be reproduced for any given state of code as committed at that time. Tracking the application code together with the configuration of the system it’s running on makes it possible to reproduce bugs related to VM configuration and versions of OSes, packages, and libraries.

Now that we’ve checked some essential basics, let’s see what a DevOps specialist should understand about today’s hardware, including cloud computing.

Cloud Technologies and Hardware

Over are the days when owning hardware in order to run an IT business was in many cases a necessity. Nowadays, companies like Amazon and Google maintain hardware and rent it out, taking monthly payments that are a mere fraction of the original hardware cost.

Renting hardware instead of buying it is a popular way to go for many businesses today. It eliminates lots of hardware setup and maintenance tasks by outsourcing them to dedicated teams. This also speeds up launching a business, or building a proof-of-concept product, or providing clients with instant dedicated trial setups. It also allows you to focus on the business logic straight away instead of spending time on setting up your own hardware.

A company can even build their own, internal cloud, by joining their hardware into a single network and abstracting over it. Hybrid colocation-cloud solutions are also popular. One way or the other, cloud computing will be present in the day-to-day job of a DevOps specialist. That is why assessing their relevant knowledge and skills will help you make sure your company won’t miss on the speed, flexibility, and scale that cloud computing offers.

To start with, ask the candidate about the benefits that cloud computing brings, like:

• Reduced setup and maintenance cost
• Ability to pay for what you use only, and add more resources as you go
• Automatic software updates by suppliers
• Robust disaster recovery

Even though the question might be less practical and more philosophical, it is always good when people have a general understanding of the technology and its goals. If this is the case, they can learn faster when needed, and, having a broader understanding, will have more intelligent applications of existing technologies to your business.

When it comes to introducing cloud computing to the company’s IT infrastructure, different service models are proposed by providers. Have your DevOps candidate say a few words about the differences between the standard models, and give examples:

• Infrastructure as a service (IaaS)
• Platform as a service (PaaS)
• Software as a service (SaaS)

It’s also worth asking about their experience using any of these models.

Next, discuss your business processes in the context of those models. For example, if your company sends emails heavily, can it use a SaaS solution like the Amazon Simple Email Service, or should it set up its own service hosted using third-party IaaS? What would be the difference? What are the limitations of both? How easy will it be to transition from one model to another? Will there be any lock-in?

Another example is implementing an internal project management or a public user support system. In both cases, there are many solutions available on the market—you’ll rarely want to build your own from scratch. Some of them are hosted and maintained by providers, and DevOps will take part in integrating it with the other software used by the team. Others should be downloaded, installed, configured, and updated regularly, which is also up to the competence of the DevOps team.

Therefore, you should expect the candidate to effortlessly answer questions like these:

• What is a cloud service API, which pricing models exist, and what is an API usage quota?
• Which of IaaS, PaaS, and SaaS are applicable to our company? (This implies the candidate read about your company and its goals before submitting their application.)
• What are the risks of renting hardware instead of owning it?
• What are the risks of using your own hardware, and when is it the only available choice? (Hint: Unreliable internet service is a prime example.)
• In which cases will buying your own hardware benefit the company?
• Which compliance issues are related to using cloud services, and how can you check whether a given service is compliant with company policies, standards, and existing certifications?

Because DevOps might alter the company infrastructure with time, it’s important that they understand standards compliance and certification. If your products or services are regulated in the company’s country of residence, DevOps will need to cooperate with your Information Security Engineer to maintain existing compliance or apply for new compliance. If this is the case, let the candidate talk about their experience in the field: How they accomplished compliance and what their role was in the process.

Here is an idea for a quick practical task that will help you assess the DevOps candidate’s knowledge in compliance and certification. If you previously applied for any, you will have lots of application forms filled out. Take an excerpt from one of them and remove all the answers, and give it to the candidate. Let them:

1. Explain the meaning of the questions, and
2. Tell you the questions they will ask the infrastructure, security, and developer departments in order to get the information needed to fill out the questions in the form.

Even if you have a dedicated compliance specialist, chances are that DevOps will be heavily engaged in both compliance application and validation. If the DevOps candidate lacks understanding in those, you are risking your compliance status.

Back to the hardware topic, knowledge about it pays off. One of the ways to unlock full software potential is by using the right hardware and combining it in the right way. For example, look at how Stack Overflow served 150 million HTTP requests in 2013 and 210 million in 2016. In the hardware list, you’ll notice that four Microsoft SQL Servers were used in 2013, and there are still only four of them in 2016. The number of the database servers did not increase, and only two of the servers got upgraded. This means that there was an intelligent plan behind the scenes. Not only does it save money on hardware directly, but also on hardware installation, colocation, and maintenance.

So, depending on the business processes in your company, you may want to ask the DevOps candidate some of the following questions:

• Which types of software require more of the following?
  • Processing power (CPU speed)
  • CPUs or cores (for parallelization)
  • GPUs
  • Computer memory
• Give an example of redundancy in hardware setup. (You can use the Stack Overflow article as a guide to familiarize yourself with one possible use case.)
• What is load balancing and when it is needed?

Your company may own zero hardware and use cloud services for its whole IT infrastructure, yet to make the best choices, it is important that a DevOps developer can measure hardware performance and understand how to tune software to work best with the given hardware. This skill is especially important if you don’t hire a dedicated server administrator.

---

Next, we’ll look at something you’ll see in every DevOps job: Eliminating regular repetitive manual work when testing and deploying software.

Build, Test, and Deployment Automation

The process of developing software has transformed considerably over the past decade thanks to the creation of new developer tools and the introduction of automation to many processes—especially when it comes to building, testing, and deployment. This is often referred to as “Continuous Delivery” (CD), but it’s worth learning about the relationship between DevOps and CD before you start interviewing a DevOps specialist.

With CD, teams tend to ship new features more often, implement big tasks in stages by dividing them into smaller subtasks, test new ideas by rolling them out to a limited number of clients, and easily roll back when a problem gets identified in a new release. All this becomes possible with the help of a DevOps developer and allows businesses to be more flexible and adapt to always-changing markets faster and with less stress.

DevOps engineers have a key role to play in setting up, adjusting, and maintaining CD. Here’s how it works.

Step 1. The DevOps team consults other teams in order to gather information about how the development, testing, and deployment processes are set up now.

Step 2. The DevOps team learns the planned features, assesses the company’s growth speed, estimates the range of current expenses on infrastructure, finds out the most time-consuming tasks in the company—all these being factors that will affect what exactly should be automated and improved, and in which order. Even though this section is focused on automating test and deployment, the two are usually an integral part of all other processes and affect them. So the more tech stack research, the better decisions can be made about test and deployment automation.

Step 3. DevOps engineers plan and prioritize their automation implementation tasks in such a way that the most critical and/or expensive processes are tackled first. At this stage, it is usually heavily discussed with the leads of other teams in order to make most out of the test/deployment automation for everyone.

Step 4. Automation is implemented in stages.

For companies with continuous delivery already in place, there might simply be a todo list ready for a new DevOps employee to take over. Yet it is still beneficial to go through the steps above—it never hurts when the DevOps engineer learns the overall setup and can propose improvements to automation while working on the tasks planned.

Many tools and concepts are at work when it comes to automating builds, tests, and deployments. It’s worth considering which are the most relevant ones to discuss with the candidate as we explore the categories below.

Unit Tests

A unit tests is a specially developed script that tests the smallest testable parts (units) of the application for proper operation.

More often than not, unit testing is automated. The job of the DevOps team is to integrate unit test runners into the virtual machines used in development as well as CI servers so that the tests can be hooked in to run automatically for every new version before it is released, or triggered manually by a developer or tester. Reporting should also be configured so that a developer finds out instantly if their new code breaks existing functionality.

Furthermore, for large applications with lots of unit tests to be run on a regular basis, DevOps developers usually set up dedicated continuous integration servers where unit tests are run in parallel while developers continue their coding in their dev environments. Without such CI servers, when run on a single developer laptop, unit test automation can sometimes take hours, if not longer.

There are various types of tests:

• Functionality tests: Do all the key app features work?
• Compatibility tests: Does the app maintain its functionality in various environments? For instance, a web application should work in all supported browsers, an app that supports multiple databases should consistently work with all of them, a driver should function correctly with all supported hardware models, and so on.
• Performance tests: How does traffic or other types of loads affect the app’s performance?
• Security tests: Can attacks reveal flaws in the app’s security mechanisms, so that protected data is accessed without authorization?
• Usability tests: Can the app’s users intuitively understand how to use it without reading its documentation? Does it work as expected?

Some of the tests from the above groups can be joined further into a “smoke test” (also acceptance or sanity test)—a test executed before a release to reveal whether the app meets a certain standard.

DevOps Tools for Build Automation

Before a new release can be issued, the source code has to be built or at least packaged, depending on your tech stack. For statically typed programming languages like C# or Java, building implies compiling source code into an executable that can be run by a computer. For interpreted languages like PHP, Python, and JavaScript, there may not be compiling per se, but e.g. JavaScript files are joined into groups and minified, SASS files are compiled into CSS files, language files with translations are built as well, and images are minified.

A DevOps developer will typically use build automation software—and the list of such pieces of software is huge. A good question to a DevOps candidate would be to let them talk about which of these tools they’ve used to solve which tasks. The tools can be categorized in two ways:

1. Tools specific to a programming language. For example, Rake is a Ruby-based build tool while Apache Ant is popular for Java.
2. Tools specific to an OS: FinalBuilder is for Windows software developers, Make-based tools (like GNU make and mk) is for Linux/Unix and Mac OS.

It is important that the DevOps engineer who is going to work in your team is familiar with build tools specific to the OS and programming languages used in your projects.

Continuous Integration, Delivery, and Deployment

Often people conflate and confuse these three terms, so let’s go through them quickly first:

Continuous integration helps to keep the mainline (the base branch of the project’s code tracked in a version control system like Git) up to date. It’s achieved by merging all developer working copies into the mainline often, i.e. a few times per week (or more) vs once per release.

Continuous delivery makes sure a deployable working state of an app is produced often, i.e. by small increments. This way you have the confidence that an application can be deployed to production when the business is ready for it.

Continuous deployment is the process of putting the app into production. The continuity of integration and delivery is a prerequisite for the deployment stage.

All three practices are possible nowadays thanks to automation and are implemented by a DevOps engineer. All three require scripting skills and familiarity with corresponding tools. That’s why you should include this topic in your interview.

Before you talk about it with the candidate, it’s important to find out from your developers, system administrators, and other technical teams about the state of the subject in your company. If any of the integration, delivery, or deployment schemes are already in place, make notes about the tools and scripting languages at work to discuss with your candidate. If none are implemented, ask the candidate if they have implemented any for projects using the same stack of languages and frameworks as yours.

It’s not critical that a candidate has experience with every last technology on your list. That’s because there are many different tools for various cases, and the main point is the understanding of their purpose, as well as being able to learn them and start using them quickly. A consultant with enough experience is usually capable of learning about new tools in a matter of days.

Monitoring

Because so many servers, services, microservices, databases, etc. form the building blocks of an application, it would be risky not to monitor the operability and availability of them all continuously and rigorously.

A DevOps engineer must be able to set up monitoring themselves, of course. But it’s no less important that they can:

• Analyze failures and understand their causes
• Fix failures and restore functionality
• Bring in measures to prevent such failures in the future
• Outline all the details in internal reports and improve internal documentation with notes, tips, and procedures related to a given issue
• Advise the product team on improving the code that caused the issues
• Help the support team explain the issues in simple words in blog posts and in support tickets

Ask the DevOps candidate to talk about how they set up monitoring and then handled some failure. From their story, you will be able to identify whether they are familiar with the points outlined above.

You should also ask the candidate about:

• What software they used for monitoring
• Whether they used third-party services (like Pingdom) or open source solutions (like Zabbix and Nagios), and
• How they connected the services/solutions to the application (e.g. via an API, embedding JavaScript code, installing and configuring complementary server packages, etc.)

Finally, when it comes to performance and failure analysis, being able to configure logging and read log output is the key to effective handling of failures.

Here is a vocabulary of terms related to monitoring for you to get familiar with before interviewing a DevOps candidate:

• Server uptime monitoring. This means testing the availability of the company’s website, applications, servers, and services.
• Uptime test locations. It makes sense to monitor website availability from different locations, especially from those where most of the website users reside.
• Status pages. These display both current and historical data of website or service uptime. There might be public and private status pages—the latter usually face the product team, the server administrators team, and other engineers, and include comprehensive monitoring details and statistics used to manage the whole infrastructure. It’s usually the responsibility of a DevOps specialist to set up such pages.
• Page speed. For web content, slow-loading pages mean a lower Google rank and loss in conversion and sales.
• Alerting. I.e. informing responsible parties through various channels when events occur. Well-thought-out alerting decreases your time to resolution.
• Log rotation. This simply means archiving dated log files on a schedule.
• Log server. This is a dedicated server used to collect and store all logs. Having one simplifies the process of searching log data, and allows you to filter it to quickly audit the system.

One good question to ask about page speed optimization is: What measures can a DevOps specialist take to optimize page speed? It’s true that many improvements are done by developers inside the code—but there are some a DevOps engineer can do as well:

• Enabling compression at the web server level
• Improving server response time by fine-tuning web servers (like Apache and Nginx) and load balancers
• Enabling the usage of a content delivery network (CDN)

What we just discussed above is usually called performance and availability monitoring. But there is another aspect of monitoring: Security monitoring. The idea is that all operations and activities of users and programs must be tracked.

No matter how big or small your clients are, your company is responsible for the confidentiality and security of the personal data you store on your servers. The same goes for:

• Intellectual assets, like your application’s source code. You want to store it securely and not leak to your competitors.
• Corporate email and other communication channels, including user support. You don’t want everyone in your company to access those.
• Internal documents, like competitor analysis, development strategy, financial reports, etc. It’s strategically important to limit and control access to them.

Therefore, it is critical to observe and record all operations and activities at all levels of the company infrastructure, from networks to applications to employees.

(We’ll say more about security in the IT Security Best Practices section below.)

Scripting and Programming Skills

You don’t want a DevOps engineer to run an ill-designed SQL query over a database with 500 million records and make it freeze.

Solving the tasks discussed above is only possible using programming. In fact, you should expect the DevOps candidate to have coding skills in multiple programming languages.

Let’s elaborate: For one thing, scripting in bash (a Unix/Linux shell and a command language) is widely used for automating server software setup and configuration, backups, builds, and deployment; running monitoring; building reports; and consuming cloud APIs. Indeed, any kind of automation will need scripting.

For another, to smoothly integrate various system services with the application, the DevOps engineer you hire will sometimes need to know the programming language used in the application.

A good example is a multitenant PHP, Ruby, or Python web application solution running in a cloud virtual server provided by Amazon Web Services (AWS). Here are some typical tasks that will require a DevOps specialist to employ his or her coding skills:

• Create a bash script to instantiate a new VPS server at AWS. The script will need to consume the AWS API to create a new virtual server under the company’s account, deploy the application to it, and create a database schema and configure application accounts in the database. It may also be a requirement to connect to a CRM (like SalesForce) and update a user record with the server ID.
• Code an internal method in the application that reports the application state to an external monitoring script.
• Implement an internal web page that reports on the status of all servers, and shows some usage statistics.
• Make changes to an existing production Dockerfile (if Docker is used for containerization).
• Create a bash script that makes scheduled backups, as well as restores from backups by request.
• Create a bash script to roll out a new application version to some (or all) servers, as well as roll it back in case a critical bug is discovered in the new version.

Actually, there are many more such tasks in the day-to-day work found in a DevOps job. That is why there is no clear boundary between developers and DevOps—they both are able to write code. The former are just better at algorithms and programming business logic in a given domain, while the latter have wider knowledge in system administration. But many tasks can be done by both.

So, it is not a bad idea to include programming questions and problems in a DevOps interview. At a minimum, you should ask the candidate which programming/scripting languages (besides bash) they’ve used.

Another option would be to use online coding tests. There are plenty services with pre-defined tests (of varying complexity) for different programming languages and domains. Some of the services would also allow you to build your own tests, thus making it possible to select candidates with coding skills specific to your company. Whether you rely on pre-built tests or compose your own, you will need to consult your product/developer team to make the test questions relevant.

Sometimes, DevOps developers are asked to extract or examine data from a database that only they have been granted access to. For these cases, being capable of writing SQL queries will make a difference. So, if your project uses a database, you may also want to discuss the candidate’s experience working with your particular database engine. You don’t want a DevOps engineer to run an ill-designed SQL query over a database with 500 million records and make it freeze.

Configuring SQL table partitioning is a complex field—c.f. the documentation on PostgreSQL table partitioning. If your DevOps hire will need to get into this, knowing SQL syntax is not enough. They’ll need a deep understanding of indexing and factors that affect database performance, and you’ll need to hear about their prior experience in it.

Version Control and Branching Strategies

It’s difficult nowadays to find an IT project that is not managed by Git, Subversion, Mercurial, or another piece of version control software (VCS). From documentation to configuration to code to database schemas—as well as revisions made to documents of any kind—such things are usually stored in and tracked using a VCS.

Thus understanding how to use VCS systems is essential for any candidate.

As per the 2018 Stack Overflow Developer Survey, the most popular VCSes among developers are by far Git (87.2%) and Subversion (16.1%). It’s usually enough for the candidate to have experience with at least one of them. While Git has become the de-facto standard for the IT industry, SVN is still used in some recognized projects—WordPress among them, for example, as of 2018.

The idea behind all VCS systems is the same—to allow for the management of changes and branching.

The management of changes, or revision management, allows you to track every single change a given document or code file, and restore it later at will. Every change tracked in VCS makes it possible to restore the original source tracked at that time, i.e. it acts as taking a snapshot of the whole project, documentation, database schema etc. Such a snapshot is usually called a commit in VCS terminology.

Branching is a technique that allows for multiple people to simultaneously collaborate on the same source effectively—it reduces the number of conflicts between revisions and automates the merging of work made by multiple people (or by the same person who worked on multiple features simultaneously.)

Because there are different approaches to branching (also called branching strategies), like Git Flow and GitHub Flow, it won’t make much sense to require the candidate to know more than one: If they understand how one of them works, they are likely to easily learn any others.

Now, let’s look at why it is important for a DevOps engineer to be able to use VCS. In fact, there are many things done by them that require VCS skills:

• Deploying a particular version of an application, or reverting back to one of its older versions
• Deploying a particular branch containing a particular feature to a Continuous Integration server, so that tests can be run and reports generated
• Comparing two versions of the software to identify any changes that may affect compliance (when the person you want to hire for DevOps will also play the role of a security engineer)
• Tracking internal scripts and corresponding documentation in VCS

Git is a sophisticated tool that offers many possibilities. Of course, it offers basic revision tracking, but it also has advanced features that help solve complex tasks. For example, the git bisect command allows you to efficiently pinpoint which commit introduced a bug. Like with programming languages, one can improve one’s skills in Git over time by solving various tasks and reading documentation and tutorials. Therefore, you should not expect a DevOps candidate to have comprehensive expertise in VCS. It should be enough to ask the candidate about just two things:

1. Whether they have used VCSes in the past, like Git, Svn, or Mercurial
2. To find out if they are familiar with any branching strategy

Even though it is easy to come up with short tasks to let the DevOps candidate demonstrate their skills in using VCS, it’s probably not necessary if you intend to have them do a trial project—just integrate the requirement there instead of testing separately.

To help you evaluate the quality of commits and branches in the trial project’s VCS repository, you should ask those who use your company’s VCS the most—your developers (the product team). Still, here is a list of things to turn your attention to when checking the repository:

• Commit messages (i.e. the messages accompanying commits/snapshots) should be descriptive and concise and follow the same style
• Every single feature, fix, or change in documentation should be committed separately (the “commit often” principle)
• There should be no commits with half work done (or it should be identified in the commit messages)
• No “log” files or other files unrelated to the project should be tracked

Best practices for using VCS have been developed over time and described in articles like this—at least skim such a resource if you will be the one reviewing the trial project.

IT Security Best Practices for DevOps

DevOps drives so many aspects of a company’s development process. So it’s naturally the perfect place to inform the wider implementation of security practices and cultivate security awareness in all teams. With that in mind, it becomes obvious why security is another area you’ll want your DevOps candidate to excel in, especially if you are not backed by a dedicated security team.

As IT solutions are employed more and more by industries, the number of penetration possibilities is growing as well: From hardware to software to firmware to drivers to business solutions, each one is a separate item that needs protection and security monitoring. As a result, the demand for security specialists is rising like never before. That is why a DevOps developer who is also good at security is a very valuable asset.

The first recommendation would be to put the requirements related to IT security up front, right in your job advertising. If you are ready to accept a candidate who has no concrete experience in implementing security best practices when solving real-world problems, it would still make sense to mention that it is required to learn the topic, and/or to make it a requirement in the personal yearly education plan.

Next, when interviewing the candidate, closer to the end of the interview and after familiarizing them with the technology stack your company uses, ask the candidate to list a few measures they would take to improve your overall security implementation. Here are a few good examples you might expect:

• Send weekly security awareness letters—as all employees are human beings susceptible to various types of attacks, and as new vulnerabilities are discovered regularly, a weekly letter that sums up the findings and gives advice may make the company less prone to such attacks.
• Launch a dedicated security channel—setting up a dedicated flow in Slack, Flowdock, or any other corporate chat is a good way to publish timely announcements and discuss them.
• Develop a security education plan—it never hurts to study security formally; this might be just short online courses taken twice per year. Taking courses has also a good side-effect of team building.
• Build the list of tools, software, and cloud services used in the company—and watch them closely for new vulnerabilities.
• Use VPN, enforce password rotation—it is a common practice to hide internal resources, like project management software and source code hosting, behind a VPN.
• Set up security monitoring—as discussed in the Monitoring section above.

Also, for some concise pre-interview reading, DevOps Security Best Practices discusses challenges and best practices in security.

Finally, if you plan a trial project as part of the interviewing process, make sure to say in the project description that security best practices will be assessed in the deliverables, if that applies to the project. (Chances are that it does, whatever project you come up with.) Then ask your security/product team to review the project and have their say.

Communication Skills

As with any job involving collaboration, communication skills are one of the keys for the team to progress quickly as a whole. DevOps specialists are no exception to this requirement.

There are many areas where DevOps professionals will apply their communication skills in their day-to-day work:

• Collecting requirements from different teams (Product, Customer Success, Sales, etc.)
• Documenting and demonstrating the tools and solutions they create
• Brainstorming to find solutions in architecting software and hardware infrastructure
• Reporting to stakeholders

Your Human Resources department can help you filter out candidates who have obvious trouble with communication. It is not uncommon for highly qualified and experienced professionals to leave companies because of communication issues. So it’s important to identify them at an early stage before you sign a contract.

At the very minimum, both during the interview and when evaluating the trial project, make sure the DevOps candidate:

• Can express their thoughts and ideas clearly, both verbally and in writing
• Shares the value of moving fast as a team vs moving fast as an individual
• Understands the culture of sharing
• Is willing to work towards a joint target

Collaborativeness is a very important personal trait. While not being testable per se, it will affect how fast the candidate will get involved in the teamwork. Thus, if a few candidates have approximately the same level of technical expertise and skills, prefer the one with the better communication and collaboration traits.

Here are a few ideas for topics to discuss during an interview:

• Have you worked remotely (or in an open space, depending on your company’s environment) before?
• Describe how you presented a tool and/or process you developed as a DevOps engineer.
• What tools did you use for collaboration and communication with your teammates?

Screening DevOps developers is one thing. What about adapting candidate requirements to your company, its type, and its structure?

Finding a Perfect Fit for Your Company

There is a conceptual difference between hiring a DevOps manager who will lead major transformation in the company and recruiting an engineer to join an existing team in a DevOps-established company.

To establish DevOps practices in a company from scratch, you will want a candidate who:

• Previously worked as a Lead DevOps Engineer, or
• Spent a few years doing DevOps in a medium or big company as part of an existing DevOps team

The idea here is that setting up DevOps infrastructure in a company that does not have it requires, besides theory, a lot of skills and empirical knowledge. Once the infrastructure is established and relied on by the other teams in the project, it will be too costly to radically change it. This means the cost of errors in design and structure is very high in DevOps. Think of it as designing software from scratch vs hiring someone to just add more features to existing software. Same goes with the DevOps profession—you should not risk betting on a person who has limited real-world experience in the field.

If, on the other hand, the new hire is to be guided and checked by an existing DevOps team, the requirements on experience may be loosened, and more focus put on attitude and communication skills.

In any case, it is important that the candidate will be able to plug into the company’s product framework, so they must be familiar with some or all of the company’s technology stack. Those include:

• Programming languages (like PHP, Ruby, Java, or Scala)
• Application frameworks (like Laravel for PHP or Play! for Scala)
• Operating systems (OSes) used on servers and other company hardware (like Ubuntu, Red Hat Linux, or Windows Server)
• Databases (like PostgreSQL, MariaDB, Oracle, or Microsoft SQL Server)
• Tools (like backup tools and asset compilation tools)

Many times a DevOps engineer will need to develop a plugin for an existing framework using the corresponding programming language. As this will happen more often than not, their familiarity with the company’s technology stack is truly crucial.

A simple way of keeping track of it is consulting your teams and building a detailed list of the technologies used at your company, then including it in the job advertisement. If that list is considered confidential, present it to the candidate straightforwardly at the beginning of an interview instead. Such a list with items checked per candidate will come in very handy when comparing candidates—sometimes that list will make your choice obvious when comparing other traits does not help.

When it comes to deciding on interviewing strategy, the following rules of thumb apply:

• Split interviewing into a few stages, and filter out by the traits critical for your project sooner than later, as well as eliminate candidates who are troublesome in terms of communication.
• When interviewing, always involve people from various departments, not just the product team (software developers). Let them formulate and ask questions that are important to them, then collect their feedback.
• As the last stage, assign a trial project.

Trial projects are a great way for candidates to show off. For some people, especially introverts, going through interviews means a lot of stress, so you might not be able to see their full potential until they deliver something amazing through a trial project.

Finally, because DevOps and software development have many things in common, we can learn a lot from the latter.

Keep Software Development Lessons in Mind

To sum up, we would like to cite a few important conclusions from the research done by Robert L. Glass in Facts and Fallacies of Software Engineering:

• Fact 22. “Eighty percent of software work is intellectual. A fair amount of it is creative. Little of it is clerical.” The right DevOps engineer, just like a software developer, will create a lot of valuable solutions that will be part of your company’s intellectual assets.
• Fact 23. “One of the two most common causes of runaway projects is unstable requirements.” That is why your DevOps candidate’s ability to continuously and effectively communicate with the project teams is as important as having technical skills.
• Fact 27. “There is seldom one best design solution to a software problem.” Here, also being about software problems, DevOps requires some intuition and empirical knowledge. Again, it’s vital to hire an experienced DevOps engineer when the global task is to setup DevOps infrastructure in the company.

So there you have it: Build a hiring plan. Focus on your company needs. Listen to your team feedback. Make a skills and quality comparison table. Identify weaknesses and plan a continuous self-education schedule including follow-ups. Use an online service for hiring management. In the end, investing in the DevOps hiring process means benefitting afterward:

“Easy choices, hard life.
“Hard choices, easy life.”
- Jerzy Gregorek