It's no surprise that machine learning jobs today are among the hottest on the market. Recent studies have shown that the position of machine learning engineer commands one of the highest base salaries among surveyed jobs and has seen a 344% increase in number of postings from 2015-2018!
So these jobs are clearly in high demand. But what does it take to secure one of these highly sought-after positions? Like any job in technology, machine learning positions require candidates to go through a rigorous series of technical interviews.
What is the structure of these interviews? What topics are covered? Are these interviews similar to software engineering? (Spoiler: sorta, kinda) Who is paying for lunch?
There are a lot of questions, but not a lot of answers out there. This is largely because the field of machine learning is still young and learning to stand on its own two feet.
This article is intended to be the missing guide for what to expect in a machine learning interview. The observations in this post are born out of collective experiences interviewing for machine learning engineer and scientist positions, comprising over 90 hours of interview time across 80+ interviews at big (FAANG) companies, small (just out of Y-Combinator) companies, and everything in-between.
Let's get started.
Machine learning interviews are diverse...sometimes
The machine learning landscape is constantly evolving. If you were entering the world of machine learning ~4 years ago, the most in-demand skill would be the ability to build and debug deep learning models. Today with the rise of tools like PyTorch and Tensorflow, a bigger issue is not so much how to build machine learning prototypes but rather how to take them all the way to a deployed system in production (also called the last-mile problem).
What this means is that over the course of several years of interviewing for these positions, we've learned that the nature of these assessments can be incredibly diverse. For example, the following are examples of real interview types/questions that we've had:
1) Read a recent paper on unsupervised learning for noise reduction and whiteboard extensions to the techniques proposed in the paper
2) Explain the nodes in the Tensorflow computational graph for a feedforward layer of a neural network
3) Give an hour-long talk about some machine learning project you have done and get grilled on its details
4) Describe how you would implement Google's autocomplete
5) Explain why L1 regularization encourages sparsity in features
Sound like these questions are all over the place? Don't get overwhelmed: they are.
Does that mean you will have to know how to do all of the above for your interview? No, definitely not. This variety is only meant to illustrate that many things are fair game depending on the company you are applying to and what they expect your responsibilities will be.
For example, if you are applying to a more research-intensive machine learning position, you may be asked to explain research-caliber work (either yours or that of others). Or if you are going to be a machine learning engineer at a young company, you may have to be more familiar with the internals of a framework or technology stack that is in high-demand. It really depends.
That being said, as the field has matured, we've seen that the responsibilities of these positions are getting more well-defined leading to more standardized assessment structures.
Nowadays for most machine learning engineer positions, you can expect your interview journey to look something like the following:
First screening call with a recruiter
Technical phone interview covering some software engineering and machine learning theory
Onsite (~4-5 interviews)
- Software engineering principles (~2 interviews)
- Machine learning theory (~1 interviews)
- Machine learning system design (~1-2 interviews)
- Interview with management (cultural fit check, typical in startups)
This can be subject to change depending on the company, but we've largely seen that common format across interviews today. Now let's discuss what to expect in terms of specific skills being evaluated.
Know your machine learning theory
When applying for machine learning jobs you will always be asked about machine learning theory. In a sense, this a "well duh" moment but the machine learning theory literature is vast so knowing what to focus on rather than working through multiple 1000-page textbooks is the key question.
Conceptual questions are asked in one of two forms: 1) in the context of some larger problem (i.e. while describing a system design, discuss model tradeoffs) or 2) in isolation (i.e. spend 30+ minutes describing how your favorite supervised learning algorithm works).
In either case, the types of questions and concepts you'll be asked about stay pretty consistent. Our survey of past machine learning interviews has shown that you will tend to be tested on topics such as the following:
1) What is the bias-variance tradeoff?
2) What is overfitting/underfitting and how do you know when your model is suffering from those issues?
3) How do you evaluate model performance (which metrics are used and why)?
4) Can you explain how some classical machine learning models work (logistic regression, k-nearest neighbors, support vector machines, decision trees, etc.)?
5) What are common techniques for unsupervised learning?
6) How do you pick the best features for your model?
That being said, because the collection of potential theory topics is so large, we've distilled the key concepts seen in interviews into a primer with practice exercises.
Software engineering is still important
While the focus of machine learning jobs isn't just software engineering, you should still expect to be asked some more traditional coding and algorithms questions. These questions will be very similar, if not identical, to those you would be asked in a standard engineering interview.
Although this may seem unrelated to what you would think a machine learning practitioner does, when you are on the job knowing how to theoretically build a model is not the same as doing the actual building. This is true whether or not you are applying to be a machine learning scientist or a machine learning engineer. You need to be proficient in coding, debugging, and thinking through algorithms exercises. As an example of topics you can expect to be asked about:
1) Recursion and memoization (you'll basically never use recursion in the real world but it's often used to evaluate your ability to think through an algorithm)
2) Memory and run-time complexity analysis
3) Standard data structures like trees, linked lists, hash maps, and general graphs
That being said, the standards for coding may not be the same in your interviews as they would be of someone applying for a pure software development position. In other words your engineering abilities might not need to be as sophisticated, especially if you are just coming out of an advanced graduate degree program in machine learning where your focus was more on pushing state-of-the-art rather than building robust training pipelines.
For software engineering, there are a number of popular books and services that can help you practice these skills.
Know how to build machine learning systems
Knowing how to architect machine learning systems is one of the most important skills you can have going into the field of machine learning. One of the big aspects of machine learning's evolution in the past 1-2 years is that companies are going all-in on integrating data-driven solutions to extract business value across their teams. At scale this can lead to tremendous improvements in the quality of a service.
When you are being brought into a team as a machine learning expert, the expectation is that you will be able to apply machine learning skills to improve some existing manual process. Oftentimes you will be presented with very vague problem descriptions (e.g. "I want these recommendations to be better") and you will have to break down and frame the problem in a form where machine learning is applicable.
This is the core of designing and architecting machine learning systems. This skill is among the most crucial ones that companies look for when evaluating candidates. A system design interview will involve being presented with a case study (e.g. "we want a tool that can help us detect offensive content on our platform") and then talking through how to set up a machine learning pipeline that can address the problem. Along the way you will be assessed on your ability to:
1) Gather and validate datasets needed for your models
2) Build training infrastructure
3) Discuss the tradeoffs among potential modelling solutions
4) Know what metrics you will track for model performance
5) Interpret model predictions and do error analysis
6) Architect a deployment solution (e.g. cloud-based, on-device, etc.)
7) Iterate on user feedback to improve the solution
The important thing to recognize in these types of interviews is that there is never one "correct" answer. Your interviewers are more interested in knowing how you think through a problem. You will be asked to justify your decisions and also adapt them based on new situations (e.g. "now imagine we don't have enough money to gather 1 million annotated images...").
When preparing for these types of interviews, there's no substitute for real-world experience. Build projects so you witness firsthand what tradeoffs are necessary to consider. If that's not possible, spend a lot of time reading through company use-cases and learning how others have solved similar problems.
Technical interviews are just one signal
The last point worth mentioning is that while technical interviews are important signals in determining whether you get the job, they are still only one of many contributing signals. In particular, many startups also place great emphasis on your cultural fit for their team which is assessed explicitly through a behavioral interview or implicitly based on how you interact with other team members.
Moreover as you are participating in these interviews, it's oftentimes more important to show how you are approaching a problem. Talk through your thought process. Getting 60% of a solution while explaining your reasoning is usually better than 80% without saying a single word.
So in the event that one or two technical interviews don't go exactly how you would've liked, don't get stressed! Not all hope is lost. We've spoken to many successful machine learning practitioners who "bombed" at least one interview for the position they are currently at. Things can always still work out.
At Confetti AI, we are making it our mission to help jumpstart the careers of the next generation of data scientists and machine learning practitioners around the world. If you want more tips for acing your machine learning and data science interviews, sign up for our updates. Good luck!