ML Engineer

Also known as: Machine Learning Engineer, Applied ML Engineer, Applied Scientist, ML Software Engineer, AI/ML Engineer.

What does an ML Engineer do?

An ML Engineer is an engineer who turns a machine learning model from an experiment into part of the product: it receives live data, responds in milliseconds and does not bankrupt the company on compute. They write production code rather than only research, and they own the model after release. Typical tasks:

• build data preparation and training pipelines that can be rerun six months later with the same result;
• choose between an in-house model, open source and an external API, and work out what each option will cost;
• deploy the model to production and integrate it into the product's services;
• set up quality and drift monitoring and retrain the model when the data changes;
• in LLM products, build retrieval, agents and evaluation of answer quality.

When does a company need an ML Engineer?

A company needs an ML Engineer when a model has proven its value as a prototype and it is time to build it into the product: recommendations, scoring, search, fraud detection, an LLM feature. Usually this is the moment when a Data Scientist has a working notebook and the team has nobody to take it to production and support it. Another common trigger is a company launching an AI line of business and assembling its first ML team. The role will not work if the company has no data yet or no clear problem for a model: then it first needs a Data Engineer or a Data Scientist to check whether there is anything to model.

ML Engineer vs Data Scientist: what's the difference?

An ML Engineer differs from a Data Scientist in being responsible for how the model runs in production, not for answering a research question. A Data Scientist finds out whether a problem can be solved with data and with which approach; an ML Engineer makes the solution work every day under load. What sets them apart from an AI / LLM Engineer is that they can train and fine-tune models themselves rather than only build products on existing ones.

ML Engineer vs related roles: comparison

Why is it hard to hire an ML Engineer?

ML Engineers are hard to hire because the role needs someone strong in both machine learning and production software, and few people are.

• One title covers very different people, from researchers to a Backend Developer who calls a model API. Searching by title brings in everyone.
• Candidates are most often strong in one half: either they know ML well and write prototype-grade code, or they write excellent code but cannot tell why a model degrades.
• Production experience is hard to see in a CV: "trained a model" does not say whether it ever reached users.
• The market has shifted quickly towards LLMs, so experience with classic ML and with LLM products now has to be checked separately.

What to decide before hiring an ML Engineer?

• Do you need research, production, or both? The answer defines the profile, and a company that has not decided screens out good candidates on the wrong criteria.
• Is the problem classic ML, LLM-based, or a mix? Recommender systems experience and RAG product experience are different profiles with different candidate pools.
• Could the role be split into two profiles? A strong Python Developer and a strong ML Engineer working as a pair are often found faster than one person who does everything.
• Who will own model infrastructure? If there is none and nobody is planned, the ML Engineer will build it, and the profile must allow for that.

Where to find an ML Engineer?

ML Engineers most often come from AI products, fintech, e-commerce, marketplaces and companies where models have long run in production: recommendations, search, scoring. A strong signal is a model used by real users and a story of what happened to it after release; some of the best candidates have no ML keywords in their profiles at all and have to be found through the products they built. NEWHR finds such candidates by the results of their work, not by applications to a job post.

How to assess an ML Engineer in an interview?

ML Engineers are assessed with a review of a model the candidate took to production and a technical interview with code. The review covers the whole chain: data, metrics, serving, incidents, cost. The technical interview is run by the company's ML Lead or a lead engineer.

Strong signals:

• describes how the model behaved after release and what had to be fixed;
• knows what inference for their model costs and how they reduced it;
• can explain why they chose a simple approach over a fashionable one.

Red flags:

• all their experience ends with a metric in a notebook;
• cannot say how they would know the model has degraded;
• discusses model architectures but not the product problem.

How long does it take to hire an ML Engineer?

We present the first candidates for the ML Engineer role within 5–7 days of the brief. A senior ML team of six took us five months.

Related roles

• Data Scientist — finds answers in data and builds prototypes, but does not own the model in production.
• MLOps / LLMOps Engineer — builds the platform all of the company's models run on.
• AI / LLM Engineer — builds products on existing LLMs, usually without training their own models.
• Evals Engineer — owns how the quality of an AI system is measured.
• AI Infrastructure / Inference Engineer — optimises inference speed and cost down to the hardware level.