MLOps / LLMOps Engineer

Also known as: MLOps Engineer, LLMOps Engineer, ML Platform Engineer, ML Infrastructure Engineer, AI Platform Engineer.

What does an MLOps / LLMOps Engineer do?

An MLOps / LLMOps Engineer is an engineer who builds and runs the platform that models go through to reach production and keep working there: from training and deployment to monitoring and the GPU bill. They make it possible for the ML team to ship a new model version in hours rather than weeks, and to roll it back. LLMOps adds routing between models, caching and control of token spend. Typical tasks:

• build pipelines for training, deploying and rolling back models;
• version data, models and experiments so any result can be reproduced;
• set up monitoring for quality, drift and latency;
• manage GPU resources and inference cost;
• for LLMs, set up request logging, answer evaluation and choosing the right model for each task.

When does a company need an MLOps / LLMOps Engineer?

A company needs an MLOps / LLMOps Engineer when it has several models in production and ML Engineers spend more time on deployment and incidents than on the models themselves. Common triggers: the GPU or LLM API bill grows faster than revenue, nobody can say which model version is running right now, quality quietly drops after a model update. Another trigger is a company building an ML function from scratch that wants the platform in place from the start. If there is one model that changes once a quarter, a separate role will not pay off: an ML Engineer together with a DevOps Engineer can take on this work.

MLOps / LLMOps Engineer vs DevOps Engineer: what's the difference?

An MLOps / LLMOps Engineer differs from a DevOps Engineer in working with models, and unlike an ordinary service a model has data, weight versions and a quality that can drop without a single error in the logs. A DevOps Engineer takes care of code delivery and infrastructure stability; an MLOps Engineer also has to understand how a model is trained and why it degrades. What separates them from an ML Engineer is focus: an ML Engineer builds a specific model, an MLOps Engineer builds the system all the models live in.

MLOps / LLMOps Engineer vs related roles: comparison

Why is it hard to hire an MLOps / LLMOps Engineer?

MLOps / LLMOps Engineers are hard to hire because the role needs experience in both infrastructure and machine learning, and most candidates come from only one of those worlds.

• DevOps Engineers who call themselves MLOps have often never dealt with model drift or training reproducibility.
• ML Engineers interested in infrastructure have rarely worked with Kubernetes, GPU clusters and on-call at serious scale.
• LLMOps is a young field: few people have experience with model routing and answer evaluation, and in a CV it is hard to tell from a weekend project.
• Strong candidates are usually already building the ML platform at their company and are not looking.

What to decide before hiring an MLOps / LLMOps Engineer?

• How many models are in production now, and how many will there be in a year? Two models and twenty need different people at different levels.
• In-house models, open source or external APIs? A platform for training your own models and the tooling around an LLM API are different jobs.
• What infrastructure: cloud, your own GPUs, Kubernetes? Experience with self-hosted GPU clusters is rarer, and you need to know this before the search.
• Who deploys models today? If ML Engineers are used to shipping on their own, the new person will need a mandate to change the process.

Where to find an MLOps / LLMOps Engineer?

MLOps / LLMOps Engineers most often come from ML platform teams in fintech, e-commerce, adtech and AI startups, and from Platform Engineers and ML Engineers who grew into infrastructure. A strong profile signal is an internal ML platform they built that other teams use, or contributions to open-source tools for model training and serving. NEWHR finds such candidates by the results of their work, not by applications to a job post.

How to assess an MLOps / LLMOps Engineer in an interview?

MLOps / LLMOps Engineers are assessed with a review of an ML platform or pipeline the candidate built and a practical task: deploy a model with monitoring and the ability to roll back. The ML Lead and someone from the infrastructure team take part on the company side.

Strong signals:

• describes an incident where a model degraded silently and how it was caught;
• knows what GPUs or tokens cost and what they did to bring that down;
• thinks about how ML Engineers will use the platform, not only about the technology.

Red flags:

• describes only CI/CD for services, with no data or model versioning;
• cannot explain how to tell a drop in model quality from an infrastructure failure;
• assembles a platform from fashionable tools without asking what the ML team needs.

How long does it take to hire an MLOps / LLMOps Engineer?

We present the first candidates for the MLOps / LLMOps Engineer role within 5–7 days of the brief. A senior ML team of six took us five months.

Related roles

• ML Engineer — builds specific models, not the platform for them.
• DevOps Engineer — builds infrastructure and code delivery for ordinary services.
• AI Infrastructure / Inference Engineer — optimises inference speed and cost more deeply, at the hardware and runtime level.
• Platform Engineer — builds the internal platform for the whole company, not only for ML.
• Evals Engineer — builds the AI quality evaluation that monitoring relies on.