ML & LLM Ops Software Engineer

This is a remote position.

About MantriAI
MantriAI is the search partner for the company on this role. Every application comes to us first. We screen, and only shortlisted profiles go to the company.

This is for a well funded early stage company building AI for heavy industry. The product reads live sensor data from critical machines and tells plant teams what is going to fail and why.

The role
You own everything between a model that works on a laptop and a model that runs reliably inside a customer's plant. There is no platform team under you and no playbook waiting for you.

What makes this different from most ML Ops work: the models here also run on the customer's own hardware, inside the plant, often behind a firewall with no internet, sometimes on one box with a single GPU. That is a different problem from a managed cloud cluster, and very few engineers in India have done it.

You work directly with the technical founder, who came from ML engineering and quantitative trading.

What you will own
- Taking models from the data science team into production, and keeping them healthy
- CI/CD for training, packaging, deployment and serving
- Serving for both halves: numerical models on sensor time series, and the LLM layer above them
- On-prem and edge deployment at customer sites, including air-gapped
- Monitoring and drift. On sensor data that means telling a bad model from a bad sensor
- LLM ops: prompt versioning, evaluation, guardrails, inference cost and latency

What we need
- 4 to 12 years total, at least 2.5 on ML systems in production.
- You have served a model on hardware you controlled. vLLM, TGI, Triton, Ollama, TorchServe, KServe or your own GPU box.
- Docker and Kubernetes, hands on. K3s is a plus
- One cloud deep. Azure, AWS or GCP
- Strong Python
- Model registry and experiment discipline. MLflow, Weights and Biases, Kubeflow, Vertex or SageMaker

Counts strongly in your favour
Air-gapped or offline deployments. Open source models in production such as Llama, Mistral or Qwen. Edge or constrained hardware such as Jetson, industrial PCs or single GPU nodes. Streaming data, Kafka, MQTT, time series databases. Sensor or IoT data. Databricks.

Not for you if
You want a data engineering seat, a modelling seat, or a mature cloud platform with a team around it.

Details
- Compensation: ₹30-45 LPA fixed plus equity
- Location: Remote today. Open to moving to Bangalore once office opens end of 2026
- Notice. We are prioritising 30 to 45 days

_Posted by MantriAI, search partner for the company. Shortlisted applications hear from us within 48 hours.
_