Machine Learning & Large Language Model Ops Software Engineer

Role Description

We are seeking an experienced ML & LLM Ops Engineer to own the operational backbone of our ML and GenAI platform, taking models from experimentation to reliable, production-grade systems and keeping them healthy over time. This is a hands-on role for someone who thrives in a startup and can make ML and GenAI run reliably at scale, including in the on-prem and edge environments common to industrial customers.

You will:

• Own the end-to-end operational lifecycle of ML and LLM systems.

• Build and maintain automated CI/CD pipelines for model training, deployment, and serving.

• Run quality monitoring, drift detection, and observability for models in production.

• Operate LLM Ops workflows: prompt versioning, evaluation, guardrails, and inference optimization.

• Ensure reliability, security, and reproducibility using best-in-class ML Ops and LLM Ops practices.

Must Have:

• Bachelor's degree in engineering or higher, plus 3+ years in ML Ops, ML platform, or ML infrastructure engineering.

• Strong experience deploying, serving, and maintaining ML models in production.

• Hands-on experience with MLflow and CI/CD automation for ML.

• Proficiency with Docker and Kubernetes/K3s.

• Experience with major cloud infrastructure across Azure, GCP, and AWS, and with Databricks.

• Experience operating LLM applications (inference serving, evaluation, guardrails) and model quality monitoring.

• Strong software engineering fundamentals, ideally in Python.

• Ownership mindset: self-motivated, adaptable, and driven to take projects from concept to production.

Preferred to have:

• Experience deploying open source LLMs (e.g., Llama, Mistral, Qwen) to on-prem or edge machines.

• Experience with inference serving frameworks such as vLLM, TGI, Triton, or Ollama.

• Experience running models in air-gapped or resource-constrained environments.

• Experience operating ML systems for real-time time series data (sensor/IoT).

About Spector.ai

Spector.ai is a well funded early-stage startup solving the $1.5 trillion challenge of industrial asset reliability. We are building an AI-first industrial agent platform that moves plant reliability from reactive to autonomous operations, combining machine learning with domain-specific AI Agents to deliver real-time diagnostics, root cause analysis, and actionable recommendations at scale. With early pilots and major industry partnerships in motion, we are pioneering the future of AI-powered plant health.