F

Fella Health

AI Engineering Lead

Engineering LeadershipConfirmed live in the last 24 hours
Full-time

About the Job

Requirements

  • You have 8+ years of software engineering experience and remain an active production contributor.
  • You have at least a bachelor's degree in Computer Science, Machine Learning, or a closely related technical field.
  • You have personally built and shipped an exceptional agentic system used by real customers that reasoned across multiple steps, used tools, changed state, and operated under real production constraints.
  • You can reason deeply about harnesses, orchestration, context construction, retrieval, memory, state, tool design, structured workflows, and error recovery.
  • You have built or meaningfully owned evaluation systems for probabilistic products, including dataset construction, evaluator design, simulations, regression detection, noisy metrics, and the relationship between offline performance and production outcomes.
  • You have strong systems-engineering fundamentals and can reason about APIs, distributed systems, concurrency, queues, databases, observability, failure modes, and production reliability.
  • You know how to let an agent act safely and have sound judgment around authorization, validation, idempotency, state transitions, auditability, recovery, and escalation.
  • You understand the capabilities and limitations of current frontier and open-weight models and can distinguish model problems from problems with context, tools, data, orchestration, or evaluation.
  • You have enough technical depth to lead fine-tuning and AIOS-controlled deployment of open-weight models when evidence supports doing so; prior production deployment is not required.
  • You have successfully led and managed a small technical engineering team, set clear direction, raised quality, developed engineers, and addressed underperformance.
  • Strong engineers trust your judgment; you can make difficult decisions, explain trade-offs clearly, and challenge unsound approaches.
  • You can explain difficult technical ideas to engineers, product leaders, clinicians, and executives without losing important details.
  • You create clarity in ambiguous environments and make high-quality decisions independently.
  • You continue to write production code and lead from within the work.
  • You take responsibility for outcomes when quality drops, costs spike, tools fail, or providers degrade.
  • You have a strong nose for exceptional AI engineers and know how to create an environment in which they do their best work.

Responsibilities

  • Turn the existing customer-support agent harness into the strongly opinionated internal platform powering Jesse, Aegis, and future AIOS agents; own its architecture, reusable primitives, supported extension points, developer experience, and integration with existing infrastructure.
  • Become the senior technical owner of Jesse and work with engineers and the Clinical Product team building Aegis; improve both systems while extracting shared foundations for context, retrieval, memory, orchestration, tools, state, and escalation.
  • Build trustworthy benchmarks using deterministic checks, simulations, model-based graders, human judgment, and production outcomes; establish the path from offline evaluation to controlled production experiments so major changes ship with evidence.
  • Turn traces, poor resolutions, escalations, incidents, tool failures, and successful outcomes into improved evaluations, architecture, models, and permanent platform capabilities.
  • Make consequential agent actions safe through authorization, validation, idempotency, auditability, recovery, and human handoff; encode compliance, privacy, security, and regional requirements into the platform where possible.
  • Own model selection, routing, fallbacks, caching, and approximately $200,000 in monthly model spend; lead data preparation, fine-tuning, evaluation, and AIOS-controlled deployment of specialized open-weight models when evidence supports it.
  • Own the shared production runtime, including tracing, observability, testing, provider resilience, capacity, and incident response; act as the senior engineering decision-maker when an AI system behaves unsafely, quality regresses, or the platform fails.
  • Set AIOS’s AI architecture and strategy with the VP of Engineering, make final calls on major technical decisions, guide engineers across product pods, and remain hands-on by writing production code and building key foundations.
  • Build the Applied AI team from one engineer to approximately five people during the first year; own technical relationships with leading model providers and represent AIOS externally when it strengthens the work.
  • Make the AIOS Agent SDK operational in production and ensure it is in safe technical hands.
  • Enable product engineers to build agents without recreating context, tool, safety, evaluation, and observability infrastructure.
  • Increase agent capability without reducing predictability.
  • Support major changes with trustworthy evidence on quality, reliability, safety, latency, and cost.
  • Use production failures to continuously strengthen evaluations, architecture, and models.
  • Establish engineers' trust in your judgment and technical direction.
  • Establish AIOS as an industry leader in applied AI for production healthcare systems.

Desired Qualifications

  • You have built runtimes, SDKs, harnesses, tool layers, evaluation platforms, or shared AI infrastructure used by other engineers.
  • You have built high-volume customer-service, commerce, or transactional agents operating across complex, multi-step customer journeys.
  • You have worked on healthcare, financial, insurance, or other systems where correctness, traceability, and careful rollout matter.
  • You have fine-tuned, distilled, evaluated, or deployed an open-weight model for a specific production workflow.
  • You have built durable memory, context compression, personalization, or agents operating across sessions and extended periods.
  • You have worked on voice agents, streaming systems, or other latency-sensitive AI experiences.
  • You have worked directly with frontier model providers on evaluations, technical issues, capacity, pricing, or early access.
  • You can translate relevant research into reliable production systems without confusing novelty with progress.
  • You can move between debugging production traces, redesigning evaluations, reviewing agent abstractions, and handling provider incidents.

About the Company

Company size
Not listed
Company stage
Not listed
Total funding
Not listed
Headquarters
Not listed
Founded
Not listed

Recently Posted Jobs

Explore more opportunities at Fella Health

See more jobs