T

ThirdLaw

Runtime AI safety platform for enterprises

Backend / Platform Engineer - AI Analytic Engines

AI Application Engineering +1Confirmed live in the last 24 hours
Full-timeRemote in USANo salary listedRemote

About the Job

Requirements

  • At least 5 years of backend software engineering experience, including designing and shipping production software services.
  • Strong coding proficiency in Python and/or Go.
  • Deep experience with streaming data pipelines, such as Kafka, Pulsar, or Redis Streams, and batch processing systems.
  • A proven track record scaling high-QPS, low-latency services; p95/p99 ownership is a plus.
  • Familiarity with vector databases, such as FAISS, Weaviate, Qdrant, or pgvector, and embedding-based matching.
  • A strong grasp of cloud-native infrastructure, including containers, Kubernetes, serverless functions, and CI/CD pipelines.
  • Exposure to structured observability patterns, including OpenTelemetry or similar tracing standards.
  • Comfort designing and defending trade-offs around build versus buy versus open-source software.
  • Strong technical skills, a proactive mindset, clear written communication, organization, and initiative.

Responsibilities

  • Architect scalable, low-latency services for running evaluations in real time and batch, integrating with streaming data pipelines and trace-based event models.
  • Design and build the core evaluation engine, applying heuristics, semantic models, and foundation model calls to detect violations across large language model inputs and outputs.
  • Build a runtime intervention layer to determine and execute enforcement actions, such as blocking, redacting, notifying, or escalating, based on evaluation results and risk context.
  • Create reusable frameworks for pluggable evaluators and intervention policies, supporting no-code authoring and automated deployment pipelines.
  • Configure and operationalize a vector database pipeline for retrieval-augmented-generation-like use cases.
  • Scale systems by mitigating blocking gRPC threads, implementing micro-batching and streaming for large language model and embedding calls, and adding reliability controls such as queue-based back-pressure and graceful degradation paths.
  • Work closely with customers to shape the products.

Desired Qualifications

  • Experience with modern Python APIs and concurrency, such as gRPC, FastAPI, asyncio, and multithreading or multiprocessing.
  • Familiarity with ClickHouse, Apache Arrow, or fast analytical storage engines.
  • Prior work on agent frameworks, such as LangChain, CrewAI, or AutoGen, or large language model orchestration.
  • Experience in trust and safety, compliance, or AI safety domains.
  • Hands-on experience with secure enterprise integrations, including authorization and authentication, webhooks, SIEM, and IAM.
  • Experience owning p95/p99 performance metrics.

About the Company

ThirdLaw

Runtime AI safety platform for enterprises

ThirdLaw offers a runtime AI safety platform for IT and security teams to monitor and enforce enterprise policies on AI usage in real time. It tracks prompts, responses, and tool calls from AI models to detect policy violations and prevent data leakage. It emphasizes real-time enforcement and policy-driven risk management within enterprise IT, rather than static checks or post-hoc audits. Its goal is to bring AI usage under the same disciplined control as other critical IT systems.

Company size
1-10
Company stage
Not listed
Total funding
Not listed
Headquarters
Berkeley, CA
Founded
2024

Recently Posted Jobs

Explore more opportunities at ThirdLaw

See more jobs