B

Bask Health

Builds software for branded telehealth businesses

ML Platform Engineer - Evaluation Systems

Applied Machine LearningPosted on Nov 12
Full-timeRemote in USANo salary listedRemote

About the Job

Requirements

  • At least 3 years of professional experience in software engineering, machine learning engineering, platform engineering, or a closely related technical field.
  • Experience building and maintaining production backend services or infrastructure.
  • Strong programming skills in Python, TypeScript, or comparable languages.
  • Familiarity with machine learning development and evaluation workflows.
  • Experience developing APIs and backend service integrations.
  • Understanding of distributed systems and asynchronous processing.
  • Experience working with cloud-based infrastructure and production deployments.
  • Familiarity with automated testing and continuous integration practices.
  • Understanding of machine learning model versioning and experiment reproducibility.
  • Experience working with relational databases and structured datasets.
  • Familiarity with data pipelines and batch processing workflows.
  • Experience designing reliable background job systems.
  • Understanding of infrastructure scalability, performance, and fault tolerance.
  • Familiarity with logging, monitoring, and production observability.
  • Experience implementing retries, timeouts, and error recovery.
  • Understanding of authentication, authorization, and secure service communication.
  • Familiarity with configuration management and deployment automation.
  • Experience writing maintainable, well-tested production code.
  • Ability to troubleshoot complex backend or infrastructure issues.
  • Understanding of software release processes and operational reliability.
  • Experience using Git and collaborative engineering workflows.
  • Ability to translate machine learning requirements into scalable infrastructure.
  • Ability to collaborate effectively with machine learning engineers, applied scientists, and software engineers.
  • Ability to independently own engineering projects from design through deployment.

Responsibilities

  • Design, build, and maintain machine learning evaluation infrastructure across Bask Health.
  • Develop platform services that execute and manage automated model evaluation workloads.
  • Build scalable systems for running large batches of model and AI application evaluations.
  • Implement job orchestration, scheduling, and execution management for evaluation pipelines.
  • Develop infrastructure for storing and versioning evaluation datasets.
  • Build systems for tracking model configurations, prompts, parameters, and evaluation results.
  • Implement reproducible experiment workflows that support consistent model comparisons.
  • Develop automated pipelines for benchmarking machine learning models.
  • Build APIs and services that allow engineering teams to submit and monitor evaluation jobs.
  • Create infrastructure for comparing evaluation results across model versions and experiments.
  • Support evaluation workflows for large language models, predictive models, and retrieval systems.
  • Develop asynchronous processing systems for computationally intensive evaluation tasks.
  • Implement monitoring for evaluation job performance, failures, latency, and resource usage.
  • Build logging and tracing capabilities that help engineers investigate evaluation problems.
  • Develop systems for managing evaluation artifacts, reports, and historical results.
  • Integrate evaluation infrastructure with model registries and machine learning development workflows.
  • Implement automated quality checks that support controlled model releases.
  • Build reliable retry, timeout, and failure recovery mechanisms for evaluation workloads.
  • Optimize evaluation execution performance, infrastructure utilization, and operational costs.
  • Support distributed evaluation workloads across cloud-based compute resources.
  • Collaborate with machine learning engineers to define evaluation infrastructure requirements.
  • Work with applied scientists to support reproducible experiments and benchmark comparisons.
  • Develop tools that simplify evaluation configuration and execution for engineering teams.
  • Implement secure access controls for evaluation datasets and results.
  • Ensure evaluation systems respect tenant boundaries and approved data-handling practices.
  • Investigate infrastructure incidents and improve evaluation platform reliability.
  • Document evaluation infrastructure, deployment procedures, and operational standards.
  • Own assigned machine learning evaluation platform projects from technical design through deployment, monitoring, and continuous improvement.

Desired Qualifications

  • Experience building infrastructure for machine learning evaluation or experimentation.
  • Familiarity with MLflow, Weights & Biases, or comparable experiment tracking platforms.
  • Experience with model registries and artifact management systems.
  • Familiarity with large language model evaluation frameworks.
  • Experience with LangSmith, DeepEval, Ragas, or comparable AI evaluation tools.
  • Experience building distributed evaluation or benchmarking pipelines.
  • Familiarity with workflow orchestration tools such as Airflow, Dagster, or Temporal.
  • Experience with containerized workloads and distributed compute infrastructure.
  • Familiarity with Docker, Kubernetes, or comparable technologies.
  • Experience working with AWS Lambda, SQS, EventBridge, or related cloud services.
  • Familiarity with AWS SageMaker or comparable machine learning platform services.
  • Experience with serverless infrastructure and infrastructure-as-code tools.
  • Familiarity with SST or comparable application deployment frameworks.
  • Experience developing data versioning and dataset management systems.
  • Understanding of reproducible machine learning experimentation.
  • Experience implementing model performance monitoring and drift detection.
  • Familiarity with large language model APIs and model comparison workflows.
  • Experience building internal developer platforms for machine learning teams.
  • Understanding of HIPAA and healthcare data privacy considerations.
  • Experience using AI-assisted development tools such as Cursor or Claude Code.

About the Company

Bask Health

Builds software for branded telehealth businesses

Bask Health builds software for entrepreneurs, medical practices and developers launching online healthcare businesses under their own brands. Its platform brings patient intake, medical records and prescription order management into a shared system rather than requiring each clinic to assemble separate tools. Customers use a visual builder to configure questionnaires and patient portals, then coordinate consultations and medication fulfillment with outside providers. The company charges subscription fees plus costs tied to visits, pharmacy orders, shipping and transactions. Bask Health was founded in 2021 and is headquartered in New York City.

Company size
51-200
Company stage
Not listed
Total funding
Not listed
Headquarters
New York, NY
Founded
2021

Recently Posted Jobs

Explore more opportunities at Bask Health

See more jobs