C

Causal Labs

Trains physics AI models to forecast weather

Member of Technical Staff - Data Infrastructure

Data EngineeringConfirmed live in the last 24 hours
Full-timeSan Francisco, CA, USANo salary listedIn-person

About the Job

Requirements

  • Demonstrated experience building large-scale data pipelines and distributed compute systems such as Spark, Ray, or Beam.
  • Knowledge of state-of-the-art methods and tools for data ingestion, storage, and loading, including file formats and storage systems such as Parquet, Zarr, or Delta Lake, and how they affect performance and scalability.
  • Deep familiarity with cloud infrastructure, data lake architectures, and batch and streaming pipelines.
  • Understanding how data-loading throughput affects large-scale training, and experience optimizing it.
  • Ability to own deliverables end-to-end, from collecting and translating requirements to autonomously driving execution.

Responsibilities

  • Design and operate petabyte-scale storage, including lakehouse architecture, file formats, and data layouts optimized for batch and real-time queries.
  • Own the shared compute and orchestration platform, including Spark, Ray, and workflow scheduling, on which ingestion and research pipelines run.
  • Optimize data strategy end-to-end from storage to loading, including high-throughput data loading into training up to the tensor boundary.
  • Build systems for cataloging, deduplication, lineage, search, and reproducibility at every stage of the data lifecycle.
  • Implement platform-level quality and monitoring tooling that data and research teams use to build their checks.
  • Scale infrastructure to improve engineering velocity and ensure reliability, with monitoring and alerting.
  • Build and operate ingestion pipelines directly for critical data sources when needed.

About the Company

Causal Labs

Trains physics AI models to forecast weather

Causal Labs is an AI research company building a foundation model of physical systems for industries whose operations turn on the weather. The company trains on atmospheric sensor and observation data rather than text or images, a domain where cause and effect are governed by physics and yesterday's prediction can be scored against what happened. It runs early pilots with commercial partners and pairs researchers with engineers who deploy the models on customer data. Unlike conventional forecasters, which report what conditions are likely, it is also trying to identify which interventions would change an outcome. The company's goal is to make AI reason about cause and effect by grounding it in the most heavily observed physical system available.

Company size
Not listed
Company stage
Not listed
Total funding
Not listed
Headquarters
Not listed
Founded
Not listed

Recently Posted Jobs

Explore more opportunities at Causal Labs

See more jobs