C

Crusoe

Builds AI data centers near cheap energy

Senior Software Engineer - Crusoe Managed Data Transfer

Backend EngineeringPosted on Nov 9
Full-timeSan Francisco, CA, USA$170k - $205kIn-person

About the Job

Requirements

  • Experience designing and shipping production backend services and APIs, ideally using Go, gRPC or REST, relational databases, and Kubernetes.
  • At least 5 years of experience building cloud infrastructure or platform services, including taking a service from zero to general availability.
  • Comfort with long-running, failure-prone work involving idempotency, retries with backoff, leases, checkpointing, and partial failure.
  • Ability to design public APIs that customers can automate against and that can grow without breaking changes.
  • Working knowledge of authentication, authorization, and least privilege, and care in handling customer credentials.
  • Experience running services in production, participating in on-call rotations, and building dashboards and alerts that keep incidents short.
  • A bachelor's degree in Computer Science, Engineering, or a related field, or equivalent professional experience.

Responsibilities

  • Design and build the data-transfer API, including create, list, get, and cancel operations, with a provider-neutral model for sources and destinations that supports adding clouds and storage types without breaking customers.
  • Build the scheduler and state machine to split bucket listings into units of work, place them on regional worker pools, and track progress, retries, and resumption across failures.
  • Make petabyte-scale jobs safe to retry and cancel, implement end-to-end checksum verification, and provide clear errors when customer permissions or destinations are incorrect.
  • Implement secure access to customer buckets using short-lived, least-privilege credentials, beginning with AWS role assumption and integrating with Crusoe IAM and audit logging.
  • Ship the command-line interface and, with partner teams, the Terraform and Console experiences for creating and monitoring transfers.
  • Instrument the service with metrics, logs, and alerts; write runbooks; and participate in the team's on-call rotation.
  • Agree on interfaces required by Managed Data Transfer with the Storage, Network, IAM, and Console teams, and work with Product and Solutions Architects on early customer onboarding.

Desired Qualifications

  • Experience with large-scale data movement, such as S3 multipart uploads, rclone, s5cmd, AWS DataSync, or Google Storage Transfer Service.
  • Hands-on experience with cross-cloud identity, such as AWS IAM and STS, GCP service-account impersonation, or Azure workload identity.
  • Experience with storage systems such as object storage, NFS, or parallel file systems, or with high-throughput networking.
  • Experience with job orchestration systems such as Temporal, Argo, or Kubernetes operators.
  • Experience building on GPU or AI infrastructure.

About the Company

Crusoe

Builds AI data centers near cheap energy

Crusoe builds and runs data centers for artificial intelligence, placing them near cheap or otherwise wasted energy and handling everything from the power supply to the cloud software. It developed the flagship campus in Abilene, Texas, for Stargate, the AI infrastructure project backed by OpenAI and Oracle. AI companies rent its GPU servers on demand or under longer contracts through Crusoe Cloud to train and run their models, while larger customers lease entire campuses that Crusoe develops and operates for them. The company was founded in 2018, when it began running computers at oil fields on natural gas that would otherwise have been burned off, and it is based in Denver.

Company size
1,001-5,000
Company stage
Debt Financing
Total funding
$14.5B
Headquarters
Denver, CO
Founded
2018

Recently Posted Jobs

Explore more opportunities at Crusoe

See more jobs