L

Lightning AI

Cloud-based AI development and deployment platform

Infrastructure Engineer - Storage

DevOps & InfrastructureConfirmed live in the last 24 hours
Full-time

About the Job

Requirements

  • At least 5 years of experience in infrastructure engineering, systems engineering, or related roles.
  • Hands-on experience operating distributed storage systems such as VAST, Ceph, or similar systems.
  • Strong Linux systems experience in production environments.
  • Proficiency in Python or similar scripting or programming languages for automation.
  • Experience working with bare-metal infrastructure and hardware-oriented systems.
  • Ability to debug complex issues across storage, operating system, hardware, and networking boundaries.
  • Experience with storage networking protocols such as NFS or similar protocols.
  • Experience with capacity planning, monitoring, and performance tuning.

Responsibilities

  • Operate and scale distributed storage systems, including VAST and S3-compatible object storage such as Ceph.
  • Improve performance, reliability, and efficiency of storage systems supporting large-scale AI/ML workloads.
  • Troubleshoot complex storage and data path issues across hardware and software layers.
  • Optimize storage performance for high-throughput, low-latency AI training and inference workloads.
  • Build and maintain automation for provisioning, managing, and monitoring storage infrastructure.
  • Develop Python-based tools and workflows to reduce manual operational overhead.
  • Improve lifecycle management of storage clusters, from deployment through maintenance and scaling.
  • Manage and operate Linux-based systems in production, including bare-metal environments.
  • Partner with infrastructure and data center teams on hardware bring-up, upgrades, and issue resolution.
  • Support capacity planning, utilization tracking, and forecasting for storage systems.
  • Use monitoring and telemetry to diagnose issues and improve system performance and reliability.
  • Work with Infrastructure Engineering, Network Engineering, and Platform teams to integrate storage into the broader platform.
  • Contribute to design discussions about new infrastructure deployments and scaling strategies.
  • Help define best practices for operating storage systems in high-performance computing environments.

Desired Qualifications

  • Experience with VAST storage systems in production environments.
  • Experience operating S3-compatible object storage at scale.
  • Data center operations experience, including working with physical hardware.
  • Familiarity with AI/ML or HPC workloads and their storage requirements.
  • Background in high-performance or low-latency distributed systems.
  • Familiarity with high-performance data transfer technologies such as RDMA and GPU Direct Storage.
  • Experience supporting GPU-based workloads or large-scale compute clusters.

About the Company

Lightning AI

Cloud-based AI development and deployment platform

Lightning AI builds a cloud-based platform for AI development that covers the full lifecycle from ideation to deployment. It provides a browser-based environment called AI Studio, a virtual laptop with persistent storage and configurable environments, enabling users to code on CPUs, debug on GPUs, and scale to multi-node deployments without local setup. The platform supports tools like PyTorch Lightning, Fabric, Lit-GPT, and torchmetrics to help data scientists and developers prototype, train on GPUs, and optimize models. Pricing is subscription-based, targeting enterprises and individual developers who need robust, scalable AI development capabilities. The goal is to streamline AI creation and deployment by simplifying access to compute, tooling, and scalable infrastructure in one cloud-based workspace.

Company size
201-500
Company stage
Late Stage VC
Total funding
$108.6M
Headquarters
New York City, New York
Founded
2015

Recently Posted Jobs

Explore more opportunities at Lightning AI

See more jobs