D

Databricks

Builds cloud software for data analysis and AI

Staff Software Engineer - GenAI inference

Applied Machine LearningConfirmed live in the last 24 hours
Full-timeSan Francisco, CA, USAIn-person

About the Job

Requirements

  • A Bachelor’s, Master’s, or PhD in Computer Science or a related field.
  • At least 6 years or equivalent of software engineering experience in performance-critical systems.
  • A proven track record of owning complex system components and driving architectural decisions end-to-end.
  • Deep understanding of machine-learning inference internals, including attention, MLPs, recurrent modules, quantization, and sparse operations.
  • Hands-on experience with CUDA, GPU programming, and libraries such as cuBLAS, cuDNN, and NCCL.
  • A strong background in distributed systems design, including RPC frameworks, queuing, RPC batching, sharding, and memory partitioning.
  • Demonstrated ability to uncover and solve performance bottlenecks across kernel, memory, networking, and scheduler layers.
  • Experience building instrumentation, tracing, and profiling tools for machine-learning models.
  • Ability to lead through influence and work closely with ML researchers to translate novel model ideas into production systems.

Responsibilities

  • Own and drive the architecture, design, and implementation of the inference engine, and collaborate on a model-serving stack optimized for large-scale LLM inference.
  • Partner closely with researchers to bring new model architectures or features, including sparsity, activation compression, and mixture-of-experts, into the engine.
  • Lead end-to-end optimization for latency, throughput, memory efficiency, and hardware utilization across GPUs and accelerators.
  • Define and guide standards for building and maintaining instrumentation, profiling, and tracing tools to uncover bottlenecks and guide optimizations.
  • Architect scalable routing, batching, scheduling, memory-management, and dynamic-loading mechanisms for inference workloads.
  • Ensure reliability, reproducibility, and fault tolerance in inference pipelines, including A/B launches, rollback, and model versioning.
  • Integrate with federated, distributed inference infrastructure by orchestrating across nodes, balancing load, and handling communication overhead.
  • Drive cross-team collaboration with platform engineers, cloud infrastructure, and security/compliance teams.
  • Represent the team externally through benchmarks, white papers, and open-source contributions.

Desired Qualifications

  • Published research or open-source contributions in ML systems, inference optimization, or model serving.

About the Company

Databricks

Builds cloud software for data analysis and AI

Databricks builds cloud software that businesses use to organize data, analyze it, and develop AI applications. Its lakehouse architecture brings the flexible storage of a data lake together with the querying and management tools of a data warehouse. Data engineers prepare and update datasets, analysts build reports, and developers train models or create applications in a shared workspace with controls over who can use the data. The platform runs on Amazon, Microsoft, and Google clouds, and customers pay for computing usage or commit to longer contracts for discounted rates. Databricks was founded in 2013 and is headquartered in San Francisco, California.

Company size
10,001+
Company stage
Late Stage VC
Total funding
$32.1B
Headquarters
San Francisco, CA
Founded
2013

Recently Posted Jobs

Explore more opportunities at Databricks

See more jobs