Senior Staff Engineer - Compute

About the role and Team 

Batch Data Platform team is responsible for building Uber's Data Lake for the rest of the company. Engineering at Uber means building for real-world impact under real-world constraints. As a Senior Staff Backend Engineer, you will lead the architecture and implementation of next-generation distributed systems that power the bedrock of Uber. This is a high-stakes, high-impact role where you’ll tackle some of the most complex infrastructure challenges in the industry: building sub-50ms latency systems that handle 1M+ requests per second, and designing resilient platforms that power tens of billions of dollars in global marketplace transactions across 70+ countries.

The team is proud to be a part of the open source community in innovating and shaping such exciting technologies as we move forward. Uber, as a business, is also growing rapidly, and Data at Uber is at the heart of almost all products e.g. Pricing predictions, Uber Pool route optimizations, Uber Eats restaurant recommendations, fraud detection, storage and processing of data collected from Autonomous vehicles etc.

By solving these business problems you will not only be helping Uber but also have a front row seat to build and innovate the future Big Data resource manager systems such as YARN and Kubernetes and contribute them back to open source. This is an exciting time to be part of the Data team at Uber. Sr Staff lead in this team will be a global lead for batch compute team including YARN and batch Kubernetes. Be sure to check out our engineering blog to learn more about the team

What you’ll do 

• Design systems architecture and deliver software and tools to advance the reliability of infrastructure including availability, performance, efficiency and scaling.
• Provide technical leadership, influence and partner with fellow engineers to architect, design and build infrastructure that can stand the test of scale and availability, while reducing operational overhead.
• Drive efficiencies in systems and processes through automation: capacity planning, configuration management, performance tuning, monitoring and root cause analysis.
• Collaborate with platform, product and security engineering teams, and enable successful use of infrastructure and foundational services, and manage upstream and downstream dependencies

Basic Qualifications 

• 12+ years of professional experience in backend development and building world-class distributed systems in large-scale production environments.
• Proven ability to solve strategically important problems by leading organization-wide development and adoption of key frameworks and coding best practices.
• Experience foreseeing architectural problems or opportunities a year in advance and working with leadership to address them before they impact the business.
• Demonstrated track record of leading large-scale engineering projects through all lifecycle stages, from inception and requirement gathering to global productionization.
• Expertise in defining standards for testing, monitoring, and alerting systems, while overseeing the execution of complex incident management at an organisational level.
• Proven experience in identifying architectural gaps and creating org-wide frameworks that improve developer velocity and reduce systemic technical debt.
• Demonstrated leadership in building strategic relationships with internal stakeholders and external partners to rally an organization around a technical vision.
• Education: Bachelor’s degree in Computer Science, Engineering, or equivalent practical experience.

Preferred Qualifications 

• Prior experience with big data resource manager systems  - YARN, Kubernetes, Containers, etc preferred.
• Experience building hardened developer tools, SDKs, and abstractions (e.g., streaming data with Flink/Kafka) used by hundreds of engineers.
• Proficiency in cloud-native infrastructure (GCP) and containerization (Kubernetes) at a massive global scale.