Senior Site Reliability Engineer II

WHAT YOU'LL DO

Site Reliability Engineers (SREs) are responsible for keeping all internal-facing services and platforms running smoothly. In a nutshell, SREs ensure site uptime. SREs blend sensible system administrators and software engineers who apply sound engineering principles, operational discipline, and mature automation to the environments and infrastructure services we provide. We specialize in systems–whether it be networking, the Linux kernel, or some more specific interest in scaling–algorithms or distributed systems.

Our team helps to improve automation, infrastructure reliability, and empowers Braze’s other engineering teams to leverage the infrastructure products and platforms we create easily. Braze operates at a massive scale with over 3.3 billion monthly active users across our customers, collecting hundreds of billions of data points each month, and sending billions of messages to end-users daily. We use a diverse technology stack rooted in Ruby on Rails, MongoDB, Redis, Kafka, Kubernetes, and more.  As a Senior Site Reliability Engineer at Braze, you will collaborate with your team and consumer engineering teams to continuously improve the infrastructure, automation, and tooling that build internal products from these technologies.

Responsibilities:

• Partner with Braze’s engineering teams on:
• Architecting products to effectively utilize infrastructure platforms in a scalable, reliable manner
• Debugging reliability and scalability issues across all stack layers, including the products built using our infrastructure platforms
• Make monitoring and alerting alerts on symptoms and not on outages
• Ensure that Braze meets our strict enterprise-grade SLAs with customers
• Develop Braze’s internal platform infrastructure:
• Create Infrastructure as code using  Chef, Terraform, and Kubernetes
• Develop deployment pipelines for applications in multiple languages using Docker, Kubernetes, etc.
• Provide centralized/common tooling, services, and automation frameworks that are critical for scaling operations, capacity management, reducing operational pain, and improving the day-to-day workflow of Braze’s engineering teams
• Manage incidents:
• Be on a PagerDuty rotation to respond to availability incidents and provide support for other engineers
• Use your on-call shift to prevent incidents from ever happening
• Retrospect everything that happens to turn lessons into system improvements/changes, automation, etc.

WHO YOU ARE

• 5+ years of experience as a Software, DevOps, or Site Reliability Engineer
• 3+ years of Data Streaming Reliability Engineering
  • Experience in monitoring, troubleshooting, and optimizing Kafka streaming applications, including diagnosing lag, partition imbalances, consumer group issues, and broker failures
  • Expertise in setting up alerting, dashboards, and runbooks for high-availability and fault-tolerant streaming pipelines
• 3+ years of Kafka performance tuning & automation
  • Strong background in scaling Kafka clusters, tuning producer/consumer configurations, and managing schema evolution.
  • Proficiency in infrastructure automation (Terraform, Ansible, Kubernetes) and CI/CD practices to streamline deployments and ensure resilient data streaming workflows.
• You think about systems - interfaces, boundaries, edge cases, failure modes, behaviors, specific implementations
• Have an urge to collaborate, document, and deliver quickly
  • Collaborating across the global remote teams, often working asynchronously
  • Document everything so you don't need to learn the same thing (or plan the same work) twice
  • Delivering fast to delight our customers–even internal ones
• Have an enthusiastic, go-for-it attitude. When you see something broken, you can't help but fix it
• Have a desire to solve everyday challenges facing software engineers and automate their toil away
• Have an excellent ability to manage multiple tasks and expectations at once
• Know your way around Linux and Unix Shell.
• Have strong programming skills - Ruby and/or Go preferred
• Have experience with Docker, Kubernetes, Terraform, or similar IaC technologies
• Have experience with MongoDB, Redis, Kafka, Postgres, or similar data technologies