P

PhonePe

Digital payments platform enabling UPI transactions

Staff Engineer - Kafka Codebase and Internals

Systems Software EngineeringPosted on Nov 17
Full-timeBengaluru, Karnataka, IndiaNo salary listedIn-person

About the Job

Requirements

  • Deep familiarity with the Kafka codebase, including reading, modifying, and debugging broker, controller, or client internals.
  • A track record of upstream contributions, such as committed patches, KIP discussions, pull request reviews, or equivalent engagement with the Apache Kafka community.
  • Strong distributed-systems fundamentals, including consensus protocols, replication, exactly-once semantics, log-structured storage, and distributed state machines.
  • Systems-level programming comfort, including JVM internals, garbage collection tuning, memory layout, JIT behavior, operating-system-level I/O, page-cache behavior, fsync semantics, zero-copy, networking, TCP tuning, and socket buffers.
  • Experience maintaining a fork or distribution, including managing local patches while staying mergeable with upstream and minimizing divergence costs.
  • At least 8 years of experience in systems or infrastructure software engineering, with a meaningful portion spent on Kafka or comparable distributed systems such as Pulsar, BookKeeper, ZooKeeper, etcd, or CockroachDB.

Responsibilities

  • Own the internal Kafka distribution through branch management, patch curation, release cutting, and upgrade planning.
  • Contribute upstream to Apache Kafka through bug fixes, KIPs, code reviews, and design discussions.
  • Diagnose and fix deep broker and controller issues involving replication, partition reassignment, KRaft consensus, log storage, ISR management, and protocol handling.
  • Build custom patches for production needs that are unsuitable for upstream, including workload-specific performance tuning, observability hooks, and operational safety valves.
  • Establish Kafka development engineering practices for testing, benchmarking, and validating patches before rollout.
  • Mentor a supporting engineer and grow the team's capability over time.
  • Advise platform teams on Kafka configuration, capacity planning, and architectural decisions requiring knowledge of broker behavior.
  • Serve as the primary interface for audit and compliance teams, ensuring the PhonePe Kafka distribution meets compliance and regulatory standards.

Desired Qualifications

  • Apache Kafka committer or PMC member status.
  • Experience with KRaft, the ZooKeeper-free controller.
  • Authored or co-authored a KIP.
  • Experience with Kafka Streams or Kafka Connect internals, beyond usage.
  • Familiarity with Kafka's test infrastructure, including system tests, Trogdor, and ducktape.

About the Company

PhonePe

Digital payments platform enabling UPI transactions

PhonePe is a digital payments platform in India that uses the Unified Payments Interface (UPI) to enable fast, secure transfers between bank accounts for individuals and merchants, and it also offers insurance and investment products. Users link their bank accounts in the PhonePe app and make transfers with a few taps, while the company earns through transaction fees, commissions on insurance and investment sales, and merchant partnerships. It stands out by commanding a large share of the UPI market and creating an open transaction ecosystem that combines payments with insurance and investments. Its goal is to broaden digital payments across India, reaching the next billion users and strengthening the country’s financial ecosystem.

Company size
10,001+
Company stage
Growth Equity (Venture Capital)
Total funding
$2.5B
Headquarters
Bengaluru, India
Founded
2015

Recently Posted Jobs

Explore more opportunities at PhonePe

See more jobs