Q

Qualys

Cloud-based vulnerability management and compliance platform

Staff Big Data Engineer

Data EngineeringConfirmed live in the last 24 hours
Full-timePune, Maharashtra, IndiaNo salary listedIn-person

About the Job

Requirements

  • A minimum of 12 years of experience in software engineering, data engineering, distributed systems engineering, or big data platform development.
  • A minimum of 6 years of hands-on experience designing, developing, and optimizing Apache Spark-based data processing solutions.
  • A minimum of 6 years of experience building and operating large-scale data pipelines processing billions of events or transactions per day.
  • A minimum of 4 years of experience administering and operating Apache Kafka in production environments.
  • A minimum of 4 years of experience designing and implementing event-driven or streaming data architectures.
  • Strong expertise in distributed data processing systems, data lake architectures, and large-scale platform design.
  • Experience delivering highly available, scalable, and fault-tolerant distributed systems in production environments.
  • Demonstrated experience leading architecture initiatives and influencing technical direction across multiple engineering teams.
  • A bachelor's degree in Computer Science, Computer Engineering, Information Technology, or a related technical discipline.

Responsibilities

  • Define and drive the technical vision, architecture, and long-term strategy for the data platform and data pipeline ecosystem.
  • Design, build, and maintain highly scalable, high-availability data processing systems using Apache Spark, Apache Kafka, and distributed data technologies.
  • Lead architecture and design decisions across multiple engineering teams, ensuring solutions meet requirements for scalability, reliability, performance, security, and cost efficiency.
  • Design and implement event-driven, streaming, and large-scale data processing architectures to support real-time and batch workloads.
  • Troubleshoot, optimize, and resolve complex performance bottlenecks across data pipelines, distributed systems, and platform infrastructure.
  • Partner with Product Management, Professional Services, and Sales Engineering teams to evaluate technical solutions, architecture trade-offs, and platform capabilities.
  • Establish engineering standards, architectural guidelines, documentation practices, and operational best practices for the data platform.
  • Research, evaluate, and recommend emerging technologies that improve platform scalability, performance, and operational efficiency.
  • Mentor and guide engineers on distributed systems design, performance optimization, and big data technologies.
  • Support architecture reviews, technical governance initiatives, and engineering excellence programs.

Desired Qualifications

  • Experience working with high-volume, multi-tenant enterprise platform environments.
  • Proficiency in Scala, Java, or Python for data processing applications.
  • Proven experience mentoring junior or mid-level engineering team members.

About the Company

Qualys

Cloud-based vulnerability management and compliance platform

Qualys provides cloud-based cybersecurity and compliance solutions that help organizations secure their IT infrastructure. It offers vulnerability management, policy compliance, web application security, and IT asset management through the Qualys Cloud Platform, enabling continuous monitoring, automated checks, policy enforcement, and real-time compliance reporting. It differentiates itself with real-time analytics, automated workflows, and a scalable cloud architecture that unify multiple security and compliance functions in one ecosystem across on-premises, cloud, and hybrid environments. Its goal is to help organizations continuously assess risk, automate security processes, and demonstrate compliance across their entire IT footprint.

Company size
1,001-5,000
Company stage
IPO
Headquarters
Redwood City, CA
Founded
1999

Recently Posted Jobs

Explore more opportunities at Qualys

See more jobs