E

Elastic

Searches enterprise data and monitors systems

Lead Data Engineer

Data EngineeringConfirmed live in the last 24 hours
Full-timeUnited States$113.1k - $215.1k

About the Job

Requirements

  • At least 8 years of experience in data engineering or related roles.
  • Experience with Apache Airflow for orchestrating data workflows.
  • Experience with Spark, Flink, Kafka, and dbt Labs for data processing and transformation.
  • Understanding of Apache Iceberg for data table formats and data warehouse and lakehouse strategies.
  • Skill using Terraform for infrastructure-as-code deployments.
  • Proficiency in Bash, Python, and SQL.
  • Knowledge of OLAP, OLTP, document, and vector databases.
  • Experience with Google Cloud Platform for cloud resource management.

Responsibilities

  • Design, maintain, and improve the pipelines and infrastructure that power the data platform.
  • Work across the data lifecycle, from ingestion and streaming through transformation and delivery.
  • Work with stakeholders to understand their needs and define how data should flow across systems.
  • Co-create data contracts, write documentation, and guide teammates in using the platform.
  • Monitor and refine pipeline performance and processes.
  • Use Terraform to provision and manage resources on Google Cloud Platform.
  • Review code and ensure good practices and standard processes are followed.
  • Process access requests and support compliance with platform policies.

Desired Qualifications

  • Proven experience with data security and access controls for SOX-compliant data.
  • Experience and knowledge of the Elastic stack.

About the Company

Elastic

Searches enterprise data and monitors systems

Elastic builds search, observability and security software for developers, IT operations teams and security analysts. Elasticsearch indexes enterprise data so applications can retrieve records, while Kibana gives teams dashboards for exploring results and investigating system activity. Customers use the tools to search business information, monitor applications and infrastructure, and investigate threats across their data. Elastic sells managed cloud services and self-managed subscriptions, with resource-based hosted pricing, usage-based serverless pricing and licenses tied to nodes and memory. Founded in 2012, Elastic lists San Francisco, California, as its headquarters.

Company size
1,001-5,000
Company stage
IPO
Headquarters
San Francisco, CA
Founded
2012

Recently Posted Jobs

Explore more opportunities at Elastic

See more jobs