A

Aubrant Digital

Custom software, cloud, AI solutions

Senior Databricks Engineer

Data EngineeringConfirmed live in the last 24 hours
Full-timeHyderabad, Telangana, IndiaNo salary listedIn-person

About the Job

Requirements

  • A bachelor's or master's degree in Computer Science, Information Systems, Engineering, or a related field, or equivalent experience.
  • Five to eight years of experience building data ingestion and transformation pipelines.
  • Strong proficiency in SQL and PySpark for large-scale data processing.
  • Hands-on experience with dbt for building, testing, and documenting modular data transformation pipelines.
  • Hands-on experience with Databricks, Spark jobs, and Delta Lake-based transformations.
  • Experience with ELT patterns and Medallion Architectures, including Bronze, Silver, and Gold layers.
  • Practical knowledge of Airbyte, ADF, or equivalent orchestration tools.
  • Experience with data quality frameworks such as Great Expectations or Databricks expectations.
  • Strong understanding of schema evolution, partitioning, optimization, and Delta Lake performance tuning.
  • Familiarity with Azure components including ADLS Gen2, Event Hub, and Synapse Serverless or Dedicated SQL.
  • Experience integrating pipelines with applications and APIs using read/write patterns.
  • Strong debugging skills for data integrity, ingestion failures, schema drift, and performance issues.
  • Strong communication, documentation, and cross-team collaboration skills.

Responsibilities

  • Build scalable ELT pipelines using Databricks with PySpark and SQL, dbt, and Delta Lake.
  • Develop Bronze, Silver, and Gold transformations aligned with enterprise modeling standards.
  • Implement ingestion using tools such as Airbyte, ADF, or custom connectors.
  • Collaborate with Data Modelers to operationalize business rules, mappings, and canonical definitions.
  • Integrate data pipelines with downstream applications, including mobile apps, web apps, and APIs.
  • Implement data quality and validation rules, error handling, and automated remediation steps.
  • Manage schema evolution, CDC ingestion, late-arriving data, and incremental refresh patterns.
  • Optimize Spark and Delta workloads through partitioning, indexing, caching, Z-ordering, and cluster tuning.
  • Develop reusable pipeline templates and frameworks for cross-domain scalability.
  • Collaborate with Cloud Engineers to ensure pipelines meet security, monitoring, and DevOps standards.
  • Produce technical documentation, lineage, and metadata for governance teams.
  • Support ML workloads by preparing feature datasets and inference-ready tables.
  • Participate in cross-functional design sessions, reviews, and architectural decisions.

Desired Qualifications

  • Experience with high-volume operational data domains such as logistics, telematics, IoT, ERP, or mobility platforms.

About the Company

Aubrant Digital

Custom software, cloud, AI solutions

Aubrant Digital provides tailored software development services to mid-market enterprises, focusing on enterprise software engineering, cloud infrastructure, and AI. It operates with a global delivery model that combines cost-effective engineering talent from India and nearshoring in Costa Rica to scale clients’ digital capabilities. Clients collaborate with distributed teams to build custom software, migrate and manage cloud infrastructure, and implement AI initiatives. What sets Aubrant apart is its globally distributed delivery approach paired with a focus on mid-market needs, offering scalable, tailored engineering resources rather than generic off-the-shelf solutions. The company aims to help clients succeed in the era of digital disruption by enabling rapid, cost-efficient software, cloud, and AI work at scale.

Company size
11-50
Company stage
Not listed
Total funding
Not listed
Headquarters
Jersey City, NJ
Founded
2013

Recently Posted Jobs

Explore more opportunities at Aubrant Digital

See more jobs