N

N-iX

Global AI software engineering and consulting

Senior/Lead Data Engineer - Databricks

Data EngineeringUpdated on Nov 17
Full-timePolandNo salary listed

About the Job

Requirements

  • Extensive experience designing, building, and evolving scalable data platforms and production data pipelines using Databricks, Spark, Spark Lakehouse on Azure, Azure SQL Database, Azure Blob Storage, MSSQL databases, and Terraform for infrastructure provisioning.
  • Experience leading technical design, mentoring engineers, and establishing engineering standards across teams.
  • Strong software engineering skills, including developing reusable frameworks, tooling, and automation for data processing using Python and SQL, and maintaining code quality with ruff and pytest.
  • Strong understanding of data modeling, schema evolution, data contracts, and validation across multiple source systems, including MSSQL and Spark Lakehouse, and experience working within and evolving monolithic architectures.
  • Practical experience implementing automated testing with pytest, linting and code quality checks with ruff, integration testing, quality gates, infrastructure automation with Terraform, and deployment pipelines using GitLab Actions.
  • Experience improving observability and monitoring with Grafana, diagnosing operational issues, and building reliable data services on Azure.
  • Experience implementing approaches in Databricks and Spark to detect data issues early and make data flows easier to understand and troubleshoot.
  • Ability to simplify engineering workflows, including project and issue tracking in Linear, and turn repeated manual work into reusable, maintainable tools.
  • Ability to identify practical AI use cases using GitHub Copilot and ChatGPT, evaluate their effectiveness, and define safeguards for quality and reliability.
  • Ability to balance technical quality, delivery speed, operational cost, and long-term maintainability.
  • English proficiency at C1 or higher, with the ability to communicate technical concepts and architectural decisions clearly.
  • Polish proficiency at C1 or native level, with the ability to collaborate effectively within the local Polish-speaking engineering team.
  • Ability to build alignment across engineering teams and domain experts.

Responsibilities

  • Shape the technical direction of the data platform, tooling, and engineering standards.
  • Design and improve reusable frameworks, internal tools, and automation for data pipeline development.
  • Define standards for testing, CI/CD, quality gates, and platform reliability.
  • Develop reusable approaches to data modeling, schemas, data contracts, validation, and schema evolution.
  • Improve observability, data quality, lineage, and developer experience across the platform.
  • Turn recurring delivery challenges into scalable platform capabilities that reduce customer onboarding effort.
  • Apply AI to engineering workflows, establish clear guardrails, and measure its impact on productivity and quality.
  • Lead technical design discussions, mentor engineers, and guide architectural decisions and technical trade-offs.
  • Document best practices and support adoption of standard platform patterns across teams.
  • Evaluate architectural decisions against time-to-value, cost-to-serve, and delivery predictability.

About the Company

N-iX

Global AI software engineering and consulting

N-iX is a global technology partner offering Pragmatic AI Software Engineering, focusing on measuring AI tool impact on codebases with engineers before scaling. It provides software engineering, AI consulting, cloud migration, data platform engineering, application modernization, and cybersecurity. Its differentiators include a large distributed team of 2,400+ engineers across Europe, the Americas, and APAC, a track record of production-grade software, and a broad ecosystem with AWS Premier Partner (AI Competency), Microsoft, Google Cloud, Palantir, Snowflake, and SAP. The goal is to help Fortune 500 and enterprise leaders adopt AI and cloud solutions with measurable results, enabling safe scaling and continuous improvement in finance, manufacturing, supply chain, and retail.

Company size
1,001-5,000
Company stage
Not listed
Total funding
Not listed
Headquarters
Valletta, Malta
Founded
2002

Recently Posted Jobs

Explore more opportunities at N-iX

See more jobs