S

Sigma Software

Global software development and IT consulting

Senior Data Engineer - Databricks Migration

Data EngineeringConfirmed live in the last 24 hours
Full-timeKraków, PolandNo salary listedRemote

About the Job

Requirements

  • At least 5 years of professional experience as a Data Engineer.
  • Strong programming skills in Python and advanced SQL.
  • Hands-on commercial experience with Databricks.
  • Strong knowledge of Apache Spark, primarily PySpark.
  • Experience designing and building modern cloud-based data platforms.
  • Experience developing ETL/ELT pipelines and large-scale data processing solutions.
  • Hands-on experience with Delta Lake.
  • Experience with Spark Declarative Pipelines.
  • Experience with cluster monitoring, metrics analysis, and performance optimization.
  • Strong understanding of distributed data processing architectures.
  • Solid understanding of data warehousing concepts and dimensional modeling.
  • Experience with Airflow or similar orchestration tools.
  • Experience optimizing complex analytical SQL workloads.
  • Experience implementing CI/CD practices for data engineering platforms.
  • Strong troubleshooting and performance optimization skills.
  • Upper-intermediate or higher English proficiency.

Responsibilities

  • Participate in migrating a large-scale analytical platform from BigQuery to Databricks.
  • Design and implement scalable Lakehouse architectures using Databricks and Delta Lake.
  • Analyze existing ETL/ELT workloads and define migration approaches.
  • Develop and optimize data pipelines processing large volumes of retail and analytical data.
  • Implement incremental processing strategies and scalable transformation frameworks.
  • Build and maintain Spark-based data processing solutions using PySpark.
  • Design and maintain medallion architecture layers including Bronze, Silver, and Gold.
  • Implement data governance and security best practices using Unity Catalog.
  • Collaborate with Data Science, Analytics, Product, and Customer Engineering teams.
  • Participate in architecture discussions and technical solution design.
  • Develop reusable data platform components and engineering standards.
  • Conduct code reviews and contribute to platform reliability and maintainability.
  • Troubleshoot and optimize complex SQL and Spark workloads.
  • Support production deployments and platform modernization activities.

Desired Qualifications

  • Experience working with GCP cloud services.
  • Experience with AWS or Azure cloud platforms.
  • Experience in retail analytics or pricing optimization domains.
  • Experience supporting machine learning or AI-related data workloads.
  • Experience with platform modernization and cloud migration initiatives.

About the Company

Sigma Software

Global software development and IT consulting

Sigma Software provides software development services and IT consulting to more than 170 clients worldwide. It builds large-scale software solutions and guides startups from concept to a ready product, across industries such as banking, automotive, advertising, telecommunications, cybersecurity, gaming, aviation, real estate, and healthcare. The company works by deploying teams of skilled engineers and consultants, delivering projects through global delivery centers and emphasizing clear communication, flexibility, and alignment with clients’ business goals. Sigma Software differentiates itself with recognition as a top outsourcing firm (IAOP World’s Top 100 Outsourcing), a wide geographic footprint, and the ability to take projects from idea through production for both established enterprises and startups. Its goal is to help clients achieve their business objectives by delivering reliable software products and expert IT advice on time and within scope.

Company size
1,001-5,000
Company stage
Not listed
Total funding
Not listed
Headquarters
Stockholms kommun, Sweden
Founded
2002

Recently Posted Jobs

Explore more opportunities at Sigma Software

See more jobs