Inetum

Global IT services for digital transformation

Data Engineer - Scala/Spark

Data EngineeringPosted on Nov 12

Full-timeMadrid, SpainNo salary listed

About the Job

Requirements

Good knowledge of Spark on Scala, CI/CD tools such as GitLab and Jenkins, HDFS, structured databases including SQL, Apache Airflow, S3 storage or COS, Parquet and ORC formats, SQL and NoSQL databases, and Hadoop.
Full understanding of Oozie, shell scripting, software development lifecycle awareness, Agile principles and ceremonies, and designing effective prompts to leverage generative AI tools across IT domains during development, testing, data generation, and documentation.
Some knowledge of Kubernetes containerization and Dremio for data virtualization.
Knowledge of the banking industry and its processes.
English proficiency at C1 level.

Responsibilities

Migrate existing Hadoop infrastructure to cloud infrastructure using Kubernetes Engine, COS, Spark as a service, and Airflow as a service.
Implement data transformation and quality measures to ensure data consistency and accuracy, using Scala, SQL, and Spark for transformation and enrichment operations.
Set up CI/CD pipelines to automate deployments, unit testing, and development management.
Write and conduct unit and validation tests to ensure the accuracy and integrity of developed code.
Automate data pipelines and streamline data ingestion through orchestrators and scheduling processes, primarily Airflow as a Service.
Write technical documentation, including specifications and operational documents, to support knowledge capitalization.
Collaborate with cross-functional teams to understand data requirements and deliver solutions.
Foster a culture of continuous learning and improvement within the team.
Occasionally travel within Italy may be required.

Desired Qualifications

Knowledge of Elasticsearch and Kibana.
Knowledge of streaming processes such as Kafka and event streams.
Knowledge of HVault.
French language knowledge is optional.
Ability to solve problems and make decisions.
Ability to support business and IT relationships, including IT operations.
Ability to understand, explain, and support change.
Ability to deliver results.
Ability to collaborate and work as a team with data squads and business teams.

Job Description

Hybrid work mode in Madrid. This service could imply a willingness to travel occasionally within Italy.

The main responsibilities are:

Migrate the existing Hadoop infrastructure to cloud infrastructure on Kubernetes Engine, COS, Spark as a service, and Airflow as a service.
Implement data transformation and quality to ensure data consistency and accuracy. Utilize programming languages such as Scala and SQL and tools like Spark for data transformation and enrichment operations.
Set up CI/CD pipelines to automate deployments, unit testing and development management.
Write and conduct unit and validation tests to ensure accuracy and integrity of code developed.
Automate data pipelines and streamline data ingestion through the implementation of different orchestrators and scheduling processes (Airflow as a Service mainly).
Writing technical documentation (specifications, operational documents) to ensure knowledge capitalization.
Collaborate with cross-functional teams to understand data requirements and deliver solutions.
Foster a culture of continuous learning and improvement within the team.

Qualifications

Good knowledge of:

Spark on Scala
CI/CD tools (Gitlab, Jenkins…)
HDFS and structured databases (SQL)
Apache Airflow
S3 storage/COS and parquet (and ORC) format
Strong knowledge of SQL and NoSQL databases
Hadoop

Full understanding of:

Oozie
Shell script
Software Development Lifecycle (SDLC) awareness
Agile principles and ceremonies
Design effective prompts to leverage Gen AI tools across IT domains (e.g., development, testing, data generation, documentation) during the development stage.

Some knowledge of:

Kubernetes containerization
Dremio as tool to virtualize data

Optionally/ as a plus:

Elasticsearch and Kibana
Streaming process (Kafka, event steam…)
HVault

Additional Information

BUSINESS AND TRANSVERSAL SKILLS

Knowledge of banking industry and processes
Problem-Solving and Decision-Making person
Business / IT relationship (including IT OPS)
Ability to understand, explain and support change
Ability to Deliver / Results driven
Ability to collaborate / Teamwork with data squads and business teams

LANGUAGES

English level C1
Optionally Fench

About the Company

Inetum

Global IT services for digital transformation

Inetum provides IT services for digital transformation in 26 countries, serving enterprises, SMEs, and public sector clients. It offers cybersecurity, cloud computing, predictive maintenance, and augmented reality, delivered through consulting, managed services, and proprietary software. It differentiates itself through a broad service portfolio, strategic partnerships with Microsoft and Sage, and a focus on tailoring solutions to fit each client’s needs. Its goal is to help clients adapt to a changing digital landscape, improving business agility and creating economic, managerial, and social opportunities.

Company size: 1,001-5,000
Company stage: IPO
Headquarters: Saint-Ouen, France
Founded: 1995

Recently Posted Jobs

Explore more opportunities at Inetum

See more jobs