Photon

Builds custom software and AI for big companies

Senior Data Engineer - Apache DolphinScheduler

Data EngineeringConfirmed live in the last 24 hours

Full-timeIndiaNo salary listedIn-person

About the Job

Requirements

Strong experience with Apache DolphinScheduler in a production environment.
Proficiency in Java, Python, or Scala for workflow scripting and automation.
Experience with big data technologies such as Hadoop, Spark, Hive, Flink, and Kafka.
Understanding of workflow orchestration, directed acyclic graph (DAG) execution, and scheduling strategies.
Strong problem-solving skills and ability to work in a fast-paced environment.

Responsibilities

Develop, configure, and optimize Apache DolphinScheduler workflows for data processing and automation.
Design and implement extract, transform, and load (ETL) pipelines, job scheduling, and workflow orchestration solutions.
Troubleshoot and resolve performance, scalability, and reliability issues in DolphinScheduler.
Integrate DolphinScheduler with big data tools such as Hadoop, Spark, Flink, Hive, and Kafka.
Work closely with data engineers, DevOps, and software teams to enhance workflow automation.
Develop custom plugins and extensions for DolphinScheduler as needed.
Monitor and optimize job execution, resource allocation, and workflow efficiency.
Maintain best practices for continuous integration and continuous delivery (CI/CD), version control, and infrastructure automation.

Key Responsibilities:

Develop, configure, and optimize Apache DolphinScheduler workflows for data processing and automation.
Design and implement ETL pipelines, job scheduling, and workflow orchestration solutions.
Troubleshoot and resolve performance, scalability, and reliability issues in DolphinScheduler.
Integrate DolphinScheduler with big data tools such as Hadoop, Spark, Flink, Hive, and Kafka.
Work closely with data engineers, DevOps, and software teams to enhance workflow automation.
Develop custom plugins and extensions for DolphinScheduler as needed.
Monitor and optimize job execution, resource allocation, and workflow efficiency.
Maintain best practices for CI/CD, version control, and infrastructure automation.

Required Skills & Qualifications:

Strong experience with Apache DolphinScheduler in a production environment.
Proficiency in Java, Python, or Scala for workflow scripting and automation.
Experience with big data technologies like Hadoop, Spark, Hive, Flink, and Kafka.
Understanding of workflow orchestration, DAG execution, and scheduling strategies.
Strong problem-solving skills and ability to work in a fast-paced environment.

About the Company

Photon

Builds custom software and AI for big companies

Photon designs, builds, and runs custom software for large companies, combining an in-house creative agency with its own engineering teams. Its flagship client work includes rebuilding Samsung's US online store as a set of smaller linked services and creating Olive Garden's mobile app, which lets diners join the waitlist from their phones instead of waiting at the door. Clients hire Photon's teams to plan and code their apps, websites, and AI tools and to keep them running afterward, paying for the engineers and designers assigned to each project. Photon is privately held and based in London, with its US arm in Dallas and engineering centers in Chennai and Bengaluru.

Company size: 5,001-10,000
Company stage: Not listed
Total funding: Not listed
Headquarters: London, United Kingdom
Founded: Not listed

Recently Posted Jobs

Explore more opportunities at Photon

See more jobs