Unison Group

Builds financial risk and compliance systems

Big Data Hadoop Developer

Data EngineeringConfirmed live in the last 24 hours

Full-timeKuala Lumpur, Federal Territory of Kuala Lumpur, Malaysia$5k - $12kIn-person

About the Job

Requirements

Hands-on experience with the Hadoop ecosystem and integration with relational databases such as MariaDB or Oracle Database for analytics and reporting.
Experience with Hadoop, including HDFS and YARN; Hive; Spark; Sqoop; MapReduce; Oozie; and Flume.
Experience with MariaDB and/or Oracle Database, SQL, and PL/SQL.
Programming experience in Java, Scala, or Python for Spark or MapReduce development.
Experience with data-ingestion tools including Sqoop, Kafka, and NiFi for integrating relational databases with Hadoop.
Knowledge of Hive tuning, partitioning, bucketing, and indexing.
Experience with Ambari, Cloudera Manager, Git, and Jenkins.
Linux/Unix shell scripting.
Strong analytical and problem-solving abilities.
Good communication skills for working with cross-functional teams.
Ability to manage priorities in a fast-paced environment.

Responsibilities

Design, develop, and optimize Hadoop-based big data solutions for batch and real-time data processing.
Integrate data from MariaDB or Oracle Database into Hadoop using data-ingestion frameworks such as Sqoop, Apache NiFi, and Kafka.
Implement Hive, Spark, and MapReduce jobs for data transformation and analytics.
Optimize Hive queries, Spark jobs, and HDFS usage for performance and cost efficiency.
Create and maintain ETL pipelines for structured and unstructured data.
Troubleshoot and resolve issues in Hadoop jobs and database connectivity.
Collaborate with business intelligence, analytics, and data science teams to provision data.
Ensure data security, governance, and compliance in all solutions.

Desired Qualifications

Experience with cloud-based big data platforms such as AWS EMR, Azure HDInsight, or GCP Dataproc.
Knowledge of NoSQL databases such as HBase and Cassandra.
Exposure to machine-learning integration with Hadoop or Spark.

Big Data Hadoop Developer

Unison Group · Kuala Lumpur, Malaysia · Full-time · Posted 2025-08-14

Salary: unknown currency 5,000–12,000

Workplace: on_site

Department: Noresh

Description

Job Summary:
We are looking for a Big Data Hadoop Developer to design, develop, and maintain large-scale data processing solutions. The ideal candidate should have strong hands-on experience with the Hadoop ecosystem and integration with relational databases such as MariaDB or Oracle DB for analytics and reporting.

Key Responsibilities:

Design, develop, and optimize Hadoop-based big data solutions for batch and real-time data processing.
Work with data ingestion frameworks to integrate data from MariaDB/Oracle DB into Hadoop (Sqoop, Apache Nifi, Kafka).
Implement Hive, Spark, and MapReduce jobs for data transformation and analytics.
Optimize Hive queries, Spark jobs, and HDFS usage for performance and cost efficiency.
Create and maintain ETL pipelines for structured and unstructured data.
Troubleshoot and resolve issues in Hadoop jobs and database connectivity.
Collaborate with BI, analytics, and data science teams for data provisioning.
Ensure data security, governance, and compliance in all solutions.

Technical Skills:

Big Data Ecosystem: Hadoop (HDFS, YARN), Hive, Spark, Sqoop, MapReduce, Oozie, Flume.
Databases: MariaDB and/or Oracle DB (SQL, PL/SQL).
Programming: Java, Scala, or Python for Spark/MapReduce development.
Data Ingestion: Sqoop, Kafka, Nifi (for integrating RDBMS with Hadoop).
Query Optimization: Hive tuning, partitioning, bucketing, indexing.
Tools: Ambari, Cloudera Manager, Git, Jenkins.
OS & Scripting: Linux/Unix shell scripting.

Soft Skills:

Strong analytical skills and problem-solving abilities.
Good communication skills for working with cross-functional teams.
Ability to manage priorities in a fast-paced environment.

Nice to Have:

Experience with cloud-based big data platforms (AWS EMR, Azure HDInsight, GCP Dataproc).
Knowledge of NoSQL databases (HBase, Cassandra).
Exposure to machine learning integration with Hadoop/Spark.

Apply

Apply at Unison Group

About the Company

Unison Group

Builds financial risk and compliance systems

Unison Group builds technology systems for financial institutions that need to manage risk, meet reporting rules and modernize their operations. Its consultants work across liquidity, market and credit risk, combining financial knowledge with software implementation and data analysis. The business also designs cloud infrastructure and develops custom applications, connecting these systems with the information and processes clients already use. It earns money through consulting, technology implementation and engineering services for enterprises, with work ranging from system design to deployment. Founded as Unison Consulting in 2012, the group is headquartered in Singapore.

Company size: 201-500
Company stage: Not listed
Total funding: Not listed
Headquarters: Not listed
Founded: 2012

Recently Posted Jobs

Explore more opportunities at Unison Group

See more jobs