Y

Yaqeen Institute

Data Engineer

Data EngineeringUpdated on Nov 11
ContractMalaysiaNo salary listedIn-person

About the Job

Requirements

  • Experience working with financial and payment data.
  • Exposure to payment APIs and automated reconciliation.
  • Advanced SQL, including performance tuning and complex analytical query design.
  • Advanced Python for pipeline development, tooling, and automation.
  • Familiarity and hands-on experience with AWS, Azure, Google Cloud Platform, AliCloud, or an equivalent cloud platform.
  • Expert knowledge of one modern data warehouse platform (Snowflake, BigQuery, Databricks, or Redshift), including architecture, storage layer design, table formats, and compute and query engine integration.
  • Experience developing and orchestrating data pipelines across ETL, ELT, change data capture, and stream processing.
  • Data modeling and tooling expertise, including dbt.
  • Experience with modern orchestration tools such as Airflow, Dagster, or Prefect.
  • Working knowledge of cloud infrastructure tooling, including Terraform, Helm, and Kubernetes or equivalent.
  • Experience defining and enforcing data contracts or schema standards in collaborative engineering environments.
  • Exposure to privacy and anonymization implementations.

Responsibilities

  • Model and structure data for analytical and operational use, owning the transformation layer and its tooling, including dbt.
  • Design data models that are clear, well documented, and easy for downstream teams to build on.
  • Define and enforce data contracts and schema standards in collaboration with upstream and downstream engineering teams.
  • Design and build scalable batch and streaming data pipelines across ETL, ELT, and change data capture patterns.
  • Orchestrate and schedule workflows using tools such as Airflow, Dagster, or Prefect.
  • Ensure pipelines are observable, recoverable, and well instrumented, with monitoring and data quality checks that catch issues before they reach consumers.
  • Architect and optimize the modern data warehouse, including storage layer design, table formats, and compute and query engine integration.
  • Provision and maintain cloud infrastructure using infrastructure as code, and manage deployment and container orchestration.
  • Implement privacy, anonymization, and governance controls to keep sensitive data protected and compliant.
  • Participate in planning and requirements discussions to raise data quality and modeling concerns before development starts.
  • Translate requirements from analysts, researchers, and product engineers into robust data models and reliable delivery.
  • Keep engineering and the wider organization connected through transparent, timely communication.

Desired Qualifications

  • Background in a SaaS, startup, or product-led environment.
  • Familiarity with data observability and lineage tooling such as Sentry or Datadog.

About the Company

Company size
Not listed
Company stage
Not listed
Total funding
Not listed
Headquarters
Not listed
Founded
Not listed

Recently Posted Jobs

Explore more opportunities at Yaqeen Institute

See more jobs