I

Institute of Foundation Models

Researches and develops foundation models

Data Engineer

Data EngineeringConfirmed live in the last 24 hours
Full-timeSunnyvale, CA, USAIn-person

About the Job

Requirements

  • A bachelor's degree in Computer Science, Data Science, Engineering, or a related technical field is required.
  • Extensive experience in data engineering, data processing, and automation using Python is required.
  • Demonstrated proficiency in designing and deploying web crawling solutions, automated data extraction, and processing pipelines is required.
  • A strong understanding of data structures, algorithms, databases, SQL, and performance optimization is required.
  • Experience with cloud infrastructure and distributed data processing frameworks such as AWS, Spark, Kafka, or Kubernetes is required.
  • The ability to rapidly address technical challenges is required.
  • Strong communication and collaboration skills with cross-functional teams are required.

Responsibilities

  • Rapidly collect, curate, and preprocess datasets based on detailed specifications provided by NLP researchers, delivering data within tight timelines.
  • Develop and maintain efficient web crawling solutions, APIs, and automated workflows to continuously improve data collection processes.
  • Refine and evaluate outputs from large language models to generate structured datasets suitable for model training and benchmarking.
  • Implement scalable data pipelines for efficient data processing, storage, retrieval, and distribution to research teams.
  • Collaborate closely with researchers and engineers to ensure collected data meets specified quality and relevance criteria.
  • Document data collection methodologies, dataset characteristics, and pipeline architecture.
  • Engage with peer teams and participate in technical reviews to uphold best practices and data quality standards.
  • Represent MBZUAI at industry and research forums, showcasing technical capabilities in large-scale data processing and AI data infrastructure.

Desired Qualifications

  • A master's degree or PhD degree, or equivalent experience, in Computer Science, Data Engineering, or related technical fields is preferred.
  • A proven track record of supporting NLP or AI research teams with rapid and reliable data delivery is preferred.
  • Experience working with large language models, including evaluation, efficient inference, and prompt engineering, is preferred.
  • Experience refining outputs from large-scale AI models, such as LLM-generated data, is preferred.
  • Contributions to open-source projects, coding competitions, or high visibility in coding communities such as GitHub or Stack Overflow are preferred.
  • Familiarity with recent advancements in NLP data processing and large language model technologies is preferred.

About the Company

Institute of Foundation Models

Researches and develops foundation models

IFM builds and studies foundation models—large AI models designed to learn broadly from diverse data and be used across many tasks. Its work centers on academic research to create open, fast, and practical models that address real-world societal needs, rather than narrow applications. The models are developed by a global team across Abu Dhabi, Paris, and Silicon Valley, with a strong emphasis on openness and collaboration to advance AI science and accessibility. Unlike typical private labs that lock models behind paywalls, IFM aims to provide publicly accessible, efficient models that can be used by researchers and developers to solve real problems. The overarching goal is to push the science of foundation models forward while ensuring their benefits reach society at large.

Company size
Not listed
Company stage
Not listed
Total funding
Not listed
Headquarters
United Arab Emirates
Founded
Not listed

Recently Posted Jobs

Explore more opportunities at Institute of Foundation Models

See more jobs