I

IDT

Low-cost international calling and money transfers

Senior Data Engineer - With AI/ML experience

Data EngineeringConfirmed live in the last 24 hours
Full-timeNo salary listed

About the Job

Requirements

  • At least 6–7 years of experience building scalable data infrastructure and 1–2 years of hands-on exposure to AI/ML workflows.
  • Excellent English communication skills.
  • Hands-on experience with Apache Spark, Hadoop, and Kafka for distributed processing and real-time data ingestion.
  • Experience designing complex data pipelines that extract data from relational database management systems, JSON, application programming interfaces, and flat-file sources.
  • Skills in SQL and PL/SQL programming, advanced mastery of business intelligence and data warehouse methodologies, and hands-on experience with one or more relational database systems and cloud-based database services such as Snowflake or Redshift.
  • Experience using Python for data engineering tasks, including transformation, advanced data manipulation, and large-scale data processing.
  • Deep understanding of vector databases and retrieval-augmented generation architectures and how they support semantic retrieval workflows.
  • Skill integrating open-source large language model frameworks into data engineering workflows for end-to-end model training, customization, and scalable inference.
  • Experience with cloud platforms such as AWS or Azure Machine Learning for managed large language model deployments.
  • Understanding of software engineering principles; skills working with Unix, Linux, or Windows operating systems; and experience with Agile methodologies.
  • Proficiency in version control systems, including managing code repositories, branching, merging, and collaborating in a distributed development environment.
  • Interest in business operations and understanding how business intelligence systems support profitability, data-driven decision-making, and strategic insights.

Responsibilities

  • Design, develop, and maintain scalable data pipelines for ingestion, transformation, and delivery into centralized feature stores, model-training workflows, and real-time inference services.
  • Design, optimize, and maintain robust ETL/ELT pipelines and data structures in the cloud data warehouse (Snowflake or Redshift) to support core business intelligence.
  • Build and optimize workflows to extract, store, and retrieve semantic representations of unstructured data for advanced search and retrieval.
  • Architect and implement lightweight analytics and dashboarding solutions that provide natural-language querying and AI-backed insights.
  • Define and execute processes for prompt engineering techniques, orchestration flows, and model fine-tuning routines for conversational interfaces.
  • Oversee vector data stores and develop efficient indexing methodologies to support retrieval-augmented generation workflows.
  • Partner with data stakeholders to gather requirements for language-model initiatives and translate them into scalable solutions.
  • Create and maintain documentation for data processes, workflows, and model deployment routines.
  • Stay informed about and learn emerging methodologies in data engineering, MLOps, and large language model operations.

Desired Qualifications

  • Experience with vector databases such as DataStax AstraDB and developing large-language-model-powered applications using open-source frameworks such as LangChain and LlamaIndex, including prompt engineering, retrieval-augmented generation, and orchestration of intelligent workflows.
  • Familiarity with evaluating and integrating open-source large language model frameworks such as Hugging Face Transformers and LLaMA-4 across end-to-end workflows, including fine-tuning and inference optimization.
  • Knowledge of MLOps tooling and continuous integration/continuous delivery pipelines for model versioning and automated deployments.

About the Company

IDT

Low-cost international calling and money transfers

IDT provides low-cost international communication and payment services, including affordable international calling and cross-border money transfers. Its BOSS Revolution service enables PIN-free international calling and, since 2013, money transfers for consumers. The company earns fees from these services and also offers gift-card incentives under the Awards2Go brand. IDT focuses on reliability through capacity stress testing and aims to help individuals and businesses access inexpensive, easy-to-use global telecom and financial services.

Company size
1,001-5,000
Company stage
IPO
Headquarters
Newark, NJ
Founded
1990

Recently Posted Jobs

Explore more opportunities at IDT

See more jobs