Full-Time

Senior Data Engineer

Databricks

Posted on 10/14/2025

Appen

Appen

10,001+ employees

Provides human-annotated data for AI training

No salary listed

Hyderabad, Telangana, India

In Person

Category
Data & Analytics (1)
Required Skills
Redshift
Python
Airflow
NoSQL
Data Structures & Algorithms
Apache Spark
SQL
Kinesis
ETL
AWS
Scala
Data Governance
Databricks
Requirements
  • 5-7 years of hands-on experience with AWS data engineering technologies, such as Amazon Redshift, AWS Glue, AWS Data Pipeline, Amazon Kinesis, Amazon RDS, and Apache Airflow
  • Hands-on experience working with Databricks, including Delta Lake, Apache Spark (Python or Scala), and Unity Catalog
  • Demonstrated proficiency in SQL and NoSQL databases, ETL tools, and data pipeline workflows
  • Experience with Python, and/or Java
  • Deep understanding of data structures, data modeling, and software architecture
  • Bachelor's Degree in Computer Science, Information Systems, or a related field. A Master's Degree is preferred
  • Strong problem-solving skills and attention to detail
  • Self-motivated and able to work independently, with excellent organizational and multitasking skills
  • Exceptional communication skills, with the ability to explain complex data concepts to non-technical stakeholders
Responsibilities
  • Design, build, and manage large-scale data infrastructures using a variety of AWS technologies such as Amazon Redshift, AWS Glue, Amazon Athena, AWS Data Pipeline, Amazon Kinesis, Amazon EMR, and Amazon RDS
  • Design, develop, and maintain scalable data pipelines and architectures on Databricks using tools such as Delta Lake, Unity Catalog, and Apache Spark (Python or Scala), or similar technologies
  • Integrate Databricks with cloud platforms like AWS to ensure smooth and secure data flow across systems
  • Build and automate CI/CD pipelines for deploying, testing, and monitoring Databricks workflows and data jobs
  • Continuously optimize data workflows for performance, reliability, and security, applying Databricks best practices around data governance and quality
  • Ensure the performance, availability, and security of datasets across the organization, utilizing AWS’s robust suite of tools for data management
  • Collaborate with data scientists, software engineers, product managers, and other key stakeholders to develop data-driven solutions and models
  • Translate complex functional and technical requirements into detailed design proposals and implement them
  • Mentor junior and mid-level data engineers, fostering a culture of continuous learning and improvement within the team
  • Identify, troubleshoot, and resolve complex data-related issues
  • Champion best practices in data management, ensuring the cleanliness, integrity, and accessibility of our data
  • Optimize and fine-tune data queries and processes for performance. Evaluate and advise on technological components, such as software, hardware, and networking capabilities, for database management systems and infrastructure
  • Stay informed on the latest industry trends and technologies to ensure our data infrastructure is modern and robust
Desired Qualifications
  • Experience with AI and machine learning technologies is highly desirable
  • A Master’s Degree is preferred

Appen creates and curates large datasets used to train and improve artificial intelligence. It relies on a global, crowdsourced workforce to produce human-annotated data across text, images, audio, and video, which powers AI systems such as search engines and social feeds. Its products include data annotation services and platforms (like Figure Eight) and specialized data like mobile location data (Quadrant) to support ML pipelines. The company differentiates itself through scale, a broad range of data types, and a history of strategic acquisitions that expand its capabilities (e.g., Leapforce for search relevance, Figure Eight for annotation tooling, Quadrant for location data), enabling end-to-end data supply for enterprise AI. Its goal is to help organizations build reliable AI by providing high-quality labeled data and to capitalize on opportunities in generative and enterprise AI while returning to profitability under new leadership.

Company Size

10,001+

Company Stage

IPO

Headquarters

Sydney, Australia

Founded

1996

Simplify Jobs

Simplify's Take

What believers are saying

  • China revenue surged 70.7% in 2024, establishing market leadership.
  • Global Product revenue grew 221.9% to $31.3 million in 2024.
  • $50 million raised in 2025 funds generative AI expansion opportunities.

What critics are saying

  • Google contract loss in 2024 cut 37% from Global segment revenue.
  • OpenAI and Meta build in-house tools, eroding Appen's outsourcing demand by 2027.
  • CrowdGen payment issues since September 2025 trigger contributor exodus.

What makes Appen unique

  • Appen leverages 30-year proprietary metadata repository from billions of data judgments.
  • Appen excels in RLHF for frontier LLMs, supporting 80% of top builders.
  • Appen's GDPR and SOC2 compliance secures government and healthcare contracts.

Help us improve and share your feedback! Did you find this helpful?

Benefits

Remote Work Options

Professional Development Budget

Flexible Work Hours

Company News

Yahoo Finance
Mar 26th, 2026
Appen forecasts up to $300M revenue as earnings expected to surge 73% annually

Appen, an AI lifecycle company specialising in data sourcing and annotation, is navigating growth prospects despite current challenges. The company expects revenue growth of 14% annually, outpacing the Australian market's 6% average, with earnings forecast to surge approximately 73% annually over the next three years. Despite reporting a net loss increase to $21.82 million in FY2025 from $20.01 million previously, Appen projects revenues between $270 million and $300 million for FY2026. The company maintains focus on research and development investments to remain competitive in the AI and data annotation sector. With a market capitalisation of A$416.77 million, Appen generates revenue through its China and Global segments, contributing $104.11 million and $127.87 million respectively.

International Finance Publications Limited
Mar 13th, 2026
Business Leader of the Week: CEO Andrew Ettinger to reinvent Hume AI

Business leader of the week: CEO Andrew Ettinger to reinvent Hume AI. March 13, 2026 Hume AI CEO Andrew Ettinger will be responsible for accelerating the tech company's momentum in research services International Finance Business Desk Hume AI, the leading voice AI research company dedicated to aligning artificial intelligence with human well-being, recently announced a new CEO. The new boss, Andrew Ettinger, who has 15 years of leadership in data and AI infrastructure, building and scaling teams responsible for over USD 2 billion in ARR (Annual Recurring Revenue) at companies like Pivotal, Astronomer, and Appen, will now accelerate the tech company's momentum in research services. Andrew Ettinger recently served as Chief Revenue Officer at Appen, where he led commercial operations supplying hyperscalers and AI labs with proprietary datasets and LLM evaluation software. Appen is a leading AI data collection company that delivers high-quality, custom data across all languages and modalities (text, image, audio, and video) to create tailored datasets for training diverse AI models. New York-based data company Astronomer specialises in DataOps and AI orchestration. The company's flagship platform, Astro, allows businesses to build, manage, and scale complex data pipelines and AI workflows. Reacting to the news of his hiring, Ettinger said, "Voice in AI is evolving from a feature to the primary interface for the next generation of applications and devices. Understanding emotion will be essential to unlocking AI's full potential, and that will require ongoing systems that incorporate human-in-the-loop feedback. That's where Hume AI's data, annotation, and reinforcement-learning infrastructure is setting the pace for the industry." Hume AI recently agreed to license certain technologies non-exclusively to Google. Additionally, co-founder Alan Cowen has joined the company led by Sundar Pichai. Tough Test Awaits Andrew Ettinger Andrew Ettinger has a rich portfolio of guiding data and AI infrastructure-related companies, and his background is rooted in scaling enterprises. He studied Business Marketing at The Ohio State University. Rather than focusing on engineering or academia, Ettinger has leaned into growth, revenue, and, most importantly, figuring out how to take emerging technologies and turn them into sustainable businesses. Over the years, Andrew Ettinger has developed a reputation for helping startups move from early traction to serious revenue scale. A significant part of that was developing go-to-market strategies, building sales teams, establishing customer success structures, forging partnerships, and addressing the operational side that often determines whether a tech company can sustain its momentum beyond its early stages. One of Andrew Ettinger's more visible roles was at Pivotal Software, where he was involved during a high-growth phase. The company expanded rapidly, and Ettinger played a part in scaling revenue significantly before its IPO. Later, at Astronomer, he worked in global sales leadership, helping expand enterprise adoption of data and open-source tools. The roles at Pivotal Software and Astronomer helped Ettinger master the art of commercialising complex technical products for large customers. As already mentioned, Andrew Ettinger served as Chief Revenue Officer at Appen before becoming CEO at Hume AI. Appen provides data and evaluation services used to train and improve machine learning systems. That role put him right in the middle of the AI infrastructure world, working with major labs and technology companies, and gave Hume AI's new CEO direct exposure to how modern AI products are built and deployed. Hume AI, in the coming months, will be eyeing a fresh restart, as its previous CEO, Alan Cowen, along with several of the top engineers, got snapped up by Google in January 2026 in another incident of talent poaching, where promising individuals from small AI startups are being "inducted" into the fold of global tech titans. Cowen and his former Hume AI colleagues will now work with DeepMind to improve Gemini's voice features, as per WIRED. While Hume AI will continue to supply its technology to other AI firms, Andrew Ettinger, who joined the company a couple of weeks back before being promoted as the CEO, told TechCrunch that Google has a "non-exclusive right to certain technologies, and we'll be infusing that into their processes." According to reports, his immediate priority will be to release new models in the coming months and set up Hume AI to bring in USD 100 million in revenue this year. Hume AI, to some extent, has become a victim of the new trend called "acqui-hire," where tech biggies poach top AI talent (including startups' teams) to stay ahead of the innovation curve, while skirting regulatory scrutiny by acquiring a startup's talented individuals rather than the company outright. In 2025, Google followed the same template by acquiring viral AI coding startup Windsurf's CEO and other top researchers. OpenAI, which itself started as a non-profit research lab in 2015, has been a prominent practitioner of acqui-hire, bringing in several startup teams in recent months, including Convogo and Roi. Hume AI, which dubs its model as the "World's Most Realistic and Expressive Voice AI," has customised the tool to understand a user's emotions and mood based on their voice. In 2024, the startup launched its "Empathetic Voice Interface," a conversational AI with emotional intelligence. The company has raised funding close to USD 80 million to date, according to PitchBook. It only made sense for Google, which has been steadily improving its Gemini Live feature, which allows a user to have conversations with the chatbot, to go after Alan Cowen and his colleagues to refine the tech giant's product further and beat the industry competition.

Appen
Oct 22nd, 2024
Message from Appen CEO Ryan Kolln: Payment Issues for Contributors Are Being Resolved; Next Steps

At the beginning of September, Appen launched its new contributor platform, CrowdGen.

Business News Australia
Oct 11th, 2024
Appen returns to underlying profitability, rattles the tin for $50m to fund GenAI opportunities

After pulling itself up by the bootstraps when a major contract fell through with Google earlier this year, Sydney-based artificial intelligence (AI) annotation and services group Appen (ASX: APX) has announced a return to underlying profitability in the September quarter and is now raising $50 million to take advantage of generative AI (GenAI) opportunities.

Appen
Oct 11th, 2024
How ReflexAI Empowers Veterans with AI Mental Health Support

To meet their goals, ReflexAI partnered with Appen to gather high-quality training data and fine-tune their AI model for realistic, empathetic conversations while prioritizing responsible AI development and human feedback.

INACTIVE