Data Infrastructure Engineer
AI/ML
Posted on 2/10/2024
Twilio

5,001-10,000 employees

Customer engagement platform & developer of communications APIs
Company Overview
Twilio's mission is to fuel the future of communications. By making communications a part of every software developer's toolkit, Twilio is enabling innovators across every industry to reinvent how companies engage with their customers.
Data & Analytics

Company Stage

N/A

Total Funding

$768.1M

Founded

2008

Headquarters

San Francisco, California

Growth & Insights
Headcount

6 month growth

-2%

1 year growth

-17%

2 year growth

-13%
Locations
Remote in USA
Experience Level
Entry
Junior
Mid
Senior
Expert
Desired Skills
Airflow
Apache Spark
AWS
Natural Language Processing (NLP)
Data Analysis
CategoriesNew
AI & Machine Learning
DevOps & Infrastructure
Requirements
  • Experience using big data technologies (Airflow, Spark, pySpark, Presto, Athena)
  • AWS experience
  • Demonstrated prior experience in creating data pipelines for text data sets NLP/ large language models
  • Experience building and maintaining ETL (managing high-quality reliable ETL pipelines)
  • Previous experience in building data pipelines for conversational AI APIs
Responsibilities
  • Dive into our dataset and design, implement, and scale data pre/post-processing pipelines
  • Work on applied ML solutions in the areas of query processing, data mining, cleaning, normalizing and modeling
  • Design and build data platforms & frameworks for processing high volumes of data, in real-time as well as batch, that will be used across engineering teams
  • Build data processing streams for cleaning and modeling text data for LLMs
  • Shape the decision-making process for data architecture to effectively deliver insights to the business
  • Research and evaluate new technologies in the big data space to guide our continuous improvement
  • Collaborate with multi-functional teams to help tune the performance of large data applications
  • Partner with Data Engineering, IT, and business units to ensure the creation of high-quality data solutions that are consistent, clean, and integrated
  • Conduct data exploration and profiling using analysis, design, and presentation tools
  • Identify and assess business impacts resulting from strategic initiatives
  • Deliver analytics solutions to support key business decisions and enhance operational efficiency