Facebook pixel

Data Engineer II
Posted on 1/14/2023
INACTIVE
Locations
United States
Experience Level
Entry
Junior
Mid
Senior
Expert
Desired Skills
Apache Hive
Apache Spark
AWS
Data Analysis
Data Science
Data Structures & Algorithms
Elasticsearch
Hadoop
Airflow
MongoDB
Redshift
Scala
Terraform
Kubernetes
Python
Yarn
Quality Assurance (QA)
Requirements
  • Hadoop, Yarn, Spark, MongoDB, Hive
  • AWS EMR/EC2/Lambda/kinesis/S3/Glue/DynamoDB/API Gateway, Redshift
  • ElasticSearch, Airflow, and Terraform
  • Scala, Python
  • 3+ years of professional experience working in data engineering
  • BS / MS in Computer Science, Engineering, Mathematics, or a related field
  • You have built large-scale data products and understand the tradeoffs made when building these features
  • You have a deep understanding of system design, data structures, and algorithms
  • Experience (or a strong interest in) working with Python or Scala
  • Experience with working with a cluster manager (YARN / Mesos / Kubernetes)
  • Experience with distributed computing and working with Spark, Hadoop, or MapReduce Framework
  • Experience working on a cloud platform such as AWS
  • Experience with ETL in general
  • Experience working with Apache Airflow
  • Experience working with AWS Glue
  • Experience in Machine Learning and Information Retrieval
Responsibilities
  • Build core components of data platform which will serve various types of consumers including but not limited to data science, engineers, product, and QA
  • Build various data ingestion and transformation job/s as and when they are needed
  • Productionize our machine learning models and algorithms into data-driven feature MVPs that scale
  • Leverage best practices in continuous integration and deployment to our cloud-based infrastructure
  • Build scalable data services to bridge the gap between analytics and application space
  • Optimize data access and consumption for our business and product colleagues
  • Develop an understanding of key product, user, and business questions
Stash

201-500 employees

Personal finance & banking platform