Lead Data Engineer
Posted on 7/19/2023
INACTIVE
Thrive Market

501-1,000 employees

Membership-based e-commerce for discounted organic products
Company Overview
Thrive Market stands out as a leading e-commerce platform dedicated to making high-quality, natural, and organic products accessible and affordable for all American families. Their unique membership model not only offers customers 30-50% off retail prices on a wide range of products, but also sponsors free memberships for those in need, demonstrating a strong commitment to social responsibility. The company's rigorous product curation and sourcing standards ensure that all items are organic or sustainably sourced, setting them apart in the industry for their dedication to health and sustainability.
Consumer Goods

Company Stage

Series C

Total Funding

$424.2M

Founded

2013

Headquarters

Los Angeles, California

Growth & Insights
Headcount

6 month growth

4%

1 year growth

10%

2 year growth

30%
Locations
Remote
Experience Level
Entry
Junior
Mid
Senior
Expert
Desired Skills
Agile
Redshift
Python
Airflow
Git
Apache Spark
SQL
Apache Kafka
Java
AWS
Jenkins
Hadoop
Data Analysis
Snowflake
CategoriesNew
Data & Analytics
Requirements
  • Advanced knowledge of data architectures, data pipelines, real time processing, streaming, networking, and security
  • Hands on experience programming in Python, Scala or Java
  • Expertise with RDBMS and Data Warehousing (Strong SQL) with Redshift,Snowflake or similar
  • In-depth knowledge and experience with data and information architecture patterns and implementation approaches for Operational Data Stores, Data Warehouses, Data Marts and Data Lakes
  • Proficiency in logical/physical data architecture, design and development
  • Experience in Data lake / Big data analytics platform implementation either cloud based or on-premise; AWS preferred
  • Experience working with high volumes of data; experience in design, implementation and support of highly distributed data applications
  • Experience with Development Tools for CI/CD, Unit and Integration testing, Automation and Orchestration E.g. GitHub, Jenkins, Concourse, Airflow, Terraform
  • Experience with writing Kafka producers and consumers or experience with AWS Kinesis
  • Hands-on experience developing a distributed data processing platform with Big Data technologies like Hadoop, Spark etc
  • A knack for independence (hands-on) as well as team work
  • Excellent analytical and problem-solving skills, often in light of ill-defined issues or conflicting information
  • Experience with streaming data ingestion, machine-learning, Apache Spark a plus
  • Adept in the ability to elicit, gather, and manage requirements in an Agile delivery environment
  • Excellent communication and presentation skills (verbal, written, presentation) across all levels of the organization. Ability to translate ambiguous concepts into tangible ideas
Responsibilities
  • Lead multiple projects and efforts to orchestrate and deliver cohesive data engineering solutions in partnership with various functional teams at Thrive Market
  • Be hands-on and take ownership of the complete cycle of data services, from data ingestion, data processing, ETL to data delivery for reporting
  • Collaborate with other technical teams to deliver data solutions which meet business and technical requirements; define technical requirements and implementation details for the underlying data lake, data warehouse and data marts
  • Identify, troubleshoot and resolve production data integrity and performance issues
  • Collaborate with all areas of data management as lead to ensure patterns, decisions, and tooling is implemented in accordance with enterprise standards
  • Perform data source gap analysis and create data source/target catalogs and mappings
  • Develop a thorough knowledge and understanding of cross system integration, interactions and relationships in order to develop an enterprise view of Thrive Market's future data needs
  • Design, coordinate and execute pilots/prototypes/POC to provide validation on specific scenarios and provide implementation roadmap
  • Recommend/Ensure technical functionality (e.g. scalability, security, performance, data recovery, reliability, etc.) for Data Engineering
  • Facilitate workshops to define requirements and develop data solution designs
  • Apply enterprise and solution architecture decisions to data architecture frameworks and data models
  • Maintain a repository of all data architecture artifacts and procedures
  • Collaborate with IT teams, software providers and business owners to predict and devise data architecture that addresses business needs for collection, aggregation and interaction with multiple data streams
  • Accountable for the delivery of short to mid term goals for assigned pods/initiatives
  • Responsible for the team's overall output, and meeting deadlines on projects
  • Assumes project management responsibilities for core work or key initiatives
  • Contributes significantly to the development of features with minimal guidance
  • Complete subtasks daily
  • Fixes defects permanently within a reasonable timeframeWrites thorough documentation, comments, and automated tests
  • Demonstrates full knowledge of data tool features, configurations, and systems in functional areas
  • Develops new ideas for teams to produce results more efficiently and rapidly
  • Mentors mid-level and senior engineers to continuously improve performance
  • Collaborates across pods, functional teams, and departments to ensure projects and features are managed to minimize roadblocks
  • Communicates clearly with team members and departments, and delivers meaningful updates to Directors and other Management
  • Demonstrated growth mindset by increasing the quality and velocity of team's work as well as their own leadership capabilities
  • Can manage projects successfully along with timelines and provides technical leadership to the team