Senior Big Data Engineer
Posted on 9/9/2022
Remote • New York, NY, USA
Data Structures & Algorithms
Google Cloud Platform
- Bachelors or Masters in Computer Engineering, Computer Science, Electrical Engineering or related field
- 5-8+ years of recent hands-on experience in Java or Scala
- Strong knowledge of collections, multi-threading, JVM memory model, etc
- Solid knowledge of SQL
- Experience in full software development, Agile, and CI/CD
- 5+ years of experience designing and building data pipelines and data-intensive applications
- Experience using Big Data frameworks (e.g., Hadoop, Spark) , databases (e.g., RedShift, Snowflake) for complex data assembly and transformation
- Experience with Spark streaming or Flink
- In-depth understanding of algorithms, performance, scalability, and reliability in a Big Data setting
- Solid understanding of OLTP and OLAP systems, database fundamentals
- Experience building production level systems in a cloud environment (AWS, Azure or GCP)
- Orchestrating data pipelines using tools such as Airflow
- Familiarity with messaging frameworks like Kafka or RabbitMQ
- Architect, design, code and maintain components for aggregating tens of billions of daily transactions
- Work on Big Data technologies such as Hadoop, MapReduce, Kafka, and/or Spark in columnar databases on AWS
- Contribute to the entire software lifecycle including hands-on development, code reviews, testing, deployment, and documentation for streaming & batch ETL and RESTful APIs
- Provide leadership, work collaboratively, and be a mentor in an awesome team
Global technology and data company
IAS's mission is to be the global benchmark for trust and transparency in digital media quality.
- Healthcare: Comprehensive health insurance
- Flexible PTO: Take what you need, when you need it
- Retirement: 401k match in the US, country-specific pension plans
- Parental leave: Full maternity and paternity leave
- Community: Volunteer opportunities
- Celebrating team players: Peer-nominated awards
- Meals and snacks: Stocked kitchens to fuel your day
- Team events: Frequent happy hours, parties and outings
- We Innovate: We build cool stuff. Innovation is at the core of what we do. We build products, deliver solutions, and generate ideas that provide valuable functions for our customers.
- We Are Accountable: We hold ourselves and each other accountable for our conduct with teammates and our customers. We take full ownership for our deliverables.
- We Are One Team: We value and rely on each other. We are inclusive. We show up for each other, and we act with empathy and consideration for the benefit of the team.
- We Do the Right Thing: Regardless of whether anyone is looking or not. We act with honesty, transparency, and integrity in working with each other and with our customers.
- We Are Customer Obsessed: We put the customer front and center of everything that we do. Our customers’ success is our success.
- We have a Bias for Action: Speed matters in business. We move at high velocity and we privilege risk-taking.