Data Engineer @ Paytm Labs

Data Engineer

Posted on 1/20/2023

INACTIVE

Locations

Toronto, ON, Canada

Experience Level

Entry

Junior

Mid

Senior

Expert

Desired Skills

Apache Hive

Apache Spark

Apache Kafka

Data Analysis

Java

Scala

Apache Flink

Python

Requirements

Degree in Computer Science, Engineering, or a related field
You have previously worked on building serious data pipelines ingesting and transforming > 10 ^6 events per minute and terabytes of data per day
You are passionate about producing clean, maintainable, and testable code as part of a real-time data pipeline
You understand how microservices work and are familiar with concepts of data modeling
You can connect different services and processes together even if you have not worked with them before and follow the flow of data through various pipelines to debug data issues
You have worked with Spark and Kafka before and have experimented with or heard about Flink/Druid/Ignite/Presto/Athena and understand when to use one over the other
On a bad day maintaining a zookeeper and bringing up a cluster doesn't bother you
You may not be a networking expert but you understand issues with ingesting data from applications in multiple data centers across geographies, on-premise, and cloud and will find a way to solve them
Proficient in Java/Scala/Python/Spark

Responsibilities

Work directly with Machine Learning Engineers and Platform Engineering Team to create reusable experimental and production data pipelines
Understand, tune, and master the processing engines (like Spark, Hive, Samza, etc) used day-to-day
Keep the data whole, safe, and flowing with expertise on high-volume data ingest and streaming platforms (like Spark Streaming, Kafka, etc)
Sheppard and shape the data by developing efficient structures and schema for the data in storage and transit
Explore as many new technology options for data processing, and storage, and share them with the team
Develop tools and contribute to open source wherever possible
Adopt problem-solving as a way of life - always go to the root cause

About Paytm Labs:

At Paytm Labs, we’re on a mission to provide useful technological solutions that enrich and empower millions of people in their daily lives. We apply big data, artificial intelligence, and machine learning to bring the next generation of financial products and services to global markets.

We recently soft-launched Pi, our full end-to-end fraud management system that is able to make real-time decisions. It works as a dynamic rule engine to make day-to-day analyses and procedures. Check out more about Pi here: pi.paytm.com.

Job Description:

If working with billions of events, petabytes of data, and optimizing for the last millisecond is something that excites you then read on! We are looking for Data Engineers who have seen their fair share of messy data sets and have been able to structure them for building useful AI products.

You will be working on writing frameworks for real-time and batch pipelines to ingest and transform events from hundreds of applications every day. Our ML and Software engineers consume these for building data products like personalization and fraud detection. You will also help optimize the feature pipelines for fast execution and work with software engineers to build event-driven microservices.

You will get to put cutting-edge tech in production and freedom to experiment with new frameworks, try new ways to optimize, and resources to build the next big thing in fintech using data!

Responsibilities:

Work directly with Machine Learning Engineers and Platform Engineering Team to create reusable experimental and production data pipelines
Understand, tune, and master the processing engines (like Spark, Hive, Samza, etc) used day-to-day
Keep the data whole, safe, and flowing with expertise on high-volume data ingest and streaming platforms (like Spark Streaming, Kafka, etc)
Sheppard and shape the data by developing efficient structures and schema for the data in storage and transit
Explore as many new technology options for data processing, and storage, and share them with the team
Develop tools and contribute to open source wherever possible
Adopt problem-solving as a way of life – always go to the root cause

Qualifications:

Degree in Computer Science, Engineering, or a related field
You have previously worked on building serious data pipelines ingesting and transforming > 10 ^6 events per minute and terabytes of data per day
You are passionate about producing clean, maintainable, and testable code as part of a real-time data pipeline
You understand how microservices work and are familiar with concepts of data modeling
You can connect different services and processes together even if you have not worked with them before and follow the flow of data through various pipelines to debug data issues
You have worked with Spark and Kafka before and have experimented with or heard about Flink/Druid/Ignite/Presto/Athena and understand when to use one over the other
On a bad day maintaining a zookeeper and bringing up a cluster doesn’t bother you
You may not be a networking expert but you understand issues with ingesting data from applications in multiple data centers across geographies, on-premise, and cloud and will find a way to solve them
Proficient in Java/Scala/Python/Spark

Why join Pi!

Flexibility to WFH
For the fourth year in a row, we are proud to announce that we have been certified as a Great Place to Work
We are an open work environment that fosters collaboration, ownership, creativity, and urgency
We ensure flexible hours outside of our core working hours
Enrolment in the Group Health Benefits plan right from day 1, no waiting period
Team building events
Online learning through the platform, UDEMY
Fuel for the day: Weekly delivery of groceries, and all types of snacks
Catered lunches and desserts on a monthly basis
Daily fun in the office with our competitive games of Ping Pong, Pool, multiple board games, and video games

Notice for Job Applicants

We are an equal opportunity employer and value diversity and uniqueness at our company. We thank all applicants, however, only those selected for an interview will be contacted.

Paytm Labs is committed to meeting the accessibility needs of all individuals in accordance with the Accessibility for Ontarians with Disabilities Act (AODA) and the Ontario Human Rights Code (OHRC). Should you require accommodations during the recruitment and selection process, please let us know.

5,001-10,000 employees

Website

Financial services

Company Overview

Paytm Labs began as a research and development division of Paytm. They pride themselves in building products and services that enrich and empower people's lives daily. They believe that culture is a product of everyday behaviours that are celebrated and amplified.

Benefits

Health & Wellbeing - Your group health benefits are 100% funded by us, and start on your first day.
Travel the World - Even though we are currently working remotely, usually our employees make frequent international business-related trips while working on exciting projects.
Continuous Learning - We're passionate about learning and strive to constantly improve and innovate. We learn from others, our challenges & our successes.
Connection - In this virtual world, we stay connected through events like game nights, movie nights, arts & crafts sessions (like Origami), meditation sessions, and much more.
Autonomy & Flexibility - You get to define how you want to make an impact. We truly believe that great things happen when people are given the trust and freedom to create their own path.
Beautiful Office - Whenever we return back to the office, you’ll get to check out our awesome workspace, complete with pool and ping pong tables, a snack-filled kitchen, and tons of cozy work spaces.

Company Core Values

Intent Overrides Ability
The Power of Asking, ‘Why?’
We Before Me
Ownership
Do Great Things, Fast