Database/Data Infrastructure Engineer

India

Posted on 7/19/2023

INACTIVE

51-200 employees

Website

Cloud-native lakehouse service company

Company Stage

Series A

Total Funding

$33M

Founded

2021

Headquarters

Menlo Park, California

Growth & Insights

Headcount

6 month growth

↑ 20%

1 year growth

↑ 73%

2 year growth

↑ 300%

Locations

Remote

Experience Level

Entry

Junior

Mid

Senior

Expert

Desired Skills

Apache Spark

Data Analysis

Data Structures & Algorithms

Java

Linux/Unix

Kubernetes

CategoriesNew

DevOps & Infrastructure

Software Engineering

Requirements

Strong, object-oriented design and coding skills (C/C++ and/or Java preferably on a UNIX or Linux platform)
Experience with inner workings of distributed (multi-tiered) systems, algorithms, and relational databases
Deal well with ambiguous/undefined problems; ability to think abstractly; articulate technical challenges and solutions
Speed and hustle → Ability to prioritize across feature development and tech debt
Ability to solve complex programming/optimization problems
Ability to quickly prototype optimization solutions and analyze large/complex data
Good communication skills

Responsibilities

Design new concurrency control and transactional capabilities, that maximizes throughput for competing writers
Design and implement new indexing schemes, specifically optimized for incremental data processing and analytical query performance
Design systems that help scale and streamline metadata and data access from different query/compute engines
Solve hard optimization problems to improve the efficiency (increase performance and lower cost) of distributed data processing algorithms over a Kubernetes cluster
Leverage data from existing systems to find inefficiencies, and quickly build and validate prototypes
Collaborate with other engineers to implement and deploy, safely rollout the optimized solutions in production

Desired Qualifications

Experience working with database systems, Query Engines or Spark codebases
Experience in optimization mathematics (linear programming, nonlinear optimization)
Existing publications of optimizing large-scale data systems in top-tier distributed system conferences
PhD degree with 2+ industrial experience in solving and delivering high-impact optimization projects

About Onehouse

Onehouse delivers a new bedrock for your data, through a cloud-native managed lakehouse service built on an open, interoperable, industry-proven technology. Founded by a former Uber data architect and the creator of Apache Hudi, Onehouse accelerates the inevitable transition of the data lake into a lakehouse, unlocking incremental processing to replace old-school batch processing on the lake. Onehouse makes it possible to blend the ease of use of a warehouse with the scale of a data lake into a fully managed product. Engineers can build data lakes in minutes, process data in seconds, and own data in open source formats instead of being locked away to individual vendors.

https://www.onehouse.ai/

Job Description

Do you live and breathe databases? Do you forget time solving complex optimization problems? Do you enjoy rolling up your sleeves to work on hard systems projects? As a database engineer on Onehouse, you will work on the guts of Apache Hudi’s transactional engine as well as optimize it for diverse customer workloads for Onehouse, by designing algorithms to improve the performance and lower costs for large-scale data processing.

Responsibilities

Design new concurrency control and transactional capabilities, that maximizes throughput for competing writers.
Design and implement new indexing schemes, specifically optimized for incremental data processing and analytical query performance.
Design systems that help scale and streamline metadata and data access from different query/compute engines.
Solve hard optimization problems to improve the efficiency (increase performance and lower cost) of distributed data processing algorithms over a Kubernetes cluster.
Leverage data from existing systems to find inefficiencies, and quickly build and validate prototypes.
Collaborate with other engineers to implement and deploy, safely rollout the optimized solutions in production.

Must Haves

Strong, object-oriented design and coding skills (C/C++ and/or Java preferably on a UNIX or Linux platform)
Experience with inner workings of distributed (multi-tiered) systems, algorithms, and relational databases
Deal well with ambiguous/undefined problems; ability to think abstractly; articulate technical challenges and solutions
Speed and hustle → Ability to prioritize across feature development and tech debt.
Ability to solve complex programming/optimization problems.
Ability to quickly prototype optimization solutions and analyze large/complex data.
Good communication skills.

Bonus Skills

Experience working with database systems, Query Engines or Spark codebases.
Experience in optimization mathematics (linear programming, nonlinear optimization)
Existing publications of optimizing large-scale data systems in top-tier distributed system conferences.
PhD degree with 2+ industrial experience in solving and delivering high-impact optimization projects.

Who We Are

At Onehouse, our mission is to aid companies of all sizes in supercharging their data engineering/data science, by automating painful data infrastructure buildout. We are a team of self-driven, inspired, and seasoned builders that have created large-scale data systems, as well as globally distributed platforms that sit at the heart of some of the most well known companies out there including Uber, Linkedin, Confluent, Microsoft. We are set out on an ambitious goal to build the world’s best fully managed and self-optimizing data lake platform. We are very well funded and backed by some of the top-tier VCs in Silicon Valley, and as well as numerous well-known angel investors from top Silicon Valley companies.

Why join us

Fun team, challenging problems! One day, we will be managing the largest database in existence!

Contribute directly to open source, including an exciting and growing data project - Apache Hudi

Create instant impact by contributing to Hudi, which is already in use by numerous large enterprises globally

Experienced team with numerous staff level engineers, to learn and grow with.

Early opportunity on a very happening space, everybody agrees the next few years will reshape the data landscape

Founding team is the creator of a large, fast-growing technology category - transactional data lakes

Benefits

Unlimited PTO

Paid Parental Leave

Equity

Flexible Schedule

We are growing fast and looking for rising talent who can grow with us to become future leaders of the team. Come help build this unicorn-to-be!