Simplify Logo

Full-Time

Senior Backline Engineer

Spark

Posted on 1/19/2024

Databricks

Databricks

5,001-10,000 employees

Unified data, analytics, and AI platform

Data & Analytics
Consulting
Hardware
Industrial & Manufacturing
Consumer Software
Enterprise Software
AI & Machine Learning

Compensation Overview

$102k - $180.5kAnnually

Senior

United States

Category
Software Engineering
Required Skills
Microsoft Azure
Python
Apache Spark
SQL
Apache Kafka
Java
AWS
Elasticsearch
Scala
Hadoop
Requirements
  • Minimum 6 years' experience developing, testing, and sustaining Python or Java or Scala-based applications
  • Experience in Big Data/Hadoop/Spark/Kafka/Elasticsearch data pipelines
  • Good understanding of AI & ML using python ex:sklearn
  • Proficiency in GQL (GraphQL) and its applications in AI platforms
  • Hands-on experience with YAML for configuration and data representation
  • Hands-on experience with SQL-based database systems
  • Experience in JVM, GC, Thread dump-based troubleshooting
  • Experience with AWS or Azure related services
  • Bachelor's degree in Computer Science or a related field
Responsibilities
  • Troubleshoot, resolve and suggest deep code-level analysis of Spark to address complex customer issues related to Spark core internals, Spark SQL, Structured Streaming and Databricks Delta
  • Provide best practices guidance around Spark runtime performance and usage of Spark core libraries and APIs for custom-built solutions developed by Databricks customers
  • Help the support team with detailed troubleshooting guides and runbooks
  • Contribute to automation and tooling programs to make daily troubleshooting efficient
  • Work with the Spark Engineering Team and spread awareness of upcoming features and releases
  • Identify Spark bugs and suggest possible workarounds
  • Demonstrate ownership and coordinate with engineering and escalation teams to achieve resolution of customer issues and requests
  • Participate in weekend and weekday on call rotation
Desired Qualifications
  • Comfortable with compiling, building and navigating the Apache Spark source code
  • Comfortable with identifying and applying patches/bug fixes to the Apache Spark source code

Databricks provides a Lakehouse Platform integrating data management, analytics, and AI capabilities across a range of industries. Employing tools like Delta Lake and Databricks SQL enhances operational efficiency and precision. As a workplace, it offers a collaborative and technology-driven environment, promoting growth through cutting-edge machine learning solutions and a commitment to leading the industry in data innovation. This makes it an inspiring place for tech professionals to advance their careers in a vibrant and forward-thinking setting.

Company Stage

Series I

Total Funding

$4.2B

Headquarters

San Francisco, California

Founded

2013

Growth & Insights
Headcount

6 month growth

15%

1 year growth

28%

2 year growth

89%

Benefits

Extended health care including dental and vision

Life/AD&D and disability coverage

Equity awards

Flexible Vacation

Gym reimbursement

Annual personal development fund

Work headphones reimbursement

Employee Assistance Program (EAP)

Business travel accident insurance

Paid Parental Leave

INACTIVE