Databricks Engineer - Data Engineer III

Our Deloitte AI & Engineering team to transform technology platforms, drive innovation, and help make a significant impact on our clients’ success. You’ll work alongside talented professionals reimagining and reengineering operations and processes that are critical to businesses. Your contributions can help clients improve financial performance, accelerate new digital ventures, and fuel growth through innovation.

Work you'll do

As a Data Engineer II on the AI & Engineering team, you will be responsible for...

• Develop and optimize data pipelines using Python, PySpark, and SQL within the Databricks Lakehouse platform
• Apply Medallion architecture patterns to organize, transform, and manage data across data layers
• Perform performance tuning for Databricks workloads and SQL queries
• Configure row-level and column-level security and manage data lineage and sharing in Unity Catalog
• Implement continuous integration/continuous delivery (CI/CD) and testing processes for Databricks workflows

A successful candidate would possess these skills:

• Ability to work independently and collaborate as part of a team
• Effective written and verbal communication skills
• Meticulous attention to detail and quality of work product
• Ability to build and sustain professional relationships
• Ability to lead projects or workstreams
• Ability to manage and prioritize multiple tasks in a fast-paced and dynamic environment
• Strong interpersonal skills and professional demeanor
• Ability to meet deadlines
• Ability to provide clear guidance to others

The team

Deloitte’s Government & Public Services (GPS) practice – our people, ideas, technology and outcomes – is designed for impact. Serving federal, state, & local government clients as well as public higher education institutions, our team of professionals brings fresh perspective to help clients anticipate disruption, reimagine the possible, and fulfill their mission promise.

Our AI & Data offering provides a full spectrum of solutions for designing, developing, and operating cutting-edge Data and AI platforms, products, insights, and services. Our offerings help clients innovate, enhance and operate their data, AI, and analytics capabilities, ensuring they can mature and scale effectively.

Qualifications

Required:

• Bachelor’s degree
• Ability to obtain and maintain the required clearance for this role
• 2+ years of Databricks engineering experience
• 4+ years of experience with Python, PySpark, and SQL
• Ability to travel 10–15%, on average, based on the work you do and the clients and industries/sectors you serve
• Must be legally authorized to work in the United States without the need for employer sponsorship, now or at any time in the future

Preferred:

• Experience using Medallion architecture in the Databricks Lakehouse platform
• Experience configuring row-level and column-level security, data lineage, and data sharing in Unity Catalog
• Experience implementing continuous integration/continuous delivery (CI/CD) and testing processes for Databricks
• Experience with performance tuning in Databricks

The wage range for this role takes into account the wide range of factors that are considered in making compensation decisions including but not limited to skill sets; experience and training; licensure and certifications; and other business and organizational needs. The disclosed range estimate has not been adjusted for the applicable geographic differential associated with the location at which the position may be filled. At Deloitte, it is not typical for an individual to be hired at or near the top of the range for their role and compensation decisions are dependent on the facts and circumstances of each case. A reasonable estimate of the current range is $107,700 to $179,500.

You may also be eligible to participate in a discretionary annual incentive program, subject to the rules governing the program, whereby an award, if any, depends on various factors, including, without limitation, individual and organizational performance.