Salary Range: 100000 to 140000 (Currency: USD) (Pay period: per-year-salary)

Responsibilities:

• Design, develop, and maintain scalable ETL/ELT pipelines using Databricks and other big data technologies.
• Optimize data workflows to handle large volumes of data efficiently.
• Build and manage data warehouses and data lakes to store structured and unstructured data.
• Utilize SQL, Python, Spark for data extraction, transformation, and loading (ETL) processes.
• Work closely with data analysts and data scientists to understand their data needs and ensure the availability of clean, reliable data.
• Integrate data from various sources, ensuring consistency and accuracy across the data ecosystem.
• Implement data quality checks to ensure data accuracy, completeness, and consistency.
• Develop and enforce data governance policies and procedures to maintain high data quality standards.
• Develop and support BI tools and dashboards, providing business insights and data-driven decision-making support.
• Work with stakeholders to understand reporting requirements and deliver actionable insights.
• Automate repetitive data processing tasks to improve efficiency and reduce manual work.
• Continuously monitor and improve data pipeline performance, addressing bottlenecks and optimizing resources.
• Document data processes, workflows, and architecture for future reference and knowledge sharing.
• Ensure compliance with data security and privacy regulations, such as GDPR, HIPAA, etc.

Education:

• Bachelor’s degree in Computer Science, Software Engineering, Mathematics, or a related technical field is preferred.

Experience:

• 8+ years of technical experience with a strong focus on Big Data technologies in any of these areas: software engineering, integrations, data warehousing, data analysis, business intelligence, preferably at a technology or biotech/pharma company
• Proficiency in Databricks for data engineering tasks.
• Advanced knowledge of SQL for complex queries, data manipulation, and performance tuning.
• Strong programming skills in Python for scripting and automation.
• Experience with Big Data tools (e.g., Spark, Hadoop) and data processing frameworks.
• Familiarity with BI tools (e.g., Tableau, Power BI) and experience in developing dashboards and reports.
• Experience with cloud platforms & tools like Azure ADF or Databricks.
• Familiarity with data modeling and data architecture design.
• Understanding of machine learning concepts and their application in data engineering.

Skills:

• Keen attention to detail and bias for action.
• Excellent organizational skills and proven ability to multi-task.
• Ability to influence without authority and lead successful teams.
• Strong interpersonal skills with the ability to work effectively with a wide variety of professionals.
• Ability to lead back-end data initiatives – identify data sources, write scripts, transfer and transform data, and automate processes.
• Ability to understand source data, its strengths, weaknesses, semantics, and formats.
• Excellent knowledge of logical and physical data modeling concepts (relational and dimensional).