Director Data Engineering

-

Core Specialty’s Data Environment is the organizational data ecosystem that supports enterprise requirements for data management, consolidation, insights, and reporting. The Data & Analytics organization ingests and manages data from over 200 distinct sources and growing, which serves business intelligence, AI/ML models, and enterprise decision-making. Maintaining the integrity, accuracy, and completeness of this extensive data environment requires a scalable and automated quality assurance strategy. The Director of Engineering – QA Automation will be responsible for designing and implementing this strategy, integrating quality processes throughout the data platform—including ingestion pipelines, semantic models, and machine learning workflows. This role will require 40% hands on involvement to start in order to build out the architecture, blueprints, and coding standards required for QA Engineers to adhere to. It may also include development efforts as needed.

Key Accountabilities/Deliverables:

• Define and lead the QA automation strategy for data engineering, analytics, and machine learning initiatives.

• Architect and implement scalable, reusable automation frameworks for data pipelines, ingestion models, semantic layers, and ML workflows.

• Collaborate with Data Engineering, Data Science, and DevOps teams to integrate automated testing into CI/CD pipelines.

• Establish coding standards, architectural patterns, and best practices for test automation across the organization.

• Develop automated validation for data quality, schema enforcement, lineage tracking, and model performance.

• Mentor and grow a high-performing QA team with deep domain knowledge in data and analytics.

• Drive continuous improvement in testing tools, processes, and observability.

• In addition to the above key responsibilities, you may be required to undertake other duties from time to time as the Company may reasonably require.

Technical Knowledge and Understanding:

• Programming Languages: Python (preferred), SQL, Java/Scala (for Spark-based pipelines)

• Automation Frameworks: PyTest, Great Expectations, dbt tests, Selenium, Airflow test suites

-

At Core Specialty, you will receive a competitive salary and opportunities for professional development and advancement.  We offer medical, dental, vision, and life insurances; short and long-term disability; a Company-match of 100% of a 6% contribution 401(k) plan; an Employee Assistance Plan; Health Savings Account, Flexible Spending Account, Health Reimbursement Account, and a wellness program