B

Black Duck

Open source risk management and audits

Principal Data Engineer

Full-Time
No salary listed
Expert
Belfast, UK
HybridHybrid work model; details TBD. Minimal travel. Collaboration hours overlap with UK and US time zones.
Data & AnalyticsData Engineering

About the job

Requirements
  • Significant experience building and operating production data platforms at scale, including on-call and operational ownership.
  • Strong SQL and Python skills used to build pipelines, services, and automation.
  • Hands-on experience running cloud systems on AWS and Google Cloud at the IaaS level, including compute, storage, networking, and IAM.
  • Practical experience with both operational databases (RDS-style) and analytics stores (columnar/OLAP), including performance tuning.
  • Strong data modeling ability, including schema evolution, conformed dimensions, and one-source-of-truth metric definitions.
  • A track record of delivering data products that other teams or customers depend on, with clear contracts and reliability expectations.
  • Ability to make sound engineering tradeoffs across latency, accuracy, cost, and security without creating brittle complexity.
  • Experience with lakehouse patterns and open table formats (or similar), including governance and table maintenance.
  • Experience with production orchestration and streaming systems for batch and real-time workloads, and safely managing backfills.
  • Familiarity with ML data needs, including training/serving splits, feature-ready datasets, and evaluation datasets, and AI-adjacent workflows.
Responsibilities
  • Lead the design and build-out of cross-product data services for multiple product lines from one governed data plane.
  • Define the customer data plane model, including canonical customer identifiers, shared dimensions, and consistent facts used across products.
  • Build and operationalize ingestion patterns for batch, streaming, and event data, with repeatable onboarding for new sources.
  • Own the operational playbook for data reliability, including data contracts, quality checks, lineage, monitoring, and incident response.
  • Implement and run access methods that make data usable, including curated datasets, secure query interfaces, and product-ready data APIs where needed.
  • Productize customer-facing data products (datasets, metrics, exports, and feeds) with versioning, documentation, and clear ownership.
  • Design data models for operational systems (RDS) and analytics stores (columnar/OLAP), including performance and cost tuning.
  • Ensure data products support ML workflows through trusted training datasets, feature-ready outputs, and consistent definitions for decision-making.
  • Enable AI automation by delivering reliable, low-latency, governed data products that can be used safely in automated workflows.
  • Partner with product, engineering, and security stakeholders to align data products with roadmap priorities and customer outcomes.
  • Raise the technical bar through architecture reviews, standards, and mentoring while staying hands-on in key systems.
Desired Qualifications
  • Experience building self-service data platforms, including catalog, discoverability, and access controls, used by multiple teams.
  • Experience in regulated or security-sensitive environments, including retention, auditing, and data access controls.

About the company

Black Duck Software helps organizations manage open source risk by offering Software Composition Analysis (SCA) and Open Source Audits. Its products scan software to find security vulnerabilities and license compliance issues in open source components and provide fixes. The Open Source Audits support due diligence for mergers and acquisitions and internal audits. Revenue comes from licenses for the tools plus professional services for audits and consultations. The platform relies on a large database of open source components, vulnerabilities, and licenses to enable fast, accurate analysis. The goal is to help security, development, and legal teams ensure software is secure and legally compliant throughout the software development lifecycle and during M&A.

Company Size

1,001-5,000

Company Stage

Acquired

Total Funding

$87.5M

Headquarters

Burlington, Massachusetts

Founded

2002

Get referred to Black Duck

See people who can refer or advise you

Simplify Jobs

Simplify's Take

What believers are saying

  • Polaris scan volumes rose over 100% in early 2026, signaling demand acceleration.
  • CRA deadlines beginning September 11, 2026 expand Black Duck’s compliance-selling window.
  • The 2026 OSSRA report showed 107% higher vulnerabilities per codebase, boosting urgency.

What critics are saying

  • Aikido, Snyk, and GitHub Advanced Security commoditize Black Duck’s developer workflows by 2026.
  • Black Duck still carries legacy on-prem complexity, slowing wins against faster SaaS competitors.
  • If AI-native rivals own open-source governance, Clearlake and Francisco Partners face a stranded asset.

What makes Black Duck unique

  • Black Duck’s 2026 Gartner Leader status validates its enterprise supply-chain security depth.
  • Its BDSA enrichment offsets NIST’s April 2026 NVD deprioritization.
  • ContextAI and Signal fuse deterministic analysis with AI governance for regulated codebases.

Help us improve and share your feedback! Did you find this helpful?

Benefits

  • Health Insurance
  • Dental Insurance
  • Vision Insurance
  • Flexible Work Hours
  • Professional Development Budget
  • Paid Vacation

Growth & Insights and Company News

6 month growth

0%

1 year growth

-2%

2 year growth

48%
PR Newswire
Sep 8th, 2026
Black Duck joins Anthropic's Project Glasswing to secure critical software with AI

Black Duck has joined Anthropic's Project Glasswing, an industry initiative aimed at securing critical software infrastructure using advanced AI for defensive cybersecurity. The application security company will apply Mythos, Anthropic's AI system, across its full security portfolio. This will combine AI-accelerated vulnerability discovery with remediation workflows, risk-based prioritisation, and compliance-driven governance. "AI is transforming the economics and speed of vulnerability discovery and exploit development," said Dipto Chakravarty, Black Duck's Chief Product & Technology Officer. He explained that pairing Mythos with Black Duck's existing capabilities will enable faster risk reduction whilst maintaining the transparency and auditability required by enterprise security teams. Black Duck specialises in application security, combining deterministic analysis with AI reasoning to identify and fix security issues in code written by developers, generated by AI, or assembled from open source.

PR Newswire
Jul 14th, 2026
Black Duck launches AI-powered Coverity with EU CRA compliance features

Black Duck has launched new AI-powered features for its Coverity static analysis solution, designed to support modern development workflows and emerging regulatory compliance requirements. The updates include AI-assisted vulnerability triage, support for the EU Cyber Resilience Act (CRA), and analysis capabilities for Rust 1.92. Coverity's AI features run on customers' own large language models, allowing organisations to maintain control over code and scan data processing — a requirement for regulated industries with strict data governance policies. The solution now offers streamlined navigation and improved issue filtering to help teams manage large volumes of security findings more efficiently. According to Chief Product & Technology Officer Dipto Chakravarty, the enhancements combine deterministic precision with AI speed whilst maintaining auditability. All announced capabilities are now generally available for customer deployment. Existing Coverity customers receive these AI-powered features whilst retaining the deterministic, auditable scan results required by regulated industries.

PR Newswire
Jun 16th, 2026
Black Duck launches AI-powered security tools to combat surge in software vulnerabilities

Black Duck has announced significant enhancements to its Polaris Platform, designed to help organisations defend against AI-driven cyberattacks and manage the surge in supply chain vulnerabilities. The updates focus on three areas: eliminating application security testing gaps, preparing for increased vulnerability disclosures, and automating remediation pipelines. The enhancements address threats from sophisticated AI models that enable attackers to exploit multiple vulnerabilities rapidly. Polaris scan volumes have increased over 100% in the first five months of 2026 as organisations accelerate security testing. New capabilities include improved vulnerability detection, rapid response tools for supply chain components, and AI-enabled application security features. Black Duck expects vulnerability disclosures to exceed 50,000 in 2026, potentially reaching 200,000 by 2028, as maintainers use AI to identify and patch security flaws.

PR Newswire
Mar 23rd, 2026
Black Duck launches Signal, AI-powered security solution for AI-generated code

Black Duck has launched Signal, an AI-powered application security solution designed to secure AI-generated code in autonomous development workflows. The platform uses an agentic AI architecture where specialised agents analyse vulnerabilities, validate exploitability and recommend fixes. Signal is powered by ContextAI, Black Duck's application security model containing over 20 years of security intelligence. This enables the system to assess risk with higher accuracy than solutions built solely on general-purpose AI models. The platform integrates directly into modern software development through model context protocol and APIs that support AI coding assistants and automated pipelines. CEO Jason Schmitt said AI is "actively authoring software", and Signal brings intelligence and governance to that reality. The solution is now generally available and will be showcased at RSA Conference in San Francisco from 23–26 May.

PR Newswire
Feb 12th, 2026
Black Duck expands Polaris integrations for automated DevSecOps across GitHub, GitLab, Azure DevOps, and Bitbucket

Black Duck has launched enhanced integrations for its Polaris Platform across major source code management systems including GitHub, GitLab, Azure DevOps and Bitbucket. The updates enable automated repository onboarding, continuous monitoring and event-based scanning for enterprises managing thousands of code repositories. The enhancements allow organisations to automatically onboard repositories without manual configuration and trigger scans during pull requests. The platform includes Black Duck Signal for AI-powered security insights and Code Sight, an IDE plugin providing real-time feedback to developers. The integrations support customisable scanning options and automatically synchronise security policies and user access controls across repositories. The features are immediately available to existing customers through Polaris Platform settings, aiming to streamline DevSecOps operations at enterprise scale.