Principal Member Technical Staff - Data engineer

Job Description

Nielsen’s Digital Content AI team builds and operates data-processing pipelines that turn digital audience and interaction signals into reliable, analytics-ready datasets. As a principal-level individual contributor, you will bring a Solution first mindset and strong data understanding to the full lifecycle: understand new data sources, build reusable ingestion and transformation workflows, validate and enrich data, publish it to downstream data platforms, and keep production processing dependable. You will lead multiple cross-functional projects through measurable business deliverables, providing technical direction through expertise and influence rather than formal people management. You will work across mobile, Connected TV (CTV), desktop/laptop and web use cases, partnering with product, data collection, platform, quality and downstream analytics teams.

● Lead the technical design and delivery of batch and near-daily data pipelines, from raw inputs through curated outputs and publication to the Master Data Layer (MDL).
● Profile and interpret incoming telemetry from diverse data sources; identify data gaps and translate source formats into dependable processing logic.
● Apply an AI-first approach to engineering and operations: identify appropriate opportunities for AI-assisted data exploration, code and test generation, classification, anomaly detection and workflow automation; validate outputs rigorously and use AI responsibly with appropriate privacy and security controls.
● Design and operate real-time and streaming event workflows alongside batch pipelines, managing event validation, schema evolution, ordering and duplication, late or replayed events, processing latency, backpressure and failure recovery.
● Build and maintain parsers, configuration-driven workflows, event/session logic, business rules and schema mappings for application usage and in-app activity across mobile, CTV, desktop/laptop and web products.
● Develop data-stitching and transformation logic that correlates relevant sources, derives sessions and engagement measures, and produces consistent intermediate and curated datasets.
● Deliver data processing for streaming/video, e-commerce and advertising use cases, including event classification, purchase and product/category/search activity, app and web usage, and incremental dataset updates.
● Integrate reference and enrichment data—including content metadata and classification inputs—where required; collaborate with machine-learning and platform teams to operationalize model-based classification and enrichment.
● Build reusable frameworks for onboarding new applications, platforms, data sources and markets; support configuration, historical backfills, reprocessing and safe daily refreshes.
● Orchestrate and operate workflows using tools such as Airflow and AWS workflow services; monitor schedules, dependencies, retries, failures and data availability, and troubleshoot production issues through root-cause analysis.
● Implement and evolve automated data-quality controls, including schema and completeness checks, freshness and volume monitoring, null/range/duplicate checks, event consistency, stitching validation and reconciliation between processing layers and MDL.
● Create and use operational dashboards, alerts and job-level telemetry to make pipeline health, data quality, performance and processing costs visible; improve reliability and efficiency based on those signals.
● Optimize distributed processing and cloud data layouts for correctness, scalability, runtime and cost, working with technologies such as Spark, Parquet, Amazon S3, AWS Glue, Athena and EMR.
● Contribute to design reviews, code reviews, testing strategy, release readiness, runbooks and technical documentation; maintain clear data contracts, schemas and operational ownership boundaries.
● Provide technical direction and mentoring to engineers, lead moderately complex initiatives through influence, and communicate risks, dependencies, estimates and delivery status to cross-functional partners.
● Drive multiple projects or workstreams end to end, translating business needs into scope, milestones and technical plans; coordinate dependencies and stakeholders; and remain accountable for quality, adoption, timelines and measurable business outcomes—not people-management responsibilities.
● Handle sensitive data responsibly and follow Nielsen’s security, privacy, access-control and engineering practices.

Qualifications

● Typically 10–15 years of relevant data engineering or data processing experience, or equivalent depth of expertise delivering complex production data systems.
● Bachelor’s degree in Computer Science, Engineering, Information Systems or a related field, or equivalent practical experience; a relevant advanced degree or specialized certification is a plus.
● Expert-level SQL and strong programming skills in Python and/or Scala/Java, with substantial hands-on experience building distributed data-processing workloads using Apache Spark or a comparable framework.
● Proven experience designing, deploying and supporting production data pipelines and data-lake/lakehouse workflows on AWS or a comparable cloud platform.
● Practical experience with object storage and columnar formats such as S3 and Parquet, workflow orchestration, metadata/catalog services, and query engines such as Athena or equivalent.
● Strong understanding of data semantics and modeling, source-to-target lineage, schema evolution, data contracts, event-driven and batch-processing patterns, partitioning, incremental loads, backfills and idempotent/recoverable processing.
● Hands-on knowledge of real-time event processing concepts, including delivery guarantees, event-time handling, ordering, deduplication, replay, latency and backpressure.
● An AI-first engineering mindset and practical judgment about where AI/ML can improve data understanding, processing, quality or operational efficiency, with the ability to verify AI-generated outputs.
● Demonstrated ability to investigate unfamiliar or inconsistent source data, convert business rules into testable processing logic, and diagnose issues across multiple pipeline stages.
● A track record of independently driving multiple complex, cross-functional projects from problem definition through production delivery, with clear ownership of business outcomes and no reliance on formal people-management authority.
● Experience building data-quality checks, reconciliation, monitoring and alerting for production data products.
● Strong analytical and problem-solving skills, sound technical judgment, and a track record of independently owning complex deliverables from discovery through production support.
● Excellent written and verbal communication skills, including the ability to explain technical trade-offs and coordinate delivery across teams.

Additional Information

● Familiarity with stitching diverse telemetry, user activity, and digital interaction signals to derive session- or content-level metrics.
● Experience with content metadata/reference-data enrichment, entity matching, e-commerce event classification, advertising datasets or ML-assisted data processing.
● Experience with AWS services such as Kinesis, Firehose, Lambda, Step Functions, Glue, Athena and EMR, and with Grafana or Datadog for operational observability.
● Experience improving reusable onboarding frameworks, pipeline resilience, data lineage, operational cost visibility or large-scale backfill processes.

Please be aware that job-seekers may be at risk of targeting by scammers seeking personal data or money. Nielsen recruiters will only contact you through official job boards, LinkedIn, or email with a nielsen.com domain. Be cautious of any outreach claiming to be from Nielsen via other messaging platforms or personal email addresses. Always verify that email communications come from an @nielsen.com address. If you're unsure about the authenticity of a job offer or communication, please contact Nielsen directly through our official website or verified social media channels.