C

CoStar Group

Commercial real estate data, analytics, marketplaces

Senior Data Engineer

Data EngineeringConfirmed live in the last 24 hours
Full-timeRichmond, VA, USA

About the Job

Requirements

  • A bachelor's degree from an accredited, not-for-profit, in-person college or university is required.
  • A track record of commitment to prior employers is required.
  • At least five years of data engineering experience, including setting technical direction on projects and influencing design across a team, is required.
  • Hands-on experience building batch and incremental ingestion pipelines from multiple heterogeneous external sources, including REST/GraphQL APIs, flat-file and SFTP feeds, third-party data vendors, and database replication/change data capture, is required.
  • Experience normalizing inconsistent third-party schemas into standardized enterprise data models, including source-to-target mapping, deduplication, and entity resolution, is required.
  • Advanced SQL and strong data modeling skills, including dimensional/star schema, normalized, or Data Vault modeling, are required.
  • Proficiency in Python, Scala, Java, or C#/.NET is required.
  • Production experience with a distributed processing engine and a cloud data warehouse or lakehouse platform, such as Databricks/Spark, Snowflake, BigQuery, Redshift, or Synapse/Fabric, is required.
  • Experience with pipeline orchestration tooling, such as Airflow, Dagster, Databricks Workflows, or Azure Data Factory, is required.
  • Experience implementing data quality, validation, and reconciliation controls for externally sourced data, with version control and continuous integration/continuous delivery in AWS, Azure, or GCP, is required.
  • Experience with agentic engineering using AI-assisted development tools such as Claude Code is required.

Responsibilities

  • Design, build, and maintain scalable data pipelines and data platforms.
  • Develop and optimize ETL/ELT processes for large, complex datasets.
  • Collaborate with engineering, product, DevOps, and database administration teams to deliver production systems.
  • Contribute to technical design decisions and mentor other engineers on the team.
  • Build a data migration and transformation process for external systems for use in the enterprise contracting system.
  • Create repeatable, customizable, and recurring applications to process data from multiple sources of varying formats, frequencies, and types, simplifying financial data conversion across systems and acquisitions.
  • Help standardize the data environment across billing systems to improve visibility into business patterns, errors, and optimizations.

Desired Qualifications

  • Production-scale expertise with Databricks, including Delta Lake, Unity Catalog, Delta Live Tables/declarative pipelines, and Databricks SQL and Workflows, or an equivalent lakehouse platform, including open table formats such as Delta Lake, Apache Iceberg, or Apache Hudi.
  • Experience with a modular transformation framework and tested, version-controlled models, such as dbt, SQLMesh, or equivalent.
  • Experience with managed ingestion and replication tooling such as Fivetran, Airbyte, Qlik Replicate, AWS DMS, Debezium, or Kafka Connect, and streaming or near-real-time ingestion using Kafka, Kinesis, Spark Structured Streaming, or Flink.
  • Experience with data quality and observability frameworks such as Great Expectations, Soda, dbt tests, or Monte Carlo, and data cataloging, lineage, and governance tools such as Unity Catalog, AWS Glue Data Catalog, Collibra, or Alation.
  • Experience with master data management, entity resolution, or record-linkage tooling, including address and geospatial standardization.
  • Experience with AWS data services such as S3, Glue, EMR, Lambda, Step Functions, or Redshift; Terraform; and backend data services in C#/.NET.
  • Experience with AI/ML and feature-engineering pipelines and applying AI-assisted or agentic workflows for code generation, review, and root-cause analysis.

About the Company

CoStar Group

Commercial real estate data, analytics, marketplaces

CoStar Group provides information, analytics, and online marketplaces for the commercial real estate industry. Its core products are detailed property data, market analyses, and tools to compare and evaluate properties, delivered via subscription access. It also runs online marketplaces like Apartments.com and LoopNet that connect renters with apartments and buyers/tenants with commercial properties, earning revenue from subscriptions, advertising, and transaction fees. The company differentiates itself through a large, integrated data platform and a suite of marketplaces that cover both residential rentals and commercial properties, with a strong presence in the U.S. and expanding globally. Its goal is to help brokers, property owners, investors, and lenders make informed real estate decisions and efficiently connect buyers, renters, and property listings across markets.

Company size
5,001-10,000
Company stage
IPO
Headquarters
Washington DC, District of Columbia
Founded
1987

Recently Posted Jobs

Explore more opportunities at CoStar Group

See more jobs