A

Alt.xyz

Platform for trading card investment services

Data Engineer - Ingestion Platform

Data EngineeringConfirmed live in the last 24 hours
Full-time

About the Job

Requirements

  • Have 3-4 years of experience in data engineering or a closely adjacent role.
  • Have modern web scraping experience at scale, using Playwright or comparable tools, and be able to impersonate a real browser at the network layer using TLS/request fingerprinting when a plain HTTP client is flagged; Selenium alone is insufficient.
  • Have strong Python skills with at least 3 years of hands-on experience and solid SQL skills for analysis and transformation.
  • Have owned an end-to-end pipeline within the past two years.
  • Use judgment about when to generalize infrastructure and when source-specific custom handling is needed.
  • Have hands-on orchestration experience with Airflow, Dagster, or a comparable DAG system, including building on it, being paged for it, and maintaining it after failures.
  • Have cloud development experience with AWS or GCP.
  • Have experience with LLM-based extraction and processing at scale, such as LLM-as-a-Judge or agent loops, beyond UI-based Q&A and simple skill building.
  • Have startup experience and be able to work at startup pace amid changing roadmaps.
  • Deliver value incrementally rather than pursuing a perfect rebuild.

Responsibilities

  • Design, optimize, and own end-to-end pipelines that scrape, process, and ingest transaction and listing data from major auction houses and marketplaces, from scraper through orchestrator to warehouse.
  • Build reusable ingestion infrastructure for authentication, retries, session and proxy management, and monitoring, while implementing custom handling where a source's structure requires it.
  • Build monitoring and alerting for latency, uptime, and coverage across every source, so missing source data is detected promptly.
  • Modernize storage and processing, reduce manual intervention, and optimize cost, performance, and reliability.
  • Partner with pricing, machine learning, product, and analytics teams to deliver clean, standardized data.
  • Build agents that can propose parser fixes for changed website markup, test them against fixtures, and open pull requests.
  • Build an orchestration triage loop that cross-references failed DAGs with logs and warehouse state, diagnoses causes, and ranks issues.
  • Use LLM extraction for low-volume sources where custom scrapers are not cost-effective.
  • Build an MCP server over the warehouse so users can ask coverage questions in natural language.
  • Use Claude Code or Cursor to work through migrations and backfills.

Desired Qualifications

  • Have experience processing large-scale dataframes with Pandas, Polars, or PySpark.
  • Collect, or have informed opinions about, the collectibles market.

About the Company

Alt.xyz

Platform for trading card investment services

Alt is a platform for researching, buying, selling, and borrowing trading cards, offering real-time pricing data, portfolio tracking, and a secure vault for asset storage. The platform leverages technologies such as instant card valuation and market trend analysis to enable users to maximize their card sales and make informed investment decisions.

Company size
51-200
Company stage
Series B
Total funding
$404.4M
Headquarters
San Francisco, California
Founded
2020

Recently Posted Jobs

Explore more opportunities at Alt.xyz

See more jobs