L

Logicbroker

SaaS platform for drop-ship automation

Lead Data Engineer

Data EngineeringUpdated on Nov 8
Full-timeNew York, NY, USANo salary listedIn-person

About the Job

Requirements

  • At least 4 years of data engineering experience, with strong hands-on experience in Spark, including PySpark and Spark SQL.
  • Proven experience implementing Medallion or a similar layered data architecture.
  • Strong data modeling skills, especially for analytics and reporting.
  • Experience leading delivery for a small engineering team; formal management experience is not required.
  • AI fluency, including daily use of AI-assisted development and understanding data needs of AI features.
  • Ability to work with partially defined requirements and refine them collaboratively.
  • Clear written and verbal communication with engineers and non-technical stakeholders.

Responsibilities

  • Turn engineering specifications into scoped tickets, sprint plans, and sequenced workstreams, and refine incomplete requirements with the Engineering Manager.
  • Assign work across full-time and contract engineers based on skill level and complexity.
  • Keep work unblocked and on track, and surface risks, dependencies, and sequencing concerns early.
  • Review pull requests and maintain a consistent quality bar across delivered work.
  • Communicate progress, blockers, and open questions clearly and proactively with stakeholders.
  • Set team norms for AI-assisted development, including what engineers verify, what needs closer review, and where AI tools help or add risk.
  • Own and directly implement the most complex transformation and modeling work in Spark and Databricks.
  • Design and implement Medallion architecture transformations from Bronze through Silver to Gold.
  • Build performant Gold-layer data models that are easy for BI and analytics tools to consume.
  • Modernize the data warehouse across multiple Databricks instances as part of a GDPR-first data strategy.
  • Improve the reliability, performance, and cost-efficiency of existing pipelines, including Spark job tuning and Databricks compute spend.
  • Make Gold-layer data well-documented and reliable enough to support AI and agentic product features.

Desired Qualifications

  • Databricks experience.
  • Experience tuning Spark performance and managing Databricks compute costs at scale.
  • Experience supporting GDPR-compliant data strategies.
  • Experience with multi-region or multi-tenant data platforms.
  • Experience with BI and analytics tools that consume warehouse data.
  • Experience with data sources across Azure, AWS, GCP, and other platforms.

About the Company

Logicbroker

SaaS platform for drop-ship automation

Logicbroker provides a SaaS platform and RESTful API that helps retailers, brands, and eCommerce platforms automate drop-ship and manage supply chain data, enabling Electronic Data Interchange (EDI) with suppliers. The product works by users accessing the platform and API to map and process supplier and order data, automate drop-ship workflows, and convert data to and from EDI, plus analytics and vendor performance reporting. It differentiates itself through scalable technology paired with strong customer service and a focus on making legacy data usable in modern digital commerce, serving enterprise clients such as Rite Aid, Wine Enthusiast, and Kroger. Its goal is to help clients expand product assortments and improve efficiency and accuracy across the supply chain to boost eCommerce performance and scalability.

Company size
51-200
Company stage
Growth Equity (Venture Capital)
Total funding
$136.6M
Headquarters
Shelton, CT
Founded
2010

Recently Posted Jobs

Explore more opportunities at Logicbroker

See more jobs