H

HUD

Live runtime data for AI coding

Research Engineer - Benchmarks

Full-Time
$100k - $230k/yr
Mid
San Francisco, CA, USA
HybridRemote candidates must have 70–80% work-hours overlap with either San Francisco or Singapore time zones.
AI & Machine LearningAI Research
H1B Sponsorship Available

About the job

Requirements
  • Proficiency in Python, Docker, and Linux environments.
  • Published papers or technical blogs on relevant topics, such as public benchmarks and their limitations or model failure modes; applicants should link these in their application.
  • Strong understanding of what makes a benchmark realistic, reliable, and useful.
  • Experience working on environments and evaluations.
  • Curiosity and ability to understand workflows in various domains.
Responsibilities
  • Own the design, implementation, and quality of HUD’s internal agent benchmarks.
  • Work with subject-matter experts to define tasks and create domain-specific benchmarks that evaluate agents on realistic workflows.
  • Build infrastructure to reliably run models and agents against benchmark tasks.
  • Develop metrics and analyses to understand benchmark difficulty, reliability, and failure modes.
  • Validate whether benchmark performance correlates with real-world evaluations, customer needs, and lab expectations.
  • Write clear documentation and benchmark reports that make results legible and credible to technical audiences.
Desired Qualifications
  • Be detail-oriented and able to spot subtle inconsistencies or edge cases in tasks.
  • Reason from first principles about task design, scoring, and failure modes.
  • Thrive in unstructured problem spaces.
  • Have early-stage startup experience and the ability to work independently in fast-paced environments.
  • Have strong communication skills for remote collaboration across time zones.

About the company

HUD builds a Runtime Code Sensor that links live production software with AI-powered coding assistants. Developers add a lightweight SDK to their apps to stream real-time information about how code behaves—such as errors, latency, performance, and how functions are used—directly to AI tools like GitHub Copilot and other code generators. The product works by feeding ongoing, context-rich data from a running system into AI models so generated code can be aware of real-world conditions rather than just static code. This makes AI-generated code safer and more reliable in production. HUD differentiates itself by enabling production-aware development, providing continuous feedback from live environments rather than relying on static analysis or old logs. The company’s goal is to help software teams build higher-quality software faster with fewer bugs and better performance by bridging the gap between AI-generated code and real-world usage.

Company Size

51-200

Company Stage

Early VC

Total Funding

$21.2M

Headquarters

New York City, New York

Founded

2025

Get referred to HUD

See people who can refer or advise you

Simplify Jobs

Simplify's Take

What believers are saying

  • HUD says over 50 businesses use its platform as of June 2026.
  • HUD reported 1.3M task runs and millions monthly in marketplace volume.
  • Thirty open roles on October 5, 2026 signal aggressive hiring across research and GTM.

What critics are saying

  • OpenAI, Browserbase, and model labs can internalize HUD’s eval stack by 2027.
  • HUD’s marketplace depends on vendor liquidity; thin supply kills pricing power fast.
  • If frontier labs standardize one benchmark suite, HUD becomes a replaceable tooling layer.

What makes HUD unique

  • HUD raised a $16M Series A led by Standard Capital on June 30, 2026.
  • HUD combines RL environment SDKs, evals, and DataVendor marketplace across computer-use tasks.
  • HUD’s benchmarks already include Autonomy-10, SheetBench-50, OSWorld-Verified, and Assemble Bench.

Help us improve and share your feedback! Did you find this helpful?

Benefits

  • Health Insurance
  • Dental Insurance
  • Vision Insurance
  • Paid Vacation
  • Paid Holidays
  • Commuter Benefits

Growth & Insights and Company News

Headcount

6 month growth

14%

1 year growth

14%

2 year growth

5%
SYMEX ECONOMICS SA
Aug 13th, 2026
ClickHouse and Hud connect the dots between AI code, production data and engineering decisions.

ClickHouse and Hud connect the dots between AI code, production data and engineering decisions. New York, USA, August 13th, 2026, FinanceWire The promise of AI-powered development has largely centered on speed. Coding agents can generate changes quickly, allowing engineering teams to increase the amount of software they produce. But faster code generation also creates a new pressure point: teams need to understand those changes quickly enough to decide whether they belong in production. Hud says AI now generates or assists with 42% of the code developers ship, with that figure expected to reach 65% by 2027. The company's new integration with ClickHouse is built around the idea that production behavior can provide some of the context needed to manage that acceleration. The integration connects ClickStack, ClickHouse's open-source observability stack, with Hud's Runtime Code Sensor. Rather than treating observability as a separate troubleshooting function, the companies are connecting it with workflows for assessing, releasing, and fixing AI-generated software. From Infrastructure Signals to Code-Level Context The distinction between the two platforms is central to the integration. ClickStack provides broad visibility across applications and infrastructure, helping teams determine the service, deployment, or endpoint associated with an issue. Hud operates at the code layer, following function-level behavior and connecting production activity to the changes behind it. Shared trace IDs allow a coding agent to connect the two systems. An engineering team investigating an issue in ClickStack can move directly toward the corresponding code-level context in Hud. "Our users already trust ClickHouse to store and query their Open Telemetry data at scale," said Mike Shi, Head of Observability of ClickHouse. "The shift now underway is from simply watching systems or investigating issues to using that data to make day-to-day engineering decisions. Hud connects observability data to the code and changes behind it, making the entire stack more useful for teams building and shipping software with AI." That approach expands the role of production data. Instead of becoming useful only after a deployment goes wrong, runtime information can help inform decisions about changes before they are released. Making the Release Process More Dynamic The integration supports pre-deployment risk assessment for code changes. Teams can evaluate changes against real-time information about how affected code behaves in production, with higher-risk changes held for additional review and deeper context while safer changes can move faster or be automatically merged. The workflow continues after deployment. ClickHouse and Hud support release verification, automatic reversion upon regression, automated detection and investigation, and agentic workflows that can create pull requests to fix underlying issues. Rollback and remediation workflows are also part of the integration. That means the same connection can support several decisions around a single change: whether it should ship, whether it is behaving as expected after deployment, and what should happen if production behavior diverges from expectations. "AI is accelerating how quickly teams can generate code, but safely shipping it with high confidence requires production context," said May Walter, CTO of Hud. "Hud and ClickHouse bring real production behavior at an unparalleled breadth and depth, so teams can build a production-aware AI SDLC: gating changes before they ship, proactively verifying them once deployed, and fixing issues as they arise - all using real runtime truth. Together with ClickHouse, we are bringing that intelligence across the entire AI SDLC." The Case for a Production-Aware AI SDLC The practical use case becomes clearer when a seemingly small change produces an unexpected production result. A query can slow down, a function can behave differently under a particular workload, or a code path can consume more resources. Hud is designed to detect these issues at the function level and provide forensic context into their causes. ClickStack contributes the wider operational view, helping teams connect the code-level behavior to what is happening across the application. "Like every modern engineering organization, a growing share of our code is now written with AI," said Rom Kadria, Senior Software Engineer, monday.com. "We write code much faster, but the challenge has shifted to shipping just as quickly while maintaining confidence that new code won't cause harm. ClickHouse gives us the wide operational picture at scale, while Hud gives us the runtime intelligence and next-level introspection needed to evaluate and ship AI-generated code confidently. When issues do arise, combining ClickHouse and Hud allows us to triage and resolve them quickly. For a company building with AI, that combination is the obvious choice." Engineering teams can get started by installing the Hud SDK and connecting it to their ClickStack service. Hud's runtime intelligence then flows alongside the OpenTelemetry data teams already collect. The broader proposition is a shift in where production information sits within software development. As coding agents generate more of the code, ClickHouse and Hud are attempting to make real-world application behavior part of the feedback loop that determines how that code is reviewed, released, monitored, and repaired. Contact. Editorial team [email protected]

Calcalist Tech
Dec 11th, 2025
Hud raises $21M for AI-era sensor offering real-time production visibility

Hud, an Israeli startup co-founded by Roee Adler, May Walter and Lemonade CEO Shai Wininger, has emerged with $21 million in early-stage funding. The company has developed what it calls the first platform offering real-time, function-level visibility into software behaviour in production environments, designed for both engineers and AI coding agents. The startup's Runtime Code Sensor installs in under one minute without configuration, recording live function-level behaviour, performance, errors and dependencies whilst correlating system issues with root causes in real time. The technology runs with minimal overhead and is already deployed across millions of services. Crucially, the data is structured for AI coding tools like Cursor, Copilot and Claude, enabling them to generate code based on actual production dynamics rather than guesswork.