Datadog AI Research — Scholars Program with Carnegie Mellon University
Datadog AI Research (DAIR) is partnering with Carnegie Mellon University to support a small number of PhD students working on open research problems grounded by ongoing efforts at Datadog/DAIR.
You will frame a problem, run your own experiments, and write up what you find, with compute and data at a scale most academic labs cannot provide. You will collaborate with colleagues working on the same questions.
The Lab And The Research:
DAIR is an industrial research lab motivated by practical challenges in observability and software operation: detecting and diagnosing failures, understanding complex production environments, and helping engineers operate software more effectively. The lab focuses on creating specialized foundation models, post-training and evaluating AI agents, and building frontier-scale machine learning systems. By combining fundamental research with Datadog's large-scale, real-world data and infrastructure, the lab develops new AI capabilities and translates them into practical systems with meaningful impact.
Internship projects are shaped with your DAIR mentor and your CMU faculty advisor. You do not need prior experience with observability, monitoring, or infrastructure.
What You'll Do:
Who You Are:
Not Required:
Datadog values people from all walks of life. We know not everyone will meet all the above qualifications on day one. That’s okay. If you’re passionate about technology and want to grow your experience, we encourage you to apply.
This job is available in various departments within our company; to conform to US export control regulations, some of these roles may require candidates to be eligible for any required authorizations from the US government.
If possible, please apply using your personal email address instead of your university email address.
Students can sign up for a free Datadog Pro account to learn more about our platform and products.
#LI-Hybrid
Datadog offers a competitive salary for this role and may include additional compensation elements. Actual compensation is based on factors such as the candidate's skills, qualifications, and experience. In addition, Datadog offers a wide range of best in class, comprehensive and inclusive employee benefits for this role including healthcare, paid time off, and benefits related to traveling and relocation, if eligible.
About Datadog:
Datadog is the leading observability and security platform for the AI era, providing businesses with unified visibility across the technology stack to manage complexity at scale. It brings applications, infrastructure, data, models, and security into one place, using AI to detect and resolve issues before they impact customers. Trusted globally by Fortune 500 companies and high-growth AI leaders, Datadog enables businesses to move faster with clarity and confidence. Learn more about #DatadogLife on Instagram, LinkedIn, and Datadog Learning Center.
Equal Opportunity at Datadog:
Datadog is proud to offer equal employment opportunity to everyone regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity, veteran status, and other characteristics protected by law. We also consider qualified applicants regardless of criminal histories, consistent with legal requirements. Here are our Candidate Legal Notices for your reference.
Datadog endeavors to make our Careers Page accessible to all users. If you would like to contact us regarding the accessibility of our website or need assistance completing the application process, please complete this form. This form is for accommodation requests only and cannot be used to inquire about the status of applications.
Privacy and AI Guidelines:
Any information you submit to Datadog as part of your application will be processed in accordance with Datadog’s Applicant and Candidate Privacy Notice. For information on our AI policy, please visit Interviewing at Datadog AI Guidelines.
The reasonably estimated yearly salary for this role at Datadog is: 110000 - 140000 USD
Datadog builds monitoring and security software for developers and operations teams that run cloud applications, giving them a shared view of system health and performance. Its platform links infrastructure monitoring with application performance and log analysis so teams trace a problem across the systems involved. Software agents and integrations collect operational data, which teams explore through dashboards and alerts to find failures, investigate threats and improve the customer experience. The company sells subscriptions to individual products and combined plans, with charges tied to the resources and data monitored. Datadog was founded in 2010 and is headquartered in New York City.
Company Size
5,001-10,000
Company Stage
IPO
Headquarters
New York City, New York
Founded
2010
See people who can refer or advise you
Help us improve and share your feedback! Did you find this helpful?
Bernstein has upgraded Datadog (NASDAQ:DDOG) to Outperform from Market Perform and raised its price target to $350 from $237. Datadog shares rose in premarket trading. The firm downgraded the stock last quarter, citing a roughly $100 million cut in annual recurring revenue (ARR) from Datadog's largest Born in AI customer and weakness in its traditional customer base, which makes up about 85% of revenue. It had expected net new ARR growth from the biggest AI labs to slow to 20% to 30% in 2027 and beyond, from 80% to 90% this year. That slowdown hasn't appeared. Alternative data suggest enterprise token consumption at the largest AI labs grew at least 50% from the prior quarter, and possibly faster among smaller Born in AI customers. Datadog reports third-quarter results on November 4.
Introducing Scout 1.0: A new SRE model powering Bits AI with frontier performance at a fraction of the cost. Datadog | Published 16 Oct 2026 Datadog has introduced Scout 1.0, its first AI model specialised for investigating production issues. Scout powers Bits Investigation, Datadog's AI site reliability engineer (SRE). The company trained Scout to reduce inference cost so that investigations are cheap enough to run an order of magnitude more often and earlier in an incident's lifecycle. Key highlights include: * Scout achieves a better price-performance curve on Datadog's target scenarios when compared with general-purpose frontier models. Scout 1.0 medium matches GPT-6.1 Sol high and Claude Opus 5.5 high at 3-8x lower estimated inference cost, and performs comparably to Fable medium and Astra medium at 12x and 19x lower cost, respectively. With test-time scaling, Scout 1.0 xhigh matches Fable xhigh at 4x lower cost. * Datadog built Scout using its in-house distributed reinforcement learning infrastructure, and relied extensively on real historical incidents and telemetry data to build high-quality training simulations that don't contain any customer data. For more technical details, see our full report. * Datadog engineering teams and a small group of customers in a closed preview are already using Scout to investigate production alerts. Sign up for the preview to try it in your environment and provide feedback before general availability. Datadog's goal is for Bits to serve as an always-on first responder, gathering evidence and identifying likely causes before an engineer steps in. Each Bits investigation involves forming hypotheses, querying telemetry, and deciding whether the evidence supports a conclusion. Those steps require repeated model calls. By reducing a model's inference cost, Datadog can give teams more room to run investigations across their environments. Scout's efficiency makes it practical to extend support to more alerts and investigate complex issues in greater depth, enabling greater breadth of coverage, reducing the burden of on-call work, and freeing more time for development. Lower inference cost also makes it practical to spend additional computation on harder or higher-impact problems. For instance, Scout 1.0 xhigh uses more computation per incident to improve investigation quality, and this added investigative depth reduces engineering workload and pressure during even the highest-stakes incidents. Datadog built Scout 1.0 on GLM-5.3 Flash and post-trained it using reinforcement learning in investigation simulations. It uses historical incidents and telemetry from real alerts to build realistic simulations with known causes. Scout practices using tools modelled on Bits Investigation to gather evidence and test competing explanations. Its training rewards finding the cause and supporting its conclusions with evidence. Datadog also built its own training infrastructure on top of open source tools, with the simulation, evaluation, and model-training systems needed to run reinforcement learning at scale. To test whether those skills transfer to real incidents, Datadog built AlertEval. The company's engineers have curated thousands of evaluation points through its incident-response workflow. AlertEval draws on that work, combining real alerts, archived telemetry, and root cause explanations written by engineers. Datadog is building Scout and Bits Investigation together, training the model for the tools Bits uses and improving those tools based on what we learn from its investigations. Datadog has also evaluated Scout on SREGym, a third-party benchmark, to test its performance outside AlertEval. The company again has found that Scout 1.0 medium significantly improves upon the base model, GLM-5.3-Flash, performance, surpassing GLM-5.3 at a more than 4x reduction in cost. Through additional test time scaling, Datadog found that Scout 1.0 high is competitive with the best models on this benchmark, while still maintaining at least 2x cost savings compared to all competitors (10x savings compared to Astra/Fable-class models). Moving forward, Datadog plans to simulate a wider range of failures and keep improving investigation quality while lowering inference cost. It also plans to extend this approach beyond investigation to help teams autonomously remediate issues to restore service faster. Together, these advances move Bits beyond its role as a first responder and toward becoming the only responder during an incident, resolving issues faster and reducing on-call burden.
Cognition's Devin Oncall sends an AI agent to fix production alerts. Cognition's new SRE agent plugs into Datadog, Sentry and Dynatrace to triage alerts, run investigations and open pull requests for fixes. · 4 hrs ago Read 5 min * Cognition launched Devin Oncall, an AI SRE that investigates alerts and opens fixes. * Works out of the box with Datadog, Dynatrace, Sentry and other observability tools. * Already processes thousands of alerts daily for large enterprise customers. * Internal data at Cognition shows under 2% of alerts actually need code changes. * Launch follows Cognition's $2B raise at a $48B valuation and $492M run-rate. * Competes with AWS DevOps Agent and built-in AIOps from Datadog and Dynatrace. Devin Oncall sends an AI agent to production alerts. Cognition has launched Devin Oncall, a product that turns production alerts into tasks for its coding agent. When an observability platform reports an incident, Devin opens an investigation, examines the available telemetry and code, then returns a diagnosis or creates a pull request. The product extends Devin from planned development work into incident response. Cognition wants its agent to complete the initial investigation before an engineer joins, reducing the time spent searching logs, tracing deployments, and identifying the relevant service or repository. Alerts become agent sessions. Devin Oncall sits between monitoring systems and engineering workflows. Cognition lists integrations with Datadog, Dynatrace, Sentry, and other observability platforms. Those systems continue to detect incidents and generate alerts; Oncall consumes the payloads and begins the investigation. * The observability platform sends Devin an alert and its associated context. * Devin opens a session with access to its shell, editor, browser, connected repositories, and permitted infrastructure. * The agent inspects logs, code, recent deployments, and relevant documentation to test possible causes. * Devin posts its findings or prepares a code change as a pull request. * Engineers review the evidence, approve changes, or take over when the incident requires broader judgment. The published workflow preserves existing review, CI, merge, and deployment controls when Devin proposes a patch. That boundary matters during an outage because a pull request is reversible and reviewable, while an automated production change carries a larger operational risk. The same Devin, triggered by production. Oncall uses the existing Devin platform as a new trigger. An alert can open the same kind of coding session that Devin uses for planned work, allowing the agent to move from diagnosis to implementation while retaining the investigation context. Cognition's Model Context Protocol tools expose sessions, playbooks, knowledge, schedules, integrations, and repository documentation. MCP provides a standard interface through which systems can supply tools and context to an AI agent. Oncall builds on that toolset, so runbooks, service documentation, and repository knowledge can guide the incident session. The agent's reach depends on the permissions attached to those integrations. Repository access allows code inspection and pull requests, while access to logs, cloud consoles, deployment history, or secrets creates additional security requirements. Teams will need to apply least-privilege credentials and preserve audit records for every action. Early data shows a triage-heavy workload. Cognition says Devin Oncall already processes thousands of alerts per day for large customers. Its customer profile for document AI company Reducto says the agent serves as the first responder and has reduced the frequency with which engineers receive pages. Cognition also uses the workflow internally, routing Datadog alerts into Slack for Devin to investigate. In a recent presentation, one of its engineers said fewer than 2% of alerts resulted in code changes. The company has not published the sample size, measurement period, or severity mix behind that figure. The under-2% rate indicates that pull requests represent a small part of the workload. Many investigations may end with a false alarm, an operational explanation, a configuration issue, or an escalation that requires human action. Useful evaluation therefore requires more than counting generated patches. Incident response is filling with agents. Cognition is expanding Devin across more stages of software delivery, including code review, automated triage, code scanning, scheduled work, and production support. The company says Devin now writes 89% of its internal pull requests. It has also reported more than $2 billion in funding at a $48 billion valuation, enterprise usage growth above tenfold since the start of the year, and $492 million in run-rate revenue. AWS has previewed a competing DevOps Agent that correlates metrics, logs, and recent deployments to identify likely causes and recommend mitigations. Datadog, Dynatrace, and New Relic also offer AI-assisted root-cause analysis. Cognition's differentiator is the connection to a general coding agent that can inspect a repository, edit code, run tests, and submit the resulting patch. Checks before granting pager access. Production incidents expose an agent to incomplete evidence, rapidly changing systems, and pressure to act. A useful deployment plan should define the agent's scope before alerts begin opening autonomous sessions. | Area | Questions to resolve | | Alert scope | Which services, severities, and alert types can start a Devin session? | | Permissions | Which repositories, logs, cloud resources, deployment systems, and secrets can the agent access? | | Operational context | Are runbooks, service ownership records, dependency maps, and deployment histories current? | | Change controls | Which tests, approvals, protected branches, and deployment gates apply to agent-created patches? | | Escalation | When must Devin page a human, and what evidence should accompany that escalation? | | Evaluation | How will the team measure diagnosis accuracy, escalation rate, time to useful evidence, pull-request acceptance, and reverted changes? | | Data handling | What retention, tenancy, regional processing, and model-training policies apply to logs and source code? | Cognition's announcement does not list public pricing, a complete integration matrix, or a service-level commitment. Deployment architecture, data retention, permission boundaries, and the availability of individual integrations will require confirmation during evaluation. The pager becomes a work queue. Devin Oncall makes production alerts another entry point into an agent-driven engineering workflow. Its practical value will depend on the quality of each diagnosis, the evidence attached to escalations, and the safety of the permissions granted to the agent. Observability platforms supply the signals, while Devin adds investigation and code execution above them. If the product consistently filters routine alerts and prepares credible fixes, on-call engineers can begin incidents with a tested hypothesis and a documented trail instead of an unexplained page.
SecurityHQ research finds an Accountability Gap in how enterprises govern AI agents. 45% of Cybersecurity Leaders Report AI-Related Incidents with Unclear Accountability; meanwhile, only 17% could rapidly reconstruct an agent's actions within minutes of a critical incident. October 15, 2026 13:45 ET | Source: SecurityHQ LONDON, Oct. 15, 2026 (GLOBE NEWSWIRE) - SecurityHQ, the global cybersecurity company pioneering Security Performance Engineering for stronger operational performance and resilience, today published new research, in collaboration with Datadog, the leading AI-powered observability and security platform for cloud applications, identifying a massive disconnect between how enterprises define accountability for AI agents and their ability to establish an agent's recorded actions. "The Accountability Gap: The State of AI Agent Security and Governance 2026" surveyed 204 cybersecurity leaders at midsize and large organizations. Of the respondents, 76% said their organization has formally defined accountability for AI-driven operational or security decisions. At the same time, 45% have already experienced an incident involving AI agents or automation where accountability was unclear. The challenge is becoming more urgent as agents gain greater access across enterprise environments. 51% of respondents said AI agents already operate broadly across their organizations, and 93% expect agents to access enterprise systems more frequently than humans within five years. "Most security teams can see that agents are working. Far fewer can explain what an agent did, in what sequence, and under whose authority. Those are the questions that matter when something goes wrong," said Feras Tappuni, CEO of SecurityHQ. "Defining accountability in a policy document is the easy part. It has to hold up in the hour after an incident, when the facts are incomplete, the business needs answers, and someone has to decide what to isolate and who has the authority to act." The findings show that many organizations still lack the visibility to answer those questions quickly. Only 49% said they could fully reconstruct every action an AI agent performed over the previous 30 days. Following a critical incident, 17% could reconstruct that activity within minutes. 51% would need hours, 23% would need days, and 9% would need longer or were unsure. "Enterprises are adding a new class of actor to their infrastructure, and most observability was designed around people and the services they run," added Tappuni. "Agents need the same level of instrumentation. When teams can see what an agent touched and when, in real time, questions about accountability become answerable." That visibility becomes more complicated when agents operate outside approved processes. Nearly half of respondents (49%) believe someone inside their organization may already be running an unapproved AI agent. Among those respondents, 51% said the activity spans multiple teams. "The fundamentals are identity, permissions, logging, and being able to follow an agent across systems that were not designed to be read together," said Aaron Hambleton, Senior Vice President of Product & Services at SecurityHQ. "That operational context is what allows security teams to reconstruct what happened to make smart and defensible decisions quickly. Without it, the first hours of an incident can be wasted assembling the timeline." The research also found a clear expectation that human judgment will remain part of consequential decisions involving AI agents. 99% of respondents identified at least one category in which human review should remain mandatory, led by financial approvals at 61%. About SecurityHQ SecurityHQ is a global cybersecurity company that helps organizations engineer, measure, and continuously improve the performance of their security operations. Founded in 2003, the company delivers flexible and technology-agnostic solutions through its Security Performance Engineering approach. Built around each customer's environment, the approach brings together managed detection and response, threat and adversary intelligence, exposure management, and advisory services. With 400+ analysts and engineers across six global SecOps Centers, SecurityHQ provides 24/7 human-led detection, response, and continuous improvement. Its work is focused on reducing noise, improving decision-making, and strengthening security performance over time.
Berlin-based OllyGarden has raised $4 million to reduce unnecessary software telemetry data. The funding round included participation from Datadog, Grafana Labs, and Dash0 — companies that typically charge based on data volume. The startup addresses a growing problem: AI-generated code creates significantly more telemetry logs, metrics, and traces. This "noise" undermines AI agents that analyse such data, producing unreliable results. OllyGarden's solution includes Rose, an AI agent that examines OpenTelemetry code and suggests improvements. The company claims it has helped firms reduce log volume by up to 85%. CEO Juraci Paixão Kröhling sits on the OpenTelemetry governance committee and previously worked at Red Hat and Grafana Labs. The 10-person team secured this funding following a $1.6 million pre-seed round in July 2025, bringing total capital raised to approximately $5.6 million.