R

Reflection AI

Builds AI models and deployment software

Web Crawl Engineer

Full-Time
No salary listed
Mid
San Francisco, CA, USA
In Person
Data & AnalyticsData Engineering
H1B Sponsorship Available

About the job

Requirements
  • Experience building large-scale web crawling, search indexing, content acquisition, or internet-scale data collection systems.
  • Strong understanding of crawling architectures, URL frontier management, scheduling, and distributed crawl coordination.
  • Experience with large-scale distributed systems using technologies such as Ray, Spark, Beam, Flink, or similar frameworks.
  • Familiarity with content extraction, HTML parsing, browser automation, rendering systems, and modern web technologies.
  • Experience operating systems that process petabyte-scale datasets.
  • Strong systems engineering skills, including reliability, observability, performance optimization, and debugging.
  • Experience designing experiments and using data to improve crawl quality, coverage, and efficiency.
  • Ability to collaborate closely with researchers, infrastructure engineers, and data quality teams.
Responsibilities
  • Build and operate web-scale crawling infrastructure capable of continuously collecting data across billions of URLs.
  • Design and optimize URL discovery, prioritization, scheduling, and crawl orchestration systems.
  • Develop distributed crawlers that efficiently acquire content while respecting site constraints and operational requirements.
  • Build systems for content extraction, rendering, parsing, and normalization across diverse web formats.
  • Improve crawl coverage, freshness, efficiency, and quality through measurement and experimentation.
  • Design infrastructure for large-scale recrawling, change detection, and incremental updates.
  • Develop specialized crawlers for high-value domains, dynamic websites, and difficult-to-access content sources.
  • Analyze crawl performance and web coverage to identify gaps, inefficiencies, and opportunities for improvement.
  • Build observability, monitoring, and reliability systems for large-scale crawl operations.
  • Debug production issues and continuously improve the performance, scalability, and resilience of crawling infrastructure.
Desired Qualifications
  • Experience building search engines, web indexes, or internet-scale crawling platforms.
  • Familiarity with anti-bot systems, dynamic web content, browser automation, and large-scale extraction pipelines.
  • Understanding of how web data is used in training and evaluating large language models.
  • Experience with distributed storage systems, content deduplication, and web-scale dataset management.

About the company

Reflection AI develops language models for software developers, companies and governments that want control over how AI is deployed and adapted. Its research concentrates on coding, reasoning and AI agents that work through tasks with software tools. The lab builds the surrounding software and infrastructure as well as models, letting customers run AI through its hosted platform or in their own computing environments. Its commercial work pairs model access with engineers who design applications and support customized enterprise deployments from development through operation. Reflection AI was founded in 2024 and is headquartered in New York City.

Company Size

201-500

Company Stage

Late Stage VC

Total Funding

$4.6B

Headquarters

New York City, New York

Founded

2024

Get referred to Reflection AI

See people who can refer or advise you

Simplify Jobs

Simplify's Take

What believers are saying

  • Federal research gives Reflection a foothold: it says its models support the Department of Energy's Genesis Mission.
  • Developers reuse existing integrations: Reflection's API supports OpenAI Chat Completions and Models through familiar SDKs.
  • Parents receive substantial leave: Reflection lists 22 paid weeks for birthing, non-birthing and adoptive parents.

What critics are saying

  • Beam's rollout starts gated: its October 2026 announcement limits early access to selected users.
  • Expect work beyond fixed scope: Reflection's culture asks staff to step in regardless of role.
  • Open releases weaken enforcement: Reflection excludes removable deployment safeguards from its assessment of models escaping operator control.

What makes Reflection AI unique

  • Enterprise deployment has a hardware route: Reflection is validated on Dell AI Factory with NVIDIA.
  • Beam emphasizes cheaper reasoning: Reflection estimates 3 to 4 times less inference compute than GLM 5.2 on advanced benchmarks.
  • Release policy favors wider access: Reflection's safety framework prioritizes reducing concentrated control when choosing how to share models.

Help us improve and share your feedback! Did you find this helpful?

Benefits

  • Company Equity

Growth & Insights and Company News

Headcount

6 month growth

6%

1 year growth

3%

2 year growth

0%
Fox Business
Oct 16th, 2026
Reflection AI CEO on risks of AI and the launch of open-weight model Beam

Reflection AI chief executive officer Misha Laskin discussed the company's release of "Beam," an open-weight AI model, on "The Claman Countdown." He also addressed the startup's valuation and reports that Nvidia is expanding its stake in it. Laskin said the risks of artificial intelligence "certainly" worry him.

REAL HACKER NEWS
Oct 16th, 2026
Techmeme: Reflection AI releases Apache 2.0-licensed weights for Beam, a 501B-parameter model, trained with RL that penalized unnecessary tokens (Matt Marshall/VentureBeat).

Techmeme: Reflection AI releases Apache 2.0-licensed weights for Beam, a 501B-parameter model, trained with RL that penalized unnecessary tokens (Matt Marshall/VentureBeat). Featured podcasts. Big Technology Podcast: Anthropic IPO Panic, A Spy At OpenAI?, The Great Food Slop Debate The Big Technology Podcast takes you behind the scenes in the tech world featuring interviews with plugged-in insiders and outside agitators. Great Chat: What happened to tech's backbone? A podcast mostly about tech. Brought to you weekly by Angela Du, Sally Shin, Mac Bohannon, Helen Min, and Ashley Mayer. The Upstarts Podcast: The $7B Startup Mapping Your Health In Just One Hour | Neko's Hjalmar Nilsonne Veteran tech reporter Alex Konrad sits down with breakout entrepreneurs taking on the status quo to shake up their fields in AI, design, nuclear energy, space, and more. Cheeky Pint: The company building underwater drones for $50,000 Stripe cofounder John Collison interviews founders, builders, and leaders over a pint. Jensen Huang | Does AI Revenue Justify Capex, '27 AI Bottlenecks, Singularity, Open vs Closed & more Open-source podcast on all things tech, markets, investing, and capitalism, hosted by Brad Gerstner. Channels with Peter Kafka: How to Run a Political Campaign in the TikTok Era (And How Not To) Media and tech aren't just intersecting - they're fully intertwined. To understand how those worlds work, Peter Kafka talks to industry leaders, upstarts and observers. Add your podcast here

AgentLensHQ
Oct 16th, 2026
Reflection Beam on Ollama: 501b-parameter open-weight MoE model, free for a limited time.

Reflection Beam on Ollama: 501b-parameter open-weight MoE model, free for a limited time. October 16, 2026 · Originally published October 15, 2026 · deepseek/deepseek-v4.1-flash Reflection released Beam, its first open-weight model, on October 16, 2026, and Ollama shipped it as a day-zero partner on Ollama Cloud - free for a limited time. The release matters because Beam is an Apache 2.0 mixture-of-experts model with roughly 501 billion total parameters, 23 billion active parameters per token, and a 1 million token context window, aimed squarely at software engineering and long-horizon agentic workloads rather than single-turn chat. What was announced. Beam is Reflection's first open-weight model and became generally available on the morning of October 16, 2026. Ollama is a day-zero launch partner, making Beam available through Ollama Cloud at no cost for a limited promotional period. Ollama's Beam endpoint runs with full zero data retention (ZDR) in collaboration with Reflection. Ollama Cloud lets developers run supported models from their apps or terminal without downloading models or managing local inference. Model architecture and specifications. Beam is a mixture-of-experts model with approximately 501 billion total parameters and 23 billion active parameters per token. The gap between total and active parameters is the defining efficiency characteristic of a sparse MoE design: only a fraction of the network is engaged per token. | Property | Value | | Total parameters | ~501 billion | | Active parameters per token | 23 billion | | Architecture | Mixture-of-experts | | Context window | 1 million tokens | | Input/output modality | Text-in, text-out | | License | Apache 2.0 | Beam's 1 million token context window is explicitly positioned to support long-running workflows. Its text-in, text-out design focuses on workloads where code, tool calls, logs, documents, and structured outputs are central. What Beam is designed to do. Beam targets workflows that involve more than a single response - writing and maintaining code, using tools across multiple steps, and working through complex tasks over time. Ollama lists four primary workload categories: * Coding agents: solving software engineering tasks, working through codebases, and helping write and maintain code. * Long-running agents: executing multi-step workflows, using tools, and working toward a goal over an extended sequence of actions. * Math and science reasoning: working through complex mathematical and scientific problems. * Custom models: fine-tuning and customizing an open-weight model for specific applications. Benchmarks. Ollama's announcement includes four benchmark charts covering agentic coding, reasoning, tool calling, and general performance. The blog post publishes these as images and does not state numeric scores in text, so no specific benchmark figures are available from this source. How to run beam. Beam is accessed through Ollama Cloud using the beam:cloud tag. Ollama directs users to download the latest version of Ollama first. Run Beam directly: ollama run beam:cloud Launch it inside coding agents: ollama launch claude -model beam:cloud ollama launch codex -model beam:cloud ollama launch opencode -model beam:cloud Why the Apache 2.0 license matters. Beam is released under Apache 2.0, which gives developers the freedom to inspect, customize, and build on the model. Combined with the open-weight release, this permits fine-tuning for specific applications - a use case Ollama explicitly lists among Beam's intended workloads. Key takeaways. * Reflection's Beam is the company's first open-weight model, generally available as of October 16, 2026. * Beam is a mixture-of-experts model with ~501B total parameters and 23B active parameters per token. * It offers a 1 million token context window and is text-in, text-out. * It is released under Apache 2.0. * Ollama is a day-zero partner offering Beam free for a limited time via Ollama Cloud, with full zero data retention. * The model is positioned for coding agents, long-running agents, math and science reasoning, and fine-tuning into custom models. Further information.

Dell
Oct 16th, 2026
Build, reason and act with Reflection Beam on Dell AI Factory.

Build, reason and act with Reflection Beam on Dell AI Factory. Reflection and Dell AI Factory bring open-weight reasoning, coding and agentic capabilities to your own data center. Key takeaways: * Reflection Beam is an open-weight reasoning model available today through Dell Enterprise Hub, with optimized containers and deployment configurations for Dell PowerEdge infrastructure. * This validated solution brings together Reflection's new Beam model with GPU compute, AI-native storage and Dell AI Data Platform so organizations can connect enterprise data to AI workflows while keeping control of their infrastructure. * This is just the first step in the collaboration bringing Reflection's frontier-grade open-weight AI together with Dell AI Factory to help organizations run governed, advanced AI tasks on premises. Reflection and Dell Technologies are working together to bring open intelligence to enterprise infrastructure. Beam, Reflection's first open-weight model, combines advanced reasoning, software engineering and agentic capabilities with Dell AI Factory so customers can run it in their own data centers, close to their data. Over the past several months, its teams have worked across model architecture, inference serving, infrastructure validation and enterprise deployment. Reflection contributes the model, inference and agent software, while Dell AI Factory provides the infrastructure, data and automation foundation needed to operate those capabilities as a system. Beam is the first model Dell Inc. is bringing to customers through this collaboration. With approximately 501 billion total parameters, 23 billion active parameters per token and a context window of up to 1 million tokens, Beam is designed for advanced reasoning, software engineering, tool use and long-horizon agentic tasks. Beam is available today through Dell Enterprise Hub, with a container and deployment configurations optimized for Dell PowerEdge infrastructure. Dell PowerEdge XE9780 systems with NVIDIA B300 GPUs are the recommended target for new production deployments. Beam can also run on supported existing PowerEdge systems with NVIDIA Hopper GPUs, giving customers flexibility as they plan for Blackwell and future generations. Bringing frontier-class intelligence on premises requires more than a model. Dell AI Factory pairs Reflection models with Dell GPU compute, AI-native storage and Dell AI Data Platform, helping customers connect their enterprise data to AI workflows while maintaining control of their infrastructure and information. Beam is the model layer of a broader enterprise AI system Dell Inc. is building with Reflection. The planned architecture brings Beam together with Mirror, Reflection's agent harness currently in early beta, and Dell AI Data Platform. Within Dell AI Factory, Dell Enterprise Hub and Dell Automation Platform simplify model access and deployment, while Dell AI Data Platform connects agents with enterprise data. The architecture also establishes a path for policy, guardrails and auditability, so agents can reason over enterprise information and act through approved tools. What Beam on Dell AI Factory means for customers. * Run on premises today: Customers can run Beam close to their data through Dell Enterprise Hub and Dell AI Factory. * An end-to-end Dell deployment path:Dell Enterprise Hub simplifies model discovery and access, while a validated Dell Automation Platform blueprint reduces integration work and helps organizations move from evaluation to production. * Connect data via a complete Dell foundation: GPU compute, AI-native storage and Dell AI Data Platform connect the model with enterprise data and applications. Beam model at a glance. * Architecture and scale: Approximately 501 billion total parameters with 23 billion active per token, using a sparse Mixture-of-Experts architecture and a context window of up to 1 million tokens. * Designed for software and agents: Beam focuses on reasoning, coding, tool calls, structured outputs and long-horizon agentic tasks through a text-in, text-out interface. * Token-efficient reasoning: Reflection trained Beam with a controllable length penalty that rewards successful solutions while discouraging unnecessary tokens. Users can adjust reasoning effort to favor shorter responses or allocate more reasoning to demanding tasks. For on-premises deployments, shorter successful traces can reduce latency, release KV-cache capacity sooner and increase useful work per GPU-hour. * Safety and alignment: Reflection trained a separate safety and alignment model using supervised fine-tuning and reinforcement learning, then combined it with Beam's capability-focused model through multi-teacher on-policy distillation. Deliberative alignment and adversarial training address harmful compliance, excessive refusals, hallucinations and unsafe tool use. Reflection also plans to publish its safety results and open-source the safety evaluations it developed. * Open model ecosystem: Beam is released under the Apache 2.0 license and integrates with familiar tools and runtimes including Hugging Face Transformers, SGLang and vLLM. Operationalizing Beam across Dell AI Factory. Beam combines strong model performance with an aggressively sparse MoE architecture, efficient reasoning capabilities and support for context lengths of up to 1 million tokens. Together, these characteristics make it a highly efficient choice for long-running coding workloads and complicated agentic applications. * Deployment - Dell Enterprise Hub provides model access, optimized containers, configuration guidance and Goodput-based sizing. * Model serving - Beam runs on Dell PowerEdge infrastructure using NVIDIA Dynamo for distributed inference orchestration and SGLang for GPU execution, with vLLM available as another runtime path. Dell Enterprise Hub provides configuration and Goodput guidance for aligning infrastructure with workload requirements. * Agents - Mirror, Reflection's agent harness currently in early beta, coordinates agent workflows and tool access. The planned enterprise layer adds policy, identity, guardrails and auditability around how agents use models, tools and data. * Enterprise data - Dell AI-native storage supplies data at AI scale, while Dell AI Data Platform helps customers ingest, process, search and activate enterprise information. Together, these capabilities make Beam part of a governed enterprise AI system rather than a standalone endpoint. Building the open AI ecosystem together. Beam is the first model in a planned Reflection family and the first result of its broader collaboration. Reflection contributes its models and open intelligence software, while Dell AI Factory provides the infrastructure, data and deployment foundation customers need to operate those capabilities with the required performance and security. Additional model and platform capabilities will follow. Start building with Reflection Beam on Dell AI Factory. Reflection Beam expands the choices available to organizations building coding assistants, research tools and agentic systems. It combines an open-weight, text-first model with a deployment path through Dell Enterprise Hub and Dell AI Factory. I'm excited to see what developers, researchers and enterprises build with it. Explore Reflection Beam on Dell Enterprise Hub and learn more about how Dell AI Factory can help turn AI innovation into business outcomes. Ihab Tarazi is the Chief Technology Officer and Senior Vice President at Dell Technologies, Infrastructure Solutions Group Core where he is responsible for technology strategy and next generation architecture for Compute, Storage and Network platforms including Edge, 5G and AI/HPC, as well as engagements with technology vendors and cloud ecosystems. Prior to this role, Mr. Tarazi served as Chief Technology Officer at Packet Host, Inc. (currently Equinix Metal), a leading bare metal cloud company built for developers, where he was responsible for the company's Product, Engineering and Technology. Prior to joining Packet, Mr. Tarazi held a number of positions in the telecom and infrastructure space, including Chief Technology Officer at data center and colocation provider Equinix, where he was responsible for developing the Equinix Cloud Exchange and strategic engagements with leading platforms in the Cloud, Security, Storage, Edge and IoT. Mr. Tarazi previously served as VP Engineering and Technology at Verizon, where he led a global team for the Engineering and Product Development of Global Enterprise Services including MPLS VPN, IP, Ethernet, Security and Managed Services. He has served as a member of the board of directors for NeoPhotonics Corp. since October 2015. He is also a member of the board for Telecom Infrastructure Project (TIP). Mr. Tarazi holds a B.S. degree in Electrical Engineering and Telecommunications from the University of Maryland, and an M.S. in Telecommunications Management from Southern Methodist University.

TechStartups
Oct 12th, 2026
Nvidia in talks to acquire open-source AI startup Reflection AI to challenge China's DeepSeek.

Nvidia in talks to acquire open-source AI startup Reflection AI to challenge China's DeepSeek. Daniel Levi Posted On October 12, 2026 Nvidia is in early talks to acquire Reflection AI, a U.S. artificial intelligence startup valued at $25 billion, as the chip giant looks to strengthen America's position against Chinese AI rivals such as DeepSeek. The potential deal could give Nvidia something it has yet to build on its own: a leading U.S. developer of advanced open-weight AI models. The discussions could result in a full acquisition, a larger investment, or an arrangement that gives Nvidia access to Reflection's researchers and technology, the Financial Times said on Saturday. "Nvidia is in talks to acquire or deepen its investment in US start-up Reflection AI, a developer of open-weight models that President Donald Trump's administration hopes will rival cheap Chinese alternatives," the Financial Times reported, citing people with direct knowledge of the matter. Nvidia has already invested $800 million in Reflection AI, making it one of the startup's largest financial backers. The company was last valued at $25 billion, though no proposed acquisition price has been disclosed. The talks remain preliminary, and no agreement is guaranteed. Nvidia and Reflection AI declined to comment on the FT report. One possibility is an *acqui-hire*, a deal in which Nvidia hires key employees and licenses a company's technology instead of buying the business outright. Such arrangements have become increasingly attractive to technology giants seeking access to AI talent without the lengthy regulatory reviews associated with traditional acquisitions. Nvidia used a similar structure in its $20 billion agreement with AI chip startup Groq in December. The interest in Reflection comes as Nvidia moves beyond selling the chips that train and run AI models. The company already develops its own Nemotron family of open-weight models, but a deal with Reflection could bring it closer to competing directly with leading AI developers. Why Reflection AI matters in the U.S.-China AI race. Founded in 2024 by former Google DeepMind researchers Misha Laskin and Ioannis Antonoglou, Reflection AI has emerged as a closely watched American challenger to China's growing collection of inexpensive, high-performing AI models. The two founders previously worked on Google's Gemini AI research, giving Reflection deep technical experience in developing advanced AI systems. The startup recently introduced Beam, its first publicly available open-weight AI model, designed to compete with Chinese offerings in coding and AI agent tasks. Reflection claims Beam delivers performance comparable to GLM-5.2 from Chinese AI developer Z.ai. Those performance claims have not been independently established across all applications. Unlike closed models such as OpenAI's most advanced proprietary systems, open-weight models let developers download model parameters, customize deployments, and run the software on their own infrastructure, subject to licensing restrictions. That flexibility can reduce operating costs and give businesses greater control over sensitive data. China's DeepSeek helped accelerate interest in this approach by demonstrating that capable AI models could be developed and deployed at costs that challenged assumptions about the resources needed to compete. For Nvidia, the strategic appeal is clear. More capable open models could encourage businesses to run AI on their own infrastructure, creating additional demand for Nvidia's chips. Reflection's importance extends beyond commercial AI. The startup has established relationships with the Pentagon and the U.S. Department of Energy and entered agreements to develop AI models for American allies, including South Korea. The Trump administration has expressed interest in strengthening domestic alternatives to Chinese open-weight models, making Reflection a potentially significant player in Washington's broader AI ambitions. TechStartups previously covered Reflection AI in March, when reports emerged that the Nvidia-backed startup was seeking $2.5 billion in funding at a $25 billion valuation. The company counts Sequoia Capital, Disruptive, and 1789 Capital, where Donald Trump Jr. is a partner, among its investors. Nvidia's interest comes as the company continues investing heavily in AI developers, cloud infrastructure providers, and businesses that depend on its chips. An acquisition would represent a significant shift for the chipmaker. Rather than supplying the hardware behind the AI industry, Nvidia would gain greater control over the models running on that hardware. For Reflection, a deal could provide access to the computing resources and capital needed to compete with much larger AI companies. The bigger question is whether Nvidia wants to remain the industry's dominant chip supplier or become a major competitor in developing the AI models those chips run.