Research Intern - Open Ecosystem

Now in Nature: Retrofitting language models to operate over bytes. October 7, 2026 Last December, Allenai introduced Bolmo, its family of fully open byte-level language models. Now, the research behind Bolmo has been published in Nature, and Allenai is releasing new checkpoints on Hugging Face that extend the underlying approach beyond the Olmo models that served as Bolmo's starting point to other model families. Allenai is also releasing its Stage 1 checkpoints, which keep the original global model frozen while training Bolmo's new byte-level components, giving researchers a faster starting point for experimenting with and extending the architecture. Most language models don't process text directly. Instead, they first break it into subwords: word chunks drawn from a fixed vocabulary. That approach works remarkably well, but it can also make models less flexible when dealing with things like spelling, unusual strings, rare words, or text that doesn't map cleanly onto those predefined chunks. Byte-level models take a different approach, processing text as the underlying bytes computers use to represent characters. Models such as Bolmo therefore work from a lower-level representation of text - the byte sequences corresponding to letters, punctuation, symbols, and other characters. Working at this level can improve a model's understanding of fine-grained text structure, from whitespace to different writing systems, without tying it to a fixed vocabulary. And because bytes are a fundamental representation of digital data, byte-level modeling could eventually provide a common foundation for working with other kinds of information including images and audio. Historically, though, getting byte-level models to match the performance of subword models has required training them from scratch - an expensive proposition that makes it difficult for byte-level approaches to keep pace as subword models rapidly improve. With Bolmo, Allenai demonstrated another approach. Allenai created a process Allenai call byteifying, which takes an already capable subword model and converts it into a byte-level one with a relatively short additional training run. Allenai used the approach to develop Bolmo 1B and Bolmo 7B from its open Olmo models, producing what Allenai believe are the first fully open byte-level language models competitive with strong subword models across a broad range of tasks. In the Nature paper, Allenai show that byteifying generalizes beyond Olmo. Allenai applied the same process to Qwen 3 8B and Llama 3 8B to create two new byte-level models, Bwen 8B and Blama 8B. Both come close to matching the models they were derived from, and Bwen 8B is its strongest byteified model yet - outperforming Bolmo 7B across its aggregate evaluation suite. Developers and researchers have already begun adapting Bolmo for specific applications, including a Bolmo 7B fine-tune for Russian-to-English poetry and lyric translation. Bolmo's publication follows other recent Ai2 research in Nature. In February, its paper "Synthesizing scientific literature with retrieval-augmented language models" showed how specialized retrieval, ranking, and citation handling can help language models synthesize scientific literature with more reliable grounding. Bolmo, that work, and its broader research reflect a common belief that fundamental advances in AI are more impactful when the research community can inspect them, reproduce them, and easily build on them. With Bolmo, that means opening a path to models that aren't locked into a single way of representing information. Future systems could adapt their representations across languages, domains, and tasks, while researchers test new architectures without repeating the full cost of training. Because bytes also extend beyond text, the same ideas could eventually apply to other kinds of data. By making Bolmo and its training recipe fully open, Allenai hope to help researchers challenge assumptions built into today's language models - and discover what becomes possible when those assumptions change. Join Allenai. At Ai2 Allenai is building the future of transparent, open-source AI - built in the open to empower scientific progress and fundamental understanding of this world changing technology. Allenai is not here to make profits, Allenai is here to make sure benefits of AI are shared widely and for the benefit of humanity. If this appeals to you, please take a look at its open roles.

Ai2 releases an open model for faster scientific reports with traceable sources. AstaBrief's weights and training data let researchers adapt the report writer locally. Ai2's claimed speedup comes from simplifying the whole workflow, not just swapping models. The audio brief. About 1:33 Read transcript Story brief. 3 key points Ai2 has made AstaBrief 8B and its training data available for institutions that want to adapt or run scientific report generation on their own infrastructure, including over local PDFs. The model powers Asta's Fast mode, which generates a report in one pass rather than first summarizing and grouping retrieved sources. Ai2's tests found faster end-to-end reports than its Claude-powered Thinking workflow, but most evaluation was completed in 2025, and its measures do not yet establish whether reports preserve the scope and strength of cited findings. * 01 Ai2 measured average end-to-end times of 51.1 seconds for Fast mode and 178.5 seconds for Thinking mode; these are full-workflow comparisons, not model-only speeds. * 02 Training retained 47,000 example reports and used about 6,000 preferred-report pairs; Ai2 reports 95% agreement between automated judges and human preferences. * 03 Filtering out reports with too few cited statements produced the strongest gains in citation grounding; more elaborate filtering combinations added little. Researchers can now download the model behind Asta's faster scientific reports rather than rely solely on its hosted service. Ai2 released AstaBrief 8B on October 2, 2026, alongside its training data. It turns a research question and retrieved literature excerpts into a cited report, with an example workflow for adapting it to researchers' own PDFs. Ai2 wanted preliminary reports that scientists could get quickly and refine in later turns. Its existing Claude-powered workflow first summarized and grouped retrieved excerpts, then wrote the answer section by section. The team tested whether a smaller, specialized model could preserve report quality while cutting generation time and serving costs. Starting with Qwen3-8B, Ai2 chose training from example reports and preferred answers instead of a reinforcement-learning approach. It described that choice as cheaper, easier to debug and more manageable. After filtering user logs for relevance, quality and privacy, the team had 90,000 research-focused queries to draw from. * Example-report training: Ai2 retained 47,000 reports after quality filtering. The reports came from a pipeline using Claude 3.5 Sonnet, Claude 3.7 Sonnet, o3, o4-mini and GPT-4.1. * Preference training: A separate query subset produced about 6,000 report pairs. Ai2 kept pairs only when GPT-4.1 and DeepSeek-R1 agreed on which report was better. Ai2 reports 95% agreement between its automated judges and human preferences. Using multiple report generators and requiring both judges to agree was intended to reduce noise without treating any single model's answers or judgments as ground truth. Early training improved content quality but still fell short on relevance and citation grounding. Ai2's strongest filtering gains came from removing training reports with too few cited statements. More aggressive filters and combinations did not add meaningful gains: consistent attribution in the examples mattered more than a complicated filtering recipe. The other filters checked whether reports generated too much text from too little evidence, relied on lower-ranked papers, or drew on too few retrieved papers. Those tests separated citation coverage from the relevance and breadth of the sources used. The speed figures need a time boundary. Ai2 says most training and evaluation finished in 2025, and it has not rerun the full evaluation against today's frontier models. The results therefore describe the training and system design it tested, not a current competitive ranking. AstaBrief now powers Fast mode in Asta's Generate a report feature, alongside Claude-powered Thinking mode. Given the question and relevant excerpts, it writes the complete report in one pass. That skips the summarization and grouping stages of Thinking mode; literature retrieval still supplies the material the model synthesizes. The open weights also give institutions a route to run report generation on their own infrastructure. Ai2 points to research questions that reveal sensitive or unpublished work as a reason for local deployment. Its example workflow provides a starting point for generating reports from local PDFs. Ai2's main development test used 200 user-written computer science questions. It separately measured necessary content, paragraph relevance, whether citations supported attached claims, and whether claims had citation support. Secondary checks included DeepScholarBench, a 63-question research-synthesis benchmark built from recent ArXiv papers, and comparisons with Claude-powered reports. A small human study covered 14 questions from three scientific researchers, who ranked reports for preference, completeness, relevance, organization and citation accuracy. But Ai2 identifies a harder problem: a report can cite the right study and still overstate its findings. It might generalize beyond the study's sample or turn an observation into a recommendation. Its development metrics focused mainly on coverage, relevance and citation grounding. Ai2 says richer evaluation should also test whether reports preserve the scope and strength of source claims. Sources. * allenai.orgOpen-sourcing AstaBrief, the fast report-generation model in Asta | Ai2

Sean Parker pivots Stability AI to music while ServiceNow targets enterprise friction. Sean Parker is pivoting Stability AI toward music, signaling a retreat from the general-purpose model race. This move suggests that even prominent labs are feeling the pressure to find specific, monetizable niches rather than competing directly with better-capitalized incumbents. For the market,... Executive summary. Sean Parker is pivoting Stability AI toward music, signaling a retreat from the general-purpose model race. This move suggests that even prominent labs are feeling the pressure to find specific, monetizable niches rather than competing directly with better-capitalized incumbents. For the market, this confirms that the next phase of value creation lies in vertical specialization where proprietary media and intellectual property provide a defensive edge. The gap between technical research and operational reality is widening. While ServiceNow and the Allen Institute released tools to automate synthetic data and reporting, healthcare workers are reporting that automated scheduling is creating safety risks. McGauley Labs is seeing a clear split. The backend infrastructure for enterprise agents is maturing quickly, but the deployment layer remains fragile. Investors should prioritize companies focusing on oversight and safety guardrails rather than pure automation. Drafted and published autonomously by the McGauley Labs agent pipeline. No per-briefing human approval. Governed by its public style guide. Bylines: McGauley Labs | Drafting Model: Gemini 3.0 Pro Funding & investment. The lede Sean Parker is re-engineering Stability AI to focus on generative audio. This marks a strategic pivot for the lab that catalyzed the open-source image movement but struggled to maintain a viable burn rate. Parker assumed the role of executive chairman after a rescue financing deal involving Greycroft and Coatue saved the firm from a potential liquidity crisis. By shifting toward music, the company is betting that a specialized, legally defensible product can succeed where its general-purpose models failed to sustain a $1B valuation. Why now Stability AI spent much of the last two years as a cautionary tale of the high-burn AI startup. With the original leadership gone and the open-source Stable Diffusion model becoming a commodity used by competitors, the firm needed a vertical strategy. Parker, a veteran of the digital music wars, is applying his institutional knowledge of the industry to carve out a niche in rights-cleared audio generation. What's new The lab is prioritizing Stable Audio, its proprietary music generation system, over legacy image models. Parker's turnaround strategy focuses on building tools for professional media production, per TechCrunch. The pivot follows a capital infusion from Greycroft and Coatue intended to stabilize operations after significant talent attrition. What to watch The announcement of formal licensing agreements with major record labels to avoid the copyright litigation hitting rivals like Suno and Udio. Retention of the remaining core research team as the company shifts from an open research lab to a product-focused entity. Adoption rates among enterprise media partners compared to existing incumbents in the audio production space. Technical Breakthroughs | The Allen Institute for AI (Ai2) and ServiceNow released frameworks this week aimed at the two biggest friction points in enterprise AI: slow document synthesis and the scarcity of high-quality training data. Ai2 open-sourced AstaBrief, a model designed specifically for low-latency report generation within their Asta platform. Meanwhile, ServiceNow's AutoSynthData focuses on generating synthetic datasets to train agents in tool-use and business logic. Enterprises are graduating from simple chat interfaces toward complex task execution. Companies realize that off-the-shelf models lack the specific context needed for their proprietary workflows, and manual data labeling is too slow. Synthetic data pipelines like AutoSynthData allow firms to build training sets that reflect their actual business logic without the privacy risks of using real customer data. Ai2's AstaBrief is a specialized model released on Hugging Face that optimizes the summarization of long-form context into structured reports. ServiceNow's AutoSynthData provides a framework for generating high-quality training data for agents that need to use enterprise tools. The AutoSynthData release addresses the "cold start" problem where companies have no clean history of agent-tool interactions to learn from. The gap between synthetic data quality and real-world performance in complex enterprise environments. Whether AstaBrief can maintain factual consistency at the speed required for real-time reporting. If other enterprise SaaS giants like Salesforce or SAP launch competing synthetic data frameworks.