Senior Manager

Eclipse PanEval: advancing AI evaluation standards in Europe and beyond. by Michael Berns Thursday, March 26, 2026 - 22:12 * home * blogs * Michael Berns's blog * Eclipse PanEval: advancing AI evaluation standards in Europe and beyond. With the EU AI Act set to come into force in August 2026, Europe is entering a new phase in how artificial intelligence is governed and evaluated. Transparency and rigorous benchmarking of General-Purpose AI (GPAI) models are becoming not just best practices, but regulatory requirements. In response, the Eclipse Foundation is proud to announce Eclipse PanEval, an open source initiative designed to support transparent, standardised AI evaluation. A European initiative in a global landscape. PanEval emerges within a growing global ecosystem of AI evaluation efforts. It builds on the FlagEval framework, originally initiated by the Beijing Academy of Artificial Intelligence (BAAI), while establishing an independent European implementation tailored to regional requirements. Rather than replicating existing systems directly, PanEval adapts this foundation to address Europe's specific needs around governance, regulatory compliance, and linguistic diversity. This includes alignment with the EU AI Act, as well as a strong emphasis on data autonomy and transparency. This approach enables PanEval to contribute to the broader evolution of AI evaluation practices while remaining fully grounded in European institutional and regulatory contexts. From regulation to implementation. The EU AI Act introduces a risk-based framework for artificial intelligence, banning certain uses, imposing strict requirements on high-risk systems, and establishing transparency mandates for GPAI models to protect safety and fundamental rights. These requirements create a clear need for objective, standardised evaluation mechanisms. An AI evaluation framework functions much like an objective 'exam' for these models, using defined datasets and metrics to assess performance, safety, robustness and bias. PanEval provides this framework as an open source, vendor-neutral resource, enabling organisations to demonstrate transparency and support compliance with emerging regulatory expectations. By translating policy requirements into practical technical workflows, PanEval helps bridge the gap between regulation and implementation. Technical foundation: independent and scalable. PanEval builds upon the advanced capabilities of the FlagEval codebase to offer a robust "evaluation-as-a-service" platform capable of handling complex benchmarking workloads across large-scale AI models, including Large Language Models (LLMs). While it builds on concepts and architecture established in FlagEval, PanEval maintains an independent codebase and development path. This architectural de-coupling serves as a deliberate "de-risking" strategy, allowing the European codebase to adapt specifically to local requirements and regional standards without being tied to a 1:1 identical core. The platform supports multi-dimensional evaluation, moving beyond simple accuracy to include metrics on safety, bias, and robustness. Its architecture is optimized for high-performance and high-concurrency workloads, enabling scalable and repeatable evaluation across diverse use cases. For developers and deployers operating in Europe, this provides a transparent and reproducible path to verify that models align with the technical documentation standards required by upcoming regulations. Neutrality and governance by design. PanEval is hosted under the Eclipse Foundation, ensuring vendor-neutral governance and open, meritocratic participation. This structure is critical for building trust in AI evaluation infrastructure. All project technical assets and trademarks are managed within a transparent, community-driven process, enabling contributions from industry, academia, and independent developers on equal footing. No single organization controls the direction of the project, reinforcing its role as a shared resource for the broader ecosystem. This model aligns closely with the expectations of regulators and stakeholders seeking trustworthy, independent mechanisms for evaluating AI systems. The road ahead: Beijing and Paris. PanEval is launching with a clear focus on community development, technical maturity, and alignment with emerging regulatory needs. Further details on the project's roadmap and technical direction will be shared at upcoming industry events, including the Global Open Source AI Innovation Forum on March 27, 2026 and GOSIM in Paris this May. As AI systems continue to evolve, the need for open, transparent, and globally relevant evaluation standards will only grow. PanEval represents a step toward meeting that need, anchored in European values, while contributing to a broader international dialogue on responsible AI. The Eclipse Foundation invites developers, researchers, and organisations to participate in shaping PanEval and advancing open approaches to AI evaluation.

Eclipse Foundation unveils full agenda for OCX 2026. Europe's cross-industry open source event expands with five high-impact collocated communities across AI, automotive, compliance, research, and tooling. BRUSSELS, Feb. 26, 2026 (GLOBE NEWSWIRE) - The Eclipse Foundation, one of the world's largest open source software organisations, has announced the full agenda for Open Community Experience (OCX 2026), taking place 21-23 April 2026 at The EGG conference centre in Brussels, Belgium. OCX is the Foundation's flagship conference and one of Europe's leading gatherings for open source professionals, featuring nearly 150 sessions across six thematic tracks and five collocated communities. Over three dynamic days, developers, architects, researchers, and industry leaders will dive into the technologies, standards, and collaboration models shaping the next era of open innovation. Following a successful debut in 2024, OCX 2026 returns with a significantly expanded technical program, growing from three to five collocated communities and increasing session volume by more than 35 percent. Each community offers deep domain expertise while connecting attendees to the broader open source ecosystem. "Like the Eclipse Foundation itself, OCX 2026 continues to grow in scope, diversity, and depth," said Mike Milinkovich, executive director of the Eclipse Foundation. "As AI reshapes software development, software-defined vehicles transform mobility, and new regulations redefine accountability, open collaboration has never been more essential. This year's agenda reflects the real challenges and real opportunities facing our community." What developers will find at OCX 2026 OCX 2026 delivers three days of practical engineering insight, strategic perspective, and meaningful community-driven collaboration. Main OCX program: Cross-domain innovation in action The core program brings together leading practitioners to address software security, enterprise Java evolution, embedded and edge computing, cloud-native architectures, governance, and cross-industry innovation. Sessions are grounded in real-world implementation and built for professionals shipping and scaling software. Attendees leave with actionable techniques, architectural insight, and proven approaches they can apply immediately. Five collocated events within a single conference experience OCX 2026 features five collocated events within the conference, each offering a concentrated experience for specific technical communities while remaining fully accessible with a single pass. * Open Community for Tooling Formerly EclipseCon, this event continues the legacy of one of the industry's most respected developer tooling communities. Topics include AI-assisted IDEs, modeling frameworks, language servers, next-generation workflows, and productivity engineering. If you build tools or depend on them to deliver high-performance software, this community is essential. Speaker Highlights: * * Programming in Every Language: Building Cultural Tools with Langium - Malik Lanlokun * AI in Action: The Ultimate Live Demo with Theia AI - Jonas Helming * Open Community for Automotive Open collaboration is accelerating the shift to software-defined vehicles. This event explores open standards, safety-conscious architectures, and real-world implementations. Engineers and platform architects will gain practical insight into the software foundations powering connected mobility. Speaker Highlights: * * Diagnostics Reimagined: How Eclipse OpenSOVD Powers Open Collaboration and Standard Evolution - Thilo Schmitt & Alexander Mohr * Fifty Shades of SDV: A Blueprint-Driven Roadmap for Orchestration Adoption - Naci Dai & Oliver Kral * Open Community for AI Open source is redefining how AI systems are built, validated, and governed. This event examines trustworthy AI frameworks, open model ecosystems, responsible governance strategies, data sharing and dataspaces, and production deployment lessons. Developers building production-grade AI systems will find both technical depth and forward-looking guidance. Speaker Highlights: * * Commit to Quality: AI-Enhanced Testing in Open Source - Shelley Lambert & Longyu Zhang * Understanding Machine Decisions - Haishi Bai * Open Community for Compliance With regulations such as the EU Cyber Resilience Act (CRA) raising expectations for secure software, compliance is now a core engineering concern. This event equips teams with practical strategies for security, licensing, and regulatory alignment without slowing innovation. If you deliver software into the European market, this content is critical. Speaker Highlights: * * Taming the SBOM Chaos - A Legal Compass for the CRA and Open Source Compliance - Hendrik Schöttle * Layered Compliance: Using the Swiss Cheese Model to Prevent Catastrophic Failure - Georg Link * Open Community for Research Presented in collaboration with the Apereo Foundation and academic partners, this event demonstrates how open source accelerates the path from research to scalable, real-world systems. Topics include reproducibility, open science practices, and production-ready implementations bridging academia and industry. Speaker Highlights: * * VOStack open-source Software Stack for the virtualization of IoT devices - Anastasios Zafeiropoulos * UniTime Overview: From Research to Practice - Tomas Muller Featured keynote speakers OCX 2026 brings together perspectives from high-performance sport, European digital policy, and global open source leadership. * Ruth Buscombe - Formula 1 race strategist and F1TV analyst, Buscombe opens the conference with "The Winning Formula: What F1 Teaches Us About Marginal Gains, Teamwork, and Data-Driven Decision Making." Her keynote connects the precision of elite motorsport with the collaborative performance culture of open source communities. * Rolf Riemenschneider - Head of Sector IoT at the European Commission, Riemenschneider leads research and innovation under Horizon Europe, coordinating strategy for Cloud-Edge Computing and the Internet of Things. His keynote will explore Europe's digital trajectory and the role of open technologies in strategic data spaces. * Mike Milinkovich - Executive Director of the Eclipse Foundation since 2004, Milinkovich is a long-standing open source leader who has served on the boards of the Open Source Initiative, the OpenJDK community, and the Executive Committee of the Java Community Process. He will share insight on the evolving role of open collaboration in a rapidly shifting technology landscape. Register now and be part of what's next OCX 2026 is built for developers, architects, researchers, and decision-makers driving modern software forward. One registration unlocks the entire experience, including the main program and all five co-located events. Discounted registration rates are available until March 16. Capacity at The EGG conference centre is limited, and early registration is recommended. Thanks to our sponsors OCX 2026 is made possible through the generous support of our sponsors, including, SAP, TypeFox, EclipseSource, Obeo, and Equo Tech, Inc, AzAzul ETAS, Eurotech, and Mercedes-Benz Tech Innovation GmbH. Their leadership and commitment to open source innovation help drive collaboration across industries and strengthen the global open source ecosystem. We sincerely thank all of our sponsors for their partnership and commitment. Organisations interested in supporting OCX 2026 and engaging with the global open source ecosystem are invited to contact [email protected] to request sponsorship information and review the sponsorship prospectus. About the Eclipse Foundation The Eclipse Foundation provides a global community of individuals and organisations with a vendor-neutral, business-friendly environment for open source collaboration and innovation. We host Adoptium, the Eclipse IDE, Jakarta EE, Open VSX, Software Defined Vehicle, and more than 400 high-impact open source projects. Headquartered in Brussels, Belgium, we are an international non-profit association supported by over 300 members. Our events, including Open Community Experience (OCX), bring together developers, industry leaders, and researchers from around the world. To learn more, follow us on X and LinkedIn, or visit eclipse.org. Media contacts: Schwartz Public Relations (Germany) Julia Rauch/Marita Bäumer Sendlinger Straße 42A 80331 Munich [email protected] +49 (89) 211 871 -70/ -62 514 Media Ltd (France, Italy, Spain) Benoit Simoneau [email protected] M: +44 (0) 7891 920 370 Nichols Communications (Global Press Contact) Jay Nichols [email protected] +1 408-772-1551