Software Engineer New Grad

FlexSysAI launches grid-flexibility pilot with ResetData, CSIRO, University of Queensland. Will test AI orchestration platform at ResetData data center September 09, 2026 Australian AI workload orchestration firm FlexSysAI has launched a pilot with ResetData, Australia's national science agency CSIRO, and the University of Queensland to test its new platform's ability to shift AI workloads without impacting performance. As part of the pilot, ResetData will provide the sovereign AI infrastructure and live AI data center environment to test the technology, with CSIRO and the University of Queensland supporting the validation of the project. The project is already underway and is managing workloads of an Nvidia H200 cluster at ResetData's AI-F1. "Early testing has been extremely promising, showing our platform can rapidly respond to electricity market signals and shift AI workloads to when and where power is more readily available. This reduces strain on the energy system, lowers operators' energy costs, and unlocks additional capacity in existing infrastructure, meaning quicker connections to the Grid," said Victor Feoktistov, co-founder of FlexSysAI. The FlexSysAI platform, launched last month, works by connecting live electricity market and grid conditions with AI workload balancing, shifting compute workloads to locations where power is cheap and abundant, while also allowing non-critical workloads to be reduced where necessary during periods of grid stress. The company claims it will allow data center operators to connect to the grid sooner, lower power bills, access low-carbon power during periods of abundance, and monetize demand response services, while providing optionality over where workloads are shifted. According to the companies, the pilot will use FlexSysAI to categorize AI workloads into distinct "Flex Tiers," ensuring that critical tasks are protected while elastic training jobs provide reliable load-shifting. "It's great to be working with FlexSysAI, CSIRO and the University of Queensland on this pilot. As Australia's sovereign AI infrastructure provider, we want to enable Australian innovation by providing the infrastructure to test and prove new ideas that can have a meaningful impact across the industry," added Marcel Zalloua, co-CEO of ResetData. FlexSysAI is one of several companies to develop a platform that can modulate AI workloads to reduce grid impact. The most notable is Emerald AI, which has developed a platform called the Emerald Conductor that mediates between the grid and data centers, orchestrating AI workloads in real time. The company has completed several demonstration projects to prove its system works within real-world data center environments. So far, it has completed two in the US, in Phoenix and Chicago, and one in the UK in partnership with the National Grid. The company announced its first commercial project in October, with the orchestration software set to be deployed at Nvidia's under-construction 96MW Aurora data center in Manassas, Virginia. More in australasia.

AI infrastructure that adapts as you grow. Deploy once. FlexAI handle the rest. FlexBench: an open-source, modular MLPerf benchmark for LLM inference. March 31, 2026 Tl;dr. AI system benchmarks like MLPerf struggle to keep pace with the rapidly evolving model landscape, making it difficult for organizations to make informed deployment decisions. FlexAI believe benchmarking should itself be treated as a machine learning problem - one where models are continuously tested and optimized across datasets, software, and hardware based on metrics like accuracy, latency, throughput, power consumption, and cost. That's why FlexAI built FlexBench: a modular, open-source version of the MLPerf LLM inference benchmark connected to Hugging Face. FlexBench aggregates existing and new benchmarking results into an Open MLPerf dataset, which can be collaboratively cleaned, extended, and used for predictive modeling. FlexAI validated FlexBench through its MLPerf Inference 5.0 submission, benchmarking DeepSeek R1 and LLaMA 3.3 on commodity H100 servers. Its long-term goal: empower teams to make cost-effective AI deployment decisions based on their available resources, requirements, and constraints. Why MLPerf falls short for LLM inference benchmarking. AI service providers, server developers, and data center operators face a critical challenge: selecting the right hardware and software stack to ensure ROI within 3-5 years in a rapidly shifting landscape [1]. MLPerf was introduced as a full-stack inference benchmark to evaluate accuracy, latency, and throughput in a standardized, reproducible manner across diverse hardware and software stacks [2]. But traditional benchmarks face a fundamental limitation: the combinatorial explosion of models, datasets, methods, and hardware configurations. Hugging Face alone hosts over a million ML models, more than 10,000 datasets, and thousands of methods - while new (and often incompatible) hardware and software ship continuously. Exploring all possible configurations is not only impractical, it's prohibitively expensive. MLPerf currently covers only a limited set of combinations - typically around a dozen - and updates just once a year. Its LLM benchmarks still focus on models like BERT, GPT-J, LLaMA 2 70B, LLaMA 3 405B, and Mixtral 8x7B, even as newer models like DeepSeek dominate production workloads. Worse, its hands-on experience with MLPerf shows that heavily over-optimized results from a few chip manufacturers are rarely achievable out of the box on other models, software versions, or hardware - significantly limiting their practical usefulness. Reframing GPU benchmarking as a machine learning problem. FlexAI believe a fundamentally different approach is needed. Drawing on its past experience using AI to improve computer systems, FlexAI propose redefining MLPerf benchmarking as a learning task - with an open dataset of results and trainable objective functions to optimize key metrics such as accuracy, latency, throughput, power consumption, and cost [3][4][5]. To support this vision, FlexAI developed FlexBench - an open-source, modular, and flexible version of the MLPerf language inference benchmark connected to the Hugging Face Hub. With a unified codebase and CLI, users can benchmark a wide range of models and datasets by adjusting just a few input parameters. FlexBench is designed for continuous evolution. FlexAI use the MLCommons CMX workflow automation framework to aggregate both existing and new benchmarking results - along with their associated metadata - into an open MLPerf dataset published on GitHub and Hugging Face. This dataset can be collaboratively cleaned, extended, and analyzed using standard data analytics techniques, including predictive modeling and feature engineering. FlexAI then use FlexBoard to visualize, compare, and predict the most suitable software/hardware configurations for different models based on user requirements and constraints. FlexBench architecture: client-server LLM inference benchmarking. FlexBench uses a client-server architecture where the FlexBench client connects to a running vLLM server. It's built on MLPerf LoadGen, the official and reusable MLPerf harness that efficiently and fairly measures inference system performance [2][9]. Its goal is to retain MLPerf's rigorous measurement standards while making the framework more flexible by abstracting models and datasets as interchangeable modules. Hugging Face or local LLMs and datasets can be used with minimal setup. FlexBench supports two standard MLPerf inference modes: * Server (streaming) mode: Queries arrive according to a Poisson distribution, mimicking real-world request patterns * Offline mode: All queries are sent simultaneously to maximize throughput FlexBench returns detailed metrics from LoadGen - including TTFT, throughput, and latency percentiles - all compliant with MLPerf standards and suitable for inclusion in the Open MLPerf dataset for further analysis and predictive analytics. FlexAI cross-validated these results against the vLLM benchmarking infrastructure and found strong alignment in performance numbers. FlexBench also provides accuracy metrics to guide further model optimizations such as quantization, pruning, and distillation. FlexAI has introduced a Query Per Second (QPS) sweep mode to help users automatically identify the optimal QPS for their specific model, software, and hardware combination. FlexBoard is implemented as a Gradio module that loads the Open MLPerf dataset via MLCommons CK/CM/CMX automations. It includes various predictive modeling and visualization plugins to help users analyze this data and predict the most efficient and cost-effective software/hardware configurations based on their requirements and constraints. Benchmarking DeepSeek R1 and LLaMA 3.3 on H100 GPUs. FlexAI validated FlexBench through its MLPerf Inference 5.0 submission by benchmarking several non-MLPerf LLM models - including DeepSeek R1 and LLaMA 3.3 - on the OpenOrca dataset, using commodity servers equipped with NVIDIA H100 GPUs. Its automation framework enabled rapid switching between models, datasets, and hardware configurations by simply modifying command-line parameters, without requiring any code changes. FlexAI also invested significant effort assembling the Open MLPerf dataset by unifying past MLPerf Inference results (v4.0) and combining them with the latest official submissions and FlexBench data. To enable predictive modeling, FlexAI cleaned the dataset, standardized disparate fields, and engineered new features such as model size and data type. FlexAI has released this cleaned and curated dataset, along with FlexBoard and predictive analytics tools, to help the broader ML, AI, and systems community accelerate benchmarking, evaluation, and optimization efforts. For example, its proof-of-concept prototype allows users to input system costs and predict optimal software/hardware configurations based on model size and data type features. What's next for FlexBench and Open MLPerf. FlexBench and FlexBoard are still in early-stage prototyping. FlexAI invite researchers and practitioners to explore the tools, provide feedback, and collaborate on the following: * Extending FlexBench to support all types of models, datasets, and systems * Expanding the Open MLPerf dataset with FlexBench results from various models across diverse software/hardware configurations from different vendors * Engineering improved features - such as model graphs, tensor shapes, compiler optimizations, accelerator capabilities, and hardware topology - to enhance predictions for previously unseen AI workloads * Extending and improving FlexBoard based on user requirements and feedback * Sharing your specific benchmarking challenges to help guide its priorities Its long-term goal is to enable anyone to run AI models efficiently and cost-effectively, tailored to their available resources, requirements, and constraints. If you're interested in its approach or would like to collaborate, reach out to the authors at FCS Labs. References. Get started today. Start building with €100 in free credits for first-time users.