Full-Time
Senior Software Engineer
Posted on 4/6/2026
Galileo
51-200 employees
Data-centric NLP ML quality and labeling
No salary listed
Bengaluru, Karnataka, India
Hybrid
Three days on-site per week in Bengaluru.
 Software Engineering (2) |
|---|
| , |
- 7+ years of experience building Data or AI/ML products
- Experience building APIs in Python (FastAPI preferred)
- Experience building large scale distributed systems and familiarity with Messaging and Pub-Sub systems such as RabbitMQ/Kafka
- Experience with Storage/database systems similar to Postgres, MongoDB, Cassandra, S3
- Experience with OLAP systems such as ClickHouse, Pinot
- Experience with Orchestration systems such as Celery
- Experience building test suites using frameworks such as PyTest/PyUnit
- Experience working effectively during the product development process, working cross-functionally with a product manager, designer, user researcher, and data scientist
- Excellent communication skills for collaborating with cross-functional partners
- High productivity and care to help teams collaborate more effectively and efficiently
- A startup mindset, biasing towards thoughtful action with minimal direction
- Design, build, and scale Galileo's products and team, focusing on data, ML, observability and reliability for GenAI
- Collaborate with cross-functional partners including product managers, designers, user researchers, and data scientists to drive product development
- Experience working with Docker and Kubernetes
- Experience building applications on AWS, Google Cloud, or Microsoft Azure
- Experience building customer facing products with ML frameworks (like PyTorch, TensorFlow, Keras, etc)
- Experience building agentic applications (like LLMs, retrieval-augmented generation, agents, etc.) with common frameworks (like OpenAI, LangChain, CrewAI, etc.)
Rungalileo.io is a platform for machine learning teams to improve models and cut annotation costs. It uses data-centric NLP techniques to quickly find and fix data issues that hurt model performance and provides a collaborative data bench to manage and track models from raw data to production. It also detects when a model goes down in production and identifies the exact data it failed on. Unlike others, it integrates with existing tools in minutes and prioritizes actionability, security, and privacy. It lets teams choose which data to label, automatically detect mis-annotated data, and bulk label all in one place. The company earns revenue by charging a subscription fee for its services.
Company Size
51-200
Company Stage
Series B
Total Funding
$68.1M
Headquarters
San Francisco, California
Founded
2021
Simplify's Take
What believers are saying
- Cisco acquires Galileo, closing July 2026, boosts Splunk AI observability.
- NVIDIA NeMo integration on March 18, 2025, accelerates GenAI data flywheel.
- $45M Series B funding expands generative AI evaluation platforms.
What critics are saying
- Cisco acquisition fails by July 2026 from antitrust scrutiny over Splunk dominance.
- Open-sourcing Agent Control lets LangChain, Glean fork technology in 6-12 months.
- Honeycomb, Datadog capture 30% Splunk pipeline in 12-18 months.
What makes Galileo unique
- Galileo integrates in minutes with OpenAI, Anthropic, Azure OpenAI, AWS Bedrock.
- Galileo Luna EFMs launched June 6, 2024, transform enterprise GenAI evaluations.
- Galileo open-sources Agent Control for scalable AI agent governance.
Help us improve and share your feedback! Did you find this helpful?
Benefits
Health Insurance
Dental Insurance
Vision Insurance
Disability Insurance
Parental Leave
Flexible Work Hours
401(k) Retirement Plan
401(k) Company Match
Growth & Insights and Company News
6 month growth
↓ -1%1 year growth
↑ 1%2 year growth
↑ 3%Cisco is acquiring Galileo, an AI observability specialist, to strengthen Splunk's position in the AI monitoring market. The deal is expected to close in July 2026. Galileo provides tools to evaluate AI output quality, detect errors before they reach users, and improve AI agent behaviour in production. The platform monitors hallucinations, bias, security risks and cost metrics across the entire agent development lifecycle, offering real-time observability for multi-agent systems. The acquisition will integrate Galileo into Splunk Observability Cloud, expanding existing AI agent monitoring capabilities. Galileo offers over 20 evaluation metrics including hallucination detection and supports major AI platforms like OpenAI, Anthropic, Azure OpenAI and AWS Bedrock. Cisco and Galileo previously collaborated on Cisco's AGNTCY initiative. Both companies will operate independently until the deal closes.
Cisco buys Galileo to strengthen Splunk's agentic monitoring capabilities - SiliconANGLE
Galileo has released Agent Control, an open source control plane enabling organisations to govern AI agents at scale. The platform allows users to write policies once and deploy them across all AI agents, addressing a critical barrier to enterprise AI adoption. CrewAI, Glean, Cisco AI Defense and Strands Agents will be the first to integrate with Agent Control. The platform provides centralised policy management, runtime mitigation for real-time updates, and supports guardrail evaluators from any vendor. Distributed under the Apache 2.0 licence, Agent Control addresses enterprise concerns around trust and governance that have prevented agents from reaching production. Use cases include preventing hallucinations, blocking data leaks, reducing token costs and enforcing brand standards. The platform is backed by Battery Ventures, Scale Venture Partners, Databricks Ventures and ServiceNow.
Join our daily and weekly newsletters for the latest updates and exclusive content on industry-leading AI coverage. Learn More. AI agents have a safety and reliability problem. Agents would allow enterprises to automate more steps in their workflows, but they can take unintended actions while executing a task, are not very flexible, and are difficult to control.Organizations have already sounded the alarm about unreliable agents, worried that once deployed, agents might forget to follow instructions. OpenAI even admitted that ensuring agent reliability would involve working with outside developers, so it opened up its Agents SDK to help solve this issue. But researchers from the Singapore Management University (SMU) have developed a new approach to solving agent reliability. AgentSpec is a domain-specific framework that lets users “define structured rules that incorporate triggers, predicates and enforcement mechanisms.” The researchers said AgentSpec will make agents work only within the parameters that users want.Guiding LLM-based agents with a new approachAgentSpec is not a new LLM but rather an approach to guide LLM-based AI agents. The researchers believe AgentSpec can be used not only for agents in enterprise settings but useful for self-driving applications. The first AgentSpec tests integrated on LangChain frameworks, but the researchers said they designed it to be framework-agnostic, meaning it can also run on ecosystems on AutoGen and Apollo. Experiments using AgentSpec showed it prevented “over 90% of unsafe code executions, ensures full compliance in autonomous driving law-violation scenarios, eliminates hazardous actions in embodied agent tasks, and operates with millisecond-level overhead.” LLM-generated AgentSpec rules, which used OpenAI’s o1, also had a strong performance and enforced 87% of risky code and prevented “law-breaking in 5 out of 8 scenarios.”Current methods are a little lackingAgentSpec is not the only method to help developers bring more control and reliability to agents
Platform Powers End-to-End Continuous Improvement of Agentic ApplicationsSAN FRANCISCO, March 18, 2025 /PRNewswire/ -- Galileo, the AI Evaluation company, today announced an integration with NVIDIA NeMo ™, enabling customers to continuously improve their custom generative AI models. Now, customers can evaluate models comprehensively across the development lifecycle, curating feedback into datasets that power additional fine-tuning. As a result, customers ship GenAI apps that are more reliable, trusted, and cost-effective.Data Flywheel for AIThe majority of enterprises are developing GenAI applications – including agents and RAG-based chatbots – but it can be challenging to ship and scale these applications due to the non-deterministic outputs of Large Language Models (LLMs). There's even more complexity when AI teams wish to test new LLMs, which are constantly evolving in capability and price point. The solution is to build an AI data flywheel, enabling continuous testing and refinement, collecting data about user interactions for subsequent improvement. When AI teams use data to improve outcomes (whether by fine-tuning, prompt engineering, or in-context learning), they gain a competitive advantage.Galileo and NVIDIA accelerate a data flywheel by collecting and curating better data about the interactions of an AI application