AI Systems Engineer – Senior - Data & State Management

The opportunity

We are seeking an AI Systems Engineer to own the stateful backbone of EY’s AI-native platform, including the data stores, memory tiers, and event streaming systems that hold and move every piece of state in EY’s Hybrid AI Multi-Environment Runtime (HAI). This role ensures that agentic AI workloads have durable, performant, and consistent access to data across cloud, on-prem, edge, client-managed, and air-gapped environments.

Within HAI, this is a distinct discipline from platform operations and model serving. This role owns everything that must remember, persist, or flow: relational and key-value state, vector and graph stores, object storage, durable workflows, and the streaming and change-data-capture pipelines that connect them. It is the layer that makes the platform stateful, reliable, and event-driven.

It is ideal for a data-infrastructure engineer who is equally comfortable operating production databases and high-throughput streaming systems, who treats data durability, consistency, and recoverability as non-negotiable in regulated client contexts, and who understands that state is the hardest part of any distributed platform to get right.

Your key responsibilities

• Own the memory and data stores: relational and durable state (PostgreSQL, DBOS durable workflows), caching (Redis/Valkey), vector stores (Qdrant/Milvus/PGVector), knowledge graphs (Neo4j), and object/block storage (MinIO, OpenEBS Mayastor), across every environment and tenant.
• Own event streaming and async messaging: Apache Kafka (Strimzi), NATS JetStream (agent-to-agent), Debezium (change data capture), Apache Flink (stream processing), and Apicurio/CloudEvents (schema and event contracts).
• Own data durability, consistency, and recoverability: replication, backup/restore, point-in-time recovery, and cross-environment data movement, tiered by RPO/RTO.
• Build and operate streaming and CDC pipelines that move data reliably between stores and services, with schema governance and evolution that prevents breaking changes across producers and consumers.
• Make state multi-tenant and portable, ensuring isolation, performance, and consistent semantics whether running on managed cloud services or self-hosted OSS in an air-gapped environment.
• Provide the data and lineage substrate that downstream governance, observability, and AI knowledge capabilities depend on, as well as integrating with lineage tooling.

Skills and attributes for success

• Deep expertise operating production databases and data stores at scale, including relational, key-value, vector, graph, and object storage.
• Strong command of streaming and event-driven architectures (Kafka, NATS, CDC, stream processing) and the consistency tradeoffs they involve.
• A durability-first mindset: thinking in terms of consistency, recoverability, blast radius, and data correctness under failure.
• Ability to operate stateful systems consistently across managed cloud and self-hosted OSS in cloud, on-prem, edge, and air-gapped environments.
• Strong grasp of schema governance and evolution, preventing breaking changes across producers and consumers.
• Strong communicator able to guide consuming teams toward the right storage and streaming patterns.
• Orientation toward reliability and toil reduction through automation and infrastructure-as-code for data systems.

To qualify you must have

• Bachelor’s or Master’s degree in Computer Science or related technical field.
• 8+ years operating production data infrastructure, streaming systems, or database platforms at scale.
• Hands-on expertise with relational databases (PostgreSQL) and caching (Redis/Valkey), including HA, replication, and backup/recovery.
• Exposure to AI/ML data patterns — embeddings, retrieval, feature/state stores for agentic workloads.
• Production experience with vector and/or graph databases (Qdrant, Milvus, PGVector, Neo4j) in AI/ML contexts.
• Deep experience with event streaming and messaging (Apache Kafka/Strimzi, NATS) and change data capture (Debezium).
• Experience with stream processing (Apache Flink) and event/schema governance (CloudEvents, schema registry).
• Experience running stateful systems on Kubernetes (operators, persistent volumes, object/block storage such as MinIO/OpenEBS).
• Ability to define clean ownership boundaries and data/schema contracts with platform, trust, runtime, and delivery teams.

Ideally, you’ll also have

• Experience with durable workflow engines (DBOS, Temporal, or equivalents).
• Experience with data lineage and metadata tooling (OpenLineage, Marquez, or equivalents).
• Proven track record operating data systems under compliance, security, or regulatory constraints, including data-at-rest and in-transit protection.
• Familiarity with multi-tenant data isolation and per-tenant performance management.
• Experience with cross-environment replication and DR strategies tiered by RPO/RTO (e.g., MirrorMaker, CloudNativePG PITR).
• Exposure to regulated delivery environments (financial services, tax, healthcare, risk).

What we offer you
At EY, we harness our collective strength to empower you to shape your future with confidence through professional growth, personal fulfillment and an inclusive culture. Learn more at ey.com/us/careers.

• The salary range for this job is:
  • New York City, Boston, and Washington DC Metro Areas, Washington State, and Southern California offices – $128,400 to $192,500
  • Bay Area California offices – $133,600 to $200,600
  • All other offices locations in the US, including Sacramento – $106,900 to $176,500
• Individual salaries within these ranges are determined through a wide variety of factors including but not limited to education, experience, knowledge, skills and geography.  In addition, our Total Rewards package includes medical and dental coverage, pension and 401(k) plans, and a wide range of paid time off options.

Are you ready to shape your future with confidence? Apply today.                                          

• To make the most of your application experience, please limit yourself to two applications within a six-month period.
• EY accepts applications for this position on an on-going basis.
• For those living in California, please click herefor additional information.
• At EY, our values set the foundation for how we work and the behaviors we expect of our people.  Any misrepresentation or falsification of information or lack of integrity at any point in the recruiting process may result in withdrawal of your candidacy, revocation of an offer or immediate termination of employment.

EY  |  Building a better working world

EY is building a better working world by creating new value for clients, people, society and the planet, while building trust in capital markets.

Enabled by data, AI and advanced technology, EY teams help clients shape the future with confidence and develop answers for the most pressing issues of today and tomorrow.

EY teams work across a full spectrum of services in assurance, consulting, tax, strategy and transactions. Fueled by sector insights, a globally connected, multi-disciplinary network and diverse ecosystem partners, EY teams can provide services in more than 150 countries and territories.

All in to shape the future with confidence.

EY provides equal employment opportunities to applicants and employees without regard to race, color, religion, age, sex, sexual orientation, gender identity/expression, pregnancy, genetic information, national origin, protected veteran status, disability status, or any other legally protected basis, including arrest and conviction records, in accordance with applicable law.  

EY is committed to providing reasonable accommodation to qualified individuals with disabilities including veterans with disabilities. If you have a disability and either need assistance applying online or need to request an accommodation during any part of the application process,  please call 1-800-EY-HELP3, select Option 2 for candidate related inquiries, then select Option 1 for candidate queries and finally select Option 2 for candidates with an inquiry which will route you to EY’s Talent Shared Services Team (TSS) or email the TSS at [email protected].