Senior Data Engineer - IT Data Engineering and Analytics

Overview

Overview 

Connecting clients to markets – and talent to opportunity

With 4,300 employees and over 400,000 retail and institutional clients from more than 80 offices spread across five continents, we’re a Fortune-100, Nasdaq-listed provider, connecting clients to the global markets – focusing on innovation, human connection, and providing world-class products and services to all types of investors.

At StoneX, we offer you the opportunity to be part of an institutional-grade financial services network that connects companies, organizations, and investors to the global markets ecosystem. As a team member, you'll benefit from our unique blend of digital platforms, comprehensive clearing and execution services, personalized high-touch support, and deep industry expertise. Elevate your career with us and make a significant impact in the world of global finance.

Business Segment Overview:

The Enterprise Data & Technology (EDT) team supports a wide variety of data assets and technologies in support of StoneX business initiatives for institutional and retail customers, while providing data services to almost all areas of the organization. Reporting to the Chief Data Officer, the EDT organization serves multiple internal stakeholders.

Corporate: Engage in a deep variety of business-critical activities that keep our company running efficiently. From strategic marketing and financial management to human resources and operational oversight, you’ll have the opportunity to optimize processes and implement game-changing policies.

Responsibilities

Job Purpose and Responsibilities

Position Purpose: The Senior Data Engineer (Forward Deployed) is a hands-on engineer who works alongside the business, not behind a ticket queue. You will embed with trading, operations, finance and risk teams to understand how they work, turn their questions into precise source-to-target mappings, and ship production Databricks pipelines on the StoneX Unified Data Platform in days and weeks, not quarters. AI is a core part of how you work: you use it to analyse sources, draft mappings, generate and test code, and build reusable frameworks that make the whole Global Data Services team faster. The role suits an engineer who enjoys a live problem, a demanding stakeholder and a fast pace, and who can explain a technical trade-off as clearly to a trader as to an engineer.

What 'Forward Deployed' means here:

• Embedded: you spend a large share of your time with business users – in their meetings, their tools and their data – and you own the outcome, not just the code.
• End to end: discovery → mapping → pipeline → validation → adoption, delivered by one accountable engineer.
• Fast and iterative: working increments in front of users early, with short feedback loops.
• Leverage: every engagement leaves something reusable behind (a framework, a template, an agent) so the next one is faster.

Technology Ecosystem:

• Data platform: Databricks (Delta Lake, Unity Catalog, Lakeflow / Workflows, Databricks SQL), Python, PySpark, SQL, SQL Server
• Integration: RedPanda (Kafka), OpenAPI / REST APIs, Airflow, Profisee MDM, Microsoft Dynamics
• AI-assisted delivery: Claude Code, GitHub Copilot, Databricks Assistant and AI/BI Genie
• Engineering: medallion architecture (Bronze / Silver / Gold), data warehousing, CI/CD (GitHub Actions, Jenkins), Docker, Kubernetes
• Consumption and cloud: Power BI, React.js; Microsoft Azure

Primary duties will include:

• Discover and translate: run discovery sessions with business stakeholders; turn loosely defined asks into clear requirements, data contracts and acceptance criteria; play your understanding back in plain language.
• Map: produce rigorous source-to-target mappings (STTM) covering field-level lineage, transformation logic, keys, defaults, SCD handling and data quality rules, signed off by engineers and business users alike.
• Build: design and build batch and streaming pipelines on Databricks (PySpark, SQL, Delta Lake, Unity Catalog, Lakeflow) that follow the medallion patterns of the Unified Data Platform.
• Accelerate with AI: use AI coding agents every day to profile sources, draft STTMs, and generate pipelines and tests; build reusable AI-enabled frameworks such as metadata-driven ingestion, STTM-to-code generation and automated data quality and reconciliation checks.
• Validate and land: reconcile outputs with the business, own data quality and monitoring, and drive adoption in Power BI and downstream systems.
• Operate: own production reliability for what you build (CI/CD, alerting, incident response, root-cause analysis), including cost and performance tuning on shared Databricks compute.
• Communicate: give crisp written and verbal updates to business and technology leaders, record decisions in Confluence and Jira, and reset expectations early when scope, dates or data change.
• Raise the bar: coach engineers on business engagement and AI-assisted delivery; review others' AI-generated code for correctness, security and performance.

Qualifications

Qualifications:

To land this role you will need:

• 6–9 years of hands-on data engineering, including 3+ years building production pipelines on Spark / Databricks.
• 2+ years in a business-facing delivery role (forward-deployed, consulting, solutions or embedded analytics engineering) where you owned outcomes with non-technical stakeholders.
• Proven ability to write source-to-target mappings and dimensional / medallion data models from ambiguous requirements.
• Expert Python, PySpark and SQL; strong Delta Lake, Unity Catalog and Databricks Workflows / Lakeflow experience on Microsoft Azure.
• Recent, demonstrable use of AI coding agents to deliver production work, with the judgment to verify, test and reject what they produce.
• Excellent verbal and written communication; confident running workshops and explaining trade-offs to senior business stakeholders.
• Thrives at pace: comfortable with shifting priorities, tight deadlines and overlap with US / UK business hours when needed.

What makes you stand out:

• Capital markets, commodities or trade-lifecycle data experience (trades, positions, postings, balances, reference data).
• You have built a reusable framework or internal tool, ideally AI-enabled, that other engineers adopted.
• Streaming with Kafka / RedPanda, and APIs (OpenAPI / REST) as data sources.
• MDM (Profisee) and CRM (Microsoft Dynamics) data integration.
• Power BI semantic models or Databricks AI/BI (Genie) spaces that business users rely on.
• Time in a start-up, consulting firm or forward-deployed engineering team.

Education / Certification Requirements:

• Bachelor's degree in Computer Science, Engineering, Mathematics or a related field, or equivalent demonstrable experience.
• Databricks Certified Data Engineer Professional (strongly preferred) or Associate.
• Azure Data Engineer Associate or equivalent.
• Generative AI certifications (e.g., Databricks Generative AI Engineer Associate) are a plus.

Working environment:

• Hybrid (as per RTO policy), Bengaluru.
• Embedded with business and technology teams across the US, UK, India and Poland; regular overlap with US / UK hours.
• Travel: occasional, for discovery workshops and go-lives.