S

State Street

Holds assets for institutions and runs index funds

Senior Data Platform Architect - Multiple Teams

Data Management +1Confirmed live in the last 24 hours
Full-timeBoston, MA, USAIn-person

About the Job

Requirements

  • Previous experience supporting workloads in analytics, regulatory reporting, artificial intelligence and machine learning, and batch applications.
  • Experience with Snowflake warehouse sizing, object hierarchy, Snowsight, role-based access control, Horizon Catalog, governance, performance, data sharing, external tables, stages, loading, stored procedures, scripts, and Snowpark.
  • Experience with Databricks, including Spark, Unity Catalog, machine-learning pipelines, Delta Lake, Iceberg, and Lakehouse Federation.
  • Experience with open platforms including Apache Iceberg, Delta Lake, Parquet, Polaris, and Trino or Starburst.
  • Experience with data virtualization, including Denodo, Starburst, Snowflake and Databricks federation, and logical data fabric.
  • Experience modernizing batch applications, including migration from Oracle and PL/SQL, mainframe, Unix ETL, and Control-M to cloud-native orchestration.
  • Experience with AWS, Azure, and Google Cloud Platform storage, identity and access management, networking, and private link.
  • Knowledge of data modeling, including Kimball, denormalization, dimensional and medallion schema patterns, bridge tables, marts, primary keys, foreign keys, clustering, and tuning implications.
  • Experience with SQL, Python, Java, JavaScript, Scala, Spark, notebooks, and Snowpark.
  • Knowledge of Linux and comfort working with EC2 servers and command-line tools such as vi and cat.
  • Experience in financial services or another regulated industry, including SOX, MRAs, audits, and handling nonpublic personal information and personally identifiable information.
  • Comfort scripting and prompting with generative AI, including knowing when scripting is more efficient and generating scripts using well-defined prompts and skill files.
  • Ability to share knowledge, learn from others, and coordinate with team members, project managers, and other teams.

Responsibilities

  • Define and govern hybrid Snowflake–Databricks architecture supporting data warehousing and data lake designs, aligning Snowflake's governed analytics layer with Databricks' engineering, streaming, and machine-learning capabilities.
  • Establish enterprise-scale, auditable reference blueprints and patterns using lakehouse and medallion approaches.
  • Contribute designs, patterns, and assets to the SSIM Center of Excellence for platform best practices and reusable use cases.
  • Lead Iceberg, Horizon, and Unity Catalog interoperability across AWS, Azure, and Google Cloud Platform.
  • Architect an open data platform strategy using Apache Iceberg, Delta Lake, Parquet, and Polaris or Unity Catalog to avoid vendor lock-in and enable cross-engine queries.
  • Define standards for open table format adoption, including business tool access protocols, drivers and standards; naming standards, folder structures, and cloud storage security; schema evolution, partitioning, and time travel; cross-engine compatibility; and catalog federation across Horizon, Unity, Glue, and Polaris.
  • Establish governance for open formats in regulated workloads, including lineage, access control, and encryption.
  • Drive interoperability between proprietary and open ecosystems to support modernization and exit strategies.
  • Collect user feedback, failures, and suggestions and provide distilled information to vendors to improve their offerings.
  • Architect data virtualization patterns that provide logical access across Snowflake, Databricks, Oracle, SQL Server, and legacy NAS or file systems without physical movement.
  • Define use cases for federated queries, virtual data products for analytics and reporting, and logical data fabrics spanning cloud and on-premises sources.
  • Establish performance, cost, security, and data-classification guardrails for virtualized access patterns.
  • Reduce data duplication, copy proliferation, and storage costs through virtualization where appropriate.
  • Lead architecture and solutioning for batch applications migrating from legacy platforms, including Oracle, mainframe, Unix/Linux ETL, and file-based workflows, to Snowflake or Databricks.
  • Define patterns for batch ingestion and orchestration; window- and service-level-agreement-driven processing with monitoring and rerun safety; migration of legacy stored procedures, PL/SQL, and shell-based ETL; and hybrid batch and streaming patterns.
  • Establish batch performance, cost, and resiliency standards, including reprocessing, transaction rollback, event logging, query timeout limits, warehouse credit limits, and identifying expensive processes from account-usage tables.
  • Partner with application owners to retire legacy infrastructure as part of data-center exit and cloud-migration programs.
  • Identify outdated drivers and application versions that may pose security risks and advise application teams on upgrades during modernization.
  • Design and enforce data security, including group-level read, read-write, and administrator roles at database, schema, and object levels, balancing security needs with administrative overhead.
  • Advise application teams on lower-level row-access policies, column masking, and secure views to provide row- and column-level security where needed.
  • Establish future grants and apply knowledge of managed-access schemas and their tradeoffs, network-address whitelisting, cloud-level roles, and security integrations that enable data access with Snowflake Horizon and Databricks Unity Catalog.
  • Ensure compliance with SOX, MRAs, internal and external audits, and data-protection controls.
  • Maintain defensible audit trails, lineage, and forensic traceability.
  • Align with GCS data residency, isolation, and classification standards.
  • Architect cloud-native integrations across S3, ADLS, GCS, identity and access management, networking, and private-link patterns.
  • Drive cross-region and cross-cloud high availability and disaster recovery designs, including Snowgrid replication and Iceberg federation.
  • Embed vendor security, networking, real-time and streaming capabilities, and disaster recovery in platform designs.
  • Act as a subject-matter expert for Snowflake, Databricks, open platforms, virtualization, and batch modernization during architecture reviews and escalations.
  • Partner with Security (GCS/GTS), Risk, Audit, Application, and Cloud Engineering teams.
  • Produce architecture diagrams, design artifacts, executive-level presentations, and access models.

About the Company

State Street

Holds assets for institutions and runs index funds

State Street holds and services investments for pension funds, insurers, fund companies, and other large institutions, and it also runs index funds through State Street Investment Management. It launched the first US-listed exchange-traded fund, the SPDR S&P 500 ETF Trust, in 1993, and in 1924 it became custodian of the first US mutual fund. As a custody bank, it safekeeps clients' stocks and bonds, completes their trades, calculates what their funds are worth, and lends their securities to other investors. It earns fees on the assets it services and manages, plus income from currency trading and from interest on client cash. State Street traces its roots to a bank chartered in 1792 and is based in Boston, Massachusetts.

Company size
10,001+
Company stage
IPO
Headquarters
Boston, MA
Founded
1792

Recently Posted Jobs

Explore more opportunities at State Street

See more jobs