Your Role

Work directly with the Nayya health data infrastructure team to power the claims data extraction and integration platform. In addition to maintaining and working on claims file feeds and API integrations, you will join the team as we build a high volume, batch file claims data pipeline.

The team maintains an internal facing API to power end user applications that leverage claims to enhance user experiences. You will help with maintaining that infrastructure, building new endpoints to power new use cases as well as working on an emerging, centralized data strategy for the organization. We will be working as a team to enhance data validation, governance, and access to organizational data from a single source of truth while building new infrastructure to exceed product requirements for our 2024 product goals and beyond. This is an excellent opportunity to get hands-on experience solidifying and scaling data infrastructure at a fast-growing startup. 

Responsibilities

• Build and maintain our Interoperability API and claims file feed integrations.
• Help design and implement new batch processing infrastructure.
• Enhance our data enrichment service to interface with external/third party data sources.
• Help design and implement a de-identified reporting and analytics platform.
• Maintain and improve an existing Python tech stack with a focus on security and scalability for data storage and API endpoints.
• Work with the internal stakeholders to build and maintain data and pipelines for various use cases.
• Recommend monitoring and analytics tools to automate common data needs and visibility.

Requirements

• Strong Python experience.
• Strong PySpark experience.
• Experience with an orchestration tool like Airflow, Dagster, etc.
• Strong SQL skills including query performance and optimization techniques including explain plans.
• Experience building and maintaining APIs to serve data to internal stakeholders.
• Familiarity with implementing and using Apache Spark, AWS Glue, and/or EMR.
• Data pipeline development and maintenance experience.
• Familiarity with batch processing pipelines.
• Familiarity with data pipeline tuning and performance testing.
• Familiarity with building data processing infrastructure from the ground up.
• Familiarity with one or more RDBMS.
• Ability to identify tradeoffs for warehousing vs data lake infrastructure and applying solutions to the appropriate use case.
• Familiar with common pitfalls in high volume, partitioned data ingestion pipelines such as orphaned records and table locks

Nice to haves:

• Experience with Terraform.
• Experience with Apache Hudi or other data lake platforms.
• Experience with data streaming pipelines and sources.
• Experience building and maintaining APIs (Flask, FastAPI, etc).
• Experience with AWS managed services and data engineering solutions.
• Experience with other languages such as Go, Ruby, and Javascript.

The salary range for New York based candidates for this role is $140,000-$175,000. We use a location factor to adjust this range for candidates that are located outside of geographic region of our New York office. Placement within the salary band is determined based on experience.

#LI-KD1

#LI-HYBRID

#BI-Hybrid