ValGenesis

Cloud-based validation lifecycle management platform

Lead Software Engineer - Data Engineering

Data EngineeringConfirmed live in the last 24 hours

Full-timeChennai, Tamil Nadu, IndiaNo salary listed

About the Job

Requirements

Six to twelve years of experience in Data Engineering or equivalent roles.
Strong proficiency in Python, SQL, and at least one compiled language: C#, Java, or Scala.
Experience with relational databases including SQL Server, PostgreSQL, and MySQL, and NoSQL systems including MongoDB and Cosmos DB.
Hands-on experience with Azure Data Lake and Databricks.
Experience with Azure Data Factory, Apache Airflow, or dbt.
Experience with Kafka, Event Hubs, or Service Bus for real-time data processing.
Familiarity with machine learning frameworks including TensorFlow and PyTorch, and MLOps concepts.
Experience with Power BI, Apache Superset, or Tableau.
Experience with Azure, Docker, Kubernetes, and GitHub Actions or Azure DevOps.
Solid understanding of data modeling, version control, and CI/CD for data systems.
Strong analytical and problem-solving ability.
Excellent communication and collaboration skills.
Attention to detail and ability to work with large, complex datasets.
Creativity and ability to automate repetitive workflows.
Passion for continuous learning and innovation in data and AI technologies.

Responsibilities

Architect, design, develop, and maintain batch and real-time data ingestion, transformation, and orchestration pipelines.
Build and optimize data Lakehouse architectures using Azure Synapse, Delta Lake, or similar frameworks.
Integrate and manage structured and unstructured data sources, including SQL and NoSQL systems, files, documents, and IoT streams.
Develop and operationalize ETL/ELT pipelines using Azure Data Factory, Databricks, or Apache Spark.
Collaborate with Data Scientists to prepare and serve ML-ready datasets for model training and inference.
Implement data quality, lineage, and governance frameworks across pipelines and storage layers.
Work with BI tools including Power BI, Superset, and Tableau to enable self-service analytics for business teams.
Deploy and maintain data APIs and ML models in production using Azure ML, Kubernetes, and CI/CD pipelines.
Ensure scalability, performance, and observability of data workflows through effective monitoring and automation.
Collaborate cross-functionally with engineering, product, and business teams to translate insights into action.
Mentor junior team members and review their work.

Desired Qualifications

Experience in knowledge graph or semantic search solutions.
Understanding of LLM-based data retrieval (RAG) patterns.
Exposure to data mesh, data fabric, or domain-oriented data architecture.
Familiarity with MLflow, Delta Live Tables, or Databricks Unity Catalog.

About ValGenesis

ValGenesis is a leading digital validation platform provider for life sciences companies. ValGenesis suite of products are used by 30 of the top 50 global pharmaceutical and biotech companies to achieve digital transformation, total compliance and manufacturing excellence/intelligence across their product lifecycle.

Learn more about working for ValGenesis, the de facto standard for paperless validation in Life Sciences: https://www.valgenesis.com/about

About the Role:

Responsibilities

Architect, Design, develop, and maintain data ingestion, transformation, and orchestration pipelines (batch and real-time).
Build and optimize data Lakehouse architectures using Azure Synapse, Delta Lake, or similar frameworks.
Integrate and manage structured and unstructured data sources (SQL/NoSQL, files, documents, IoT streams).
Develop and operationalize ETL/ELT pipelines using Azure Data Factory, Databricks, or Apache Spark.
Collaborate with Data Scientists to prepare and serve ML-ready datasets for model training and inference.
Implement data quality, lineage, and governance frameworks across pipelines and storage layers.
Work with BI tools (Power BI, Superset, Tableau) to enable self-service analytics for business teams.
Deploy and maintain data APIs and ML models in production using Azure ML, Kubernetes, and CI/CD pipelines.
Ensure scalability, performance, and observability of data workflows through effective monitoring and automation.
Collaborate cross-functionally with engineering, product, and business teams to translate insights into action.
Mentor junior team members and review their work

Requirements

Experience: 6 to 12 years in Data Engineering or equivalent roles.
Programming: Strong in Python, SQL, and at least one compiled language (C#, Java, or Scala).
Databases: Experience with relational (SQL Server, PostgreSQL, MySQL) and NoSQL (MongoDB, Cosmos DB) systems.
Data Platforms: Hands-on experience with Azure Data Lake, Databricks.
ETL/ELT Tools: Azure Data Factory, Apache Airflow, or dbt.
Messaging & Streaming: Kafka, Event Hubs, or Service Bus for real-time data processing.
AI/ML Exposure: Familiarity with ML frameworks (TensorFlow, PyTorch) and MLOps concepts.
Visualization & Analytics: Power BI, Apache Superset, or Tableau.
Cloud & DevOps: Azure, Docker, Kubernetes, GitHub Actions/Azure DevOps.
Best Practices: Solid understanding of data modeling, version control, and CI/CD for data systems.

Preferred skills

Preferred Skills (Nice to Have):
Experience in knowledge graph or semantic search solutions.
Understanding of LLM-based data retrieval (RAG) patterns.
Exposure to data mesh, data fabric, or domain-oriented data architecture.
Familiarity with MLflow, Delta Live Tables, or Data Bricks Unity Catalog.

Soft skills

Strong analytical and problem-solving ability.
Excellent communication and collaboration skills.
Attention to detail and ability to work with large, complex datasets.
Creativity and ability to automate repetitive workflows.
Passion for continuous learning and innovation in data and AI technologies.

We’re on a Mission

In 2005, we disrupted the life sciences industry by introducing the world’s first digital validation lifecycle management system. ValGenesis VLMS® revolutionized compliance-based corporate validation activities and has remained the industry standard.

Today, we continue to push the boundaries of innovation ― enhancing and expanding our portfolio beyond validation with an end-to-end digital transformation platform. We combine our purpose-built systems with world-class consulting services to help every facet of GxP meet evolving regulations and quality expectations.

The Team You’ll Join

Our customers’ success is our success. We keep the customer experience centered in our decisions, from product to marketing to sales to services to support. Life sciences companies exist to improve humanity’s quality of life, and we honor that mission.

We work together. We communicate openly, support each other without reservation, and never hesitate to wear multiple hats to get the job done.

We think big. Innovation is the heart of ValGenesis. That spirit drives product development as well as personal growth. We never stop aiming upward.

We’re in it to win it. We’re on a path to becoming the number one intelligent validation platform in the market, and we won’t settle for anything less than being a market leader.

How We Work

Our Chennai, Hyderabad and Bangalore offices are onsite, 5 days per week. We believe that in-person interaction and collaboration fosters creativity, and a sense of community, and is critical to our future success as a company.

ValGenesis is an equal-opportunity employer that makes employment decisions on the basis of merit. Our goal is to have the best-qualified people in every job. All qualified applicants will receive consideration for employment without regard to race, religion, sex, sexual orientation, gender identity, national origin, disability, or any other characteristics protected by local law.

About the Company

ValGenesis

Cloud-based validation lifecycle management platform

ValGenesis runs a cloud-based Validation Lifecycle Management System (VLMS) for life sciences, turning validation work into a digital, paperless process. It works as a SaaS platform that guides users through planning, executing, reporting, and archiving validations, with AI-enabled modules like iVal, iClean, and iRisk, plus the option of VLMS Express for quick deployments. It differentiates itself with a fully end-to-end, paperless lifecycle and a growing suite of AI tools, supported by a large installed base that includes 30 of the top 50 global life sciences companies, plus ongoing expansion after acquiring 4Tune Engineering. Its goal is to cut validation costs and cycle times while preserving regulatory compliance and traceability.

Company size: 501-1,000
Company stage: Debt Financing
Total funding: $40.4M
Headquarters: Santa Clara, CA
Founded: 2005

Recently Posted Jobs

Explore more opportunities at ValGenesis

See more jobs