Data Engineer - Python/PySpark

Job Summary

Synechron is seeking a Data Engineer with 5+ years of experience building data marts, ETL pipelines, and large-scale data processing solutions. The role will develop reliable data products using Python, PySpark, SQL, and data warehousing technologies while supporting the full software development lifecycle.

Software Requirements

Required

• Python: 5+ years of development experience.
• PySpark/Spark: 3+ years of experience.
• Oracle SQL: 3+ years, including query optimization and data analysis.
• Pandas and Jupyter Notebook.
• ETL development and data warehousing.
• Hadoop, Hive, and MapReduce.
• SQL and NoSQL databases.
• Git, CI/CD tools, automated testing, and data validation.
• Experience supporting development, UAT, production deployment, and production issues.

Preferred

• Cloud-based data platforms.
• Workflow orchestration tools.
• Data quality and monitoring tools.
• Banking or financial services data experience.

Overall Responsibilities

• Design, develop, and maintain scalable ETL pipelines using Python and PySpark.
• Build and optimize data marts and transformation processes.
• Process structured, semi-structured, and unstructured data.
• Analyze and resolve data issues using Oracle SQL, PySpark, and data analysis techniques.
• Support development, testing, UAT, deployment, and production operations.
• Collaborate with engineering, QA, business, and operations teams.
• Apply coding standards, testing practices, and CI/CD processes.
• Improve data quality, pipeline performance, reliability, and maintainability.

Technical Skills (By Category)

Programming Languages

• Essential: Python, PySpark, Spark, SQL, and Pandas.
• Preferred: Scala or another data-processing language.

Databases and Data Management

• Essential: Oracle, SQL and NoSQL databases, data marts, data warehousing, ETL, data validation, and query optimization.
• Preferred: Data modeling, partitioning, indexing, metadata management, and data quality frameworks.

Cloud Technologies

• Essential: Understanding of distributed data environments and production data platforms.
• Preferred: Experience with cloud storage, compute, managed Spark services, and cloud data warehouses.

Frameworks and Libraries

• Essential: PySpark, Apache Spark, Pandas, Hadoop, Hive, MapReduce, and Jupyter Notebook.
• Preferred: Workflow orchestration, streaming, and data quality frameworks.

Development Tools and Methodologies

• Essential: Git, CI/CD, Agile delivery, code reviews, automated testing, debugging, deployment, and production support.
• Preferred: Infrastructure as code, monitoring, alerting, and automated release management.

Security Protocols

• Essential: Secure handling of data, access controls, secrets management, encryption, and data privacy practices.
• Preferred: Experience with audit controls and regulated data
• environments.

Experience Requirements

• 5+ years of experience in data engineering or software development.
• Proven experience developing ETL pipelines, data marts, and large-scale data processing solutions.
• Experience with Python, PySpark, Spark, SQL, Oracle, Hadoop, and Hive.
• Experience supporting the complete SDLC, including UAT, deployment, bug fixes, and production support.
• Banking or financial services experience is preferred.
• Equivalent experience in data development, analytics engineering, platform engineering, or production data support may be considered.

Day-to-Day Activities

• Develop and enhance Python and PySpark data pipelines.
• Build data marts and optimize transformations and SQL queries.
• Investigate data quality issues and pipeline failures.
• Review logs, job results, database records, and performance metrics.
• Create unit, integration, and data validation tests.
• Support UAT, deployments, releases, and production incidents.
• Participate in planning, requirements discussions, code reviews, defect triage, and retrospectives.
• Document data flows, technical decisions, support procedures, and known issues.
• Recommend improvements to pipeline design, performance, testing, and reliability.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Engineering, or a related field, or equivalent practical experience.
• Certifications in data engineering, cloud, database, or big-data technologies are preferred.
• Complete Synechron training on security, data protection, SDLC, and operational procedures.
• Maintain current knowledge of Python, Spark, data architecture, cloud platforms, data quality, and engineering practices.

Professional Competencies

• Structured problem-solving and data investigation.
• Ability to analyze complex data issues and identify root causes.
• Clear communication with technical and business stakeholders.
• Effective collaboration across development, QA, product, and operations teams.
• Ability to prioritize development, support, and production issues.
• Commitment to quality, documentation, testing, and maintainable code.
• Willingness to learn new data technologies and improve existing processes.

S​YNECHRON’S DIVERSITY & INCLUSION STATEMENT
 

Diversity & Inclusion are fundamental to our culture, and Synechron is proud to be an equal opportunity workplace and is an affirmative action employer. Our Diversity, Equity, and Inclusion (DEI) initiative ‘Same Difference’ is committed to fostering an inclusive culture – promoting equality, diversity and an environment that is respectful to all. We strongly believe that a diverse workforce helps build stronger, successful businesses as a global company. We encourage applicants from across diverse backgrounds, race, ethnicities, religion, age, marital status, gender, sexual orientations, or disabilities to apply. We empower our global workforce by offering flexible workplace arrangements, mentoring, internal mobility, learning and development programs, and more.

All employment decisions at Synechron are based on business needs, job requirements and individual qualifications, without regard to the applicant’s gender, gender identity, sexual orientation, race, ethnicity, disabled or veteran status, or any other characteristic protected by law.

Candidate Application Notice