Full-Time

Data Engineer 3

Posted on 8/29/2025

PDI Technologies

PDI Technologies

1,001-5,000 employees

Retail payment, loyalty, shopper insights

No salary listed

Chennai, Tamil Nadu, India

In Person

Category
Data & Analytics (1)
Required Skills
Agile
Redshift
Python
Airflow
Data Science
R
Apache Spark
SQL
Java
AWS
Hadoop
Data Analysis
Requirements
  • At least 7 years’ experience in data engineering, Data modeling, SQL, ETL, Big database, Data warehousing & basic-level administration of data infrastructure
  • At least 7 years coding experience in Java, Python, R or other equivalent programming language
  • At least 4 years big data experience
  • At least 2 years’ experience as technical lead architecting data lake/data warehouse/lake house systems & mentoring team of Data Engineers / DBAs
  • Proficient in a variety of big-data tools and technologies, including Hadoop, Spark, etc., handling different storage formats like Parquet, Delta/Apache Hudi/Apache Iceberg, & managing/working with different OLTP systems & data ingestion mechanisms.
  • Experience building large scale data lakes & warehouse systems leveraging cloud environments including AWS technologies –S3, Redshift, Aurora/RDS, DMS, EC2, Glue spark, Cloud watch, & EMR
  • Experience with data orchestration tools like Airflow
  • Experience in building pipelines, ingesting data from heterogeneous systems onto data lake environments in real time
  • Proficient in Python, SQL and PL-SQL, query tuning, optimization, ETL, ELT, and Data Warehousing
  • Proficient in Business Intelligence (BI), analytic database and products, able to take business requirements and translate into database tables, pipeline design and tasks.
  • General data-specific concepts, frameworks and processes
  • Agile development practices
  • Working within an SDLC
  • Designing data warehouses including definition, structure, documentation, maintenance, long-range requirements, operational guidelines, and protection
  • Data governance, meta data management, data security measures, retention & archival across data lake layers
Responsibilities
  • Utilize architectural and design skills for a variety of data-oriented tasks.
  • Establish and maintain strong relationships with business stakeholders.
  • Build highly reliable & scalable data systems with good data governance that provide complete, secure, accurate and timely data for analytics.
  • Expertise in data & dimensional modeling with good business acumen in supporting new product features and services - including financial reporting.
  • Assist development and analytics organization to meet its delivery goals with regards to data design, flow and infrastructure.
  • Perform Data Modeling, Big data architecture design, support and implementation of OLTP, real time OLAP/data warehouse systems and ETL pipelines.
  • Track/Build & monitor data/system KPI’s like storage, performance & usage
  • Hands on administration of big data databases/compute engines (Redshift, EMR, Spark or similar) and optimization/tuning techniques
  • Prepare required documentation as outlined by departmental standards.
  • Identify, analyze and back track data problems as needed.
  • Provide recommendations for application and system improvements.
  • Mentor other team members
  • Work with business users to understand business requirements, issues and business and/or client processes.
  • Develop, test, and maintain high-performance of our data systems to meet the requirements of the business and/clients while adhering to departmental standards.
  • Perform quality assurance testing for all work performed.
  • Meet with agile teams as required to define and document application requirements.
  • Follow project development, deployment process & maintain industry standards and best coding practices.
  • Maintain security and organization of the company’s data.
  • Provide off-hour support as assigned.
  • Gather business requirements/features, estimate the tasks, design & develop scalable solutions.
  • Work with manager to ascertain the company’s data requirements.
  • Plan work to meet project deadlines, accommodate demands by development teams, set priorities and escalate issues appropriately.
  • Provide recommendations to development teams for data restructuring and performing complex maintenance.
  • Deploy and manage changes in development, staging and production.
  • Assist development and dev ops teams on SQL queries and tuning.
Desired Qualifications
  • Cloud experience, such as AWS, GCP or Azure
  • Building/Managing data lake & lake house architectures.
  • Experience working on different data lake storage formats.
  • Excellent oral and written communication
  • Multi-tasking and managing multiple priorities.
  • Working well in a team-oriented, collaborative environment with people from different disciplines and varying degrees of technical experience
  • Working in an Agile team environment.

PDI Technologies provides payment solutions, loyalty programs, and shopper insights tailored for retail and convenience sectors, including petroleum wholesale. It offers software and related services that help businesses process payments, run loyalty initiatives, and analyze shopper data to improve engagement. The product suite works by integrating payment processing with loyalty and data analytics, enabling clients to manage customer programs, track transactions, and extract insights while ensuring data security. The company differentiates itself through deep industry specialization, more than four decades of experience, and a large, diverse client base (over 1,500 companies) in retail, convenience, and petroleum wholesale, which enables more precise, sector-specific solutions. Its goal is to help clients streamline operations, boost profitability, and grow customer loyalty by leveraging data-driven decisions and secure, integrated technology.

Company Size

1,001-5,000

Company Stage

N/A

Total Funding

N/A

Headquarters

Alpharetta, Georgia

Founded

1983

Simplify Jobs

Simplify's Take

What believers are saying

  • PDIQ beta with 400 AI engineers drives operational gains by 2026.
  • $750M revenue in 2025 supports 200,000 global locations expansion.
  • EBizCharge integration streamlines credit card processing in PDI Enterprise.

What critics are saying

  • PDIQ hallucinations cause incorrect payments, losses in 6-12 months.
  • ValidiFI failure exposes PDI to fraud across retailers in 12-18 months.
  • P97 integration loss triggers CITGO, ExxonMobil migration in 12-18 months.

What makes PDI Technologies unique

  • PDIQ embeds AI into ERP, POS, and loyalty for convenience retail.
  • Acquired P97 Networks enhances mobile commerce for CITGO, ExxonMobil, Shell.
  • ValidiFI integration enables real-time pay-by-bank enrollments via GasBuddy.

Help us improve and share your feedback! Did you find this helpful?

Benefits

Flexible Work Hours

Unlimited Paid Time Off

Professional Development Budget

Company News

The Associated Press
Mar 26th, 2026
PDI Technologies launches PDIQ, AI foundation for convenience retail and fuel operations

PDI Technologies has launched PDIQ, an embedded AI foundation for the convenience retail and petroleum wholesale sectors. The platform integrates AI directly into PDI's existing solutions, including ERP, POS and loyalty systems, to automate tasks and deliver personalised insights. Before external release, PDI deployed PDIQ internally across engineering, customer support and sales functions. The company has over 400 AI-certified engineers and 600 intelligent agents in active use. PDIQ maintains human oversight, isolates customer data to permitted use cases, and ensures full auditability. The platform is currently in beta with select customers. A hands-on lab will debut at Connections Live 2026 in August. PDI operates at the intersection of convenience retail and petroleum wholesale, providing ERP, logistics, loyalty and payment solutions globally.

GlobeNewswire
Mar 26th, 2026
Introducing PDIQ: the AI foundation purpose-built for the convenience and fuel ecosystem.

Introducing PDIQ: the AI foundation purpose-built for the convenience and fuel ecosystem. Embedded across PDI solutions, PDIQ brings AI into industry workflows to help teams make quicker decisions, reduce manual processes, and act with confidence. ATLANTA, GEORGIA, March 26, 2026 (GLOBE NEWSWIRE) - PDI Technologies today announced PDIQ, the company's embedded AI foundation that provides the building blocks for smarter work, better insights, and faster action across the PDI ecosystem. PDIQ will bring intelligence directly into the PDI solutions customers already rely on by embedding AI inside workflows across its portfolio, including ERP, POS, loyalty, and other business solutions and services that serve convenience retail, petroleum wholesale, and adjacent markets. PDIQ automates routine tasks, coordinates workflows across customer solutions with ready-made connectors, and delivers personalized insights that help teams grow their business. In addition, PDIQ leverages existing data and systems of record to power intelligent agents that can plan, recommend, and take action - with users always in control. Before releasing PDIQ to customers for beta testing, PDI has deployed the technology across its own operations, with demonstrated use cases spanning engineering, customer support, sales, and other corporate functions. PDI is now introducing PDIQ to customers through a phased external rollout designed to deliver value quickly and safely. "Operators want to modernize, but they demand practical performance and clear governance from AI they can trust," said Jimmy Frangis, Chief Executive Officer, PDI Technologies. "With PDIQ, we're bringing intelligence into the core workflows that run this industry and helping customers unlock tangible operational gains." PDIQ at a glance * Built in, not bolted on: PDIQ will appear inside the PDI applications customers currently rely on, with the MyPDI portal providing access to insights and the PDIQ Agentic Studio for turnkey, co-built, or custom AI agents. * Proven performance: PDIQ has already delivered meaningful results in live deployments and PDI will expand those capabilities to additional customer environments. * Operational governance: PDIQ is designed to maintain human-in-the-loop oversight, keep customer data isolated to permitted use cases, and ensure full end-to-end auditability. * Connected scalability: PDIQ will support a flexible multi-model architecture as foundational AI capabilities evolve, giving customers confidence that their systems will grow with the technology. * Human empowerment: PDIQ will enhance the work people do by automating routine steps while keeping users fully in control, freeing up teams to focus on higher-value tasks. "PDIQ builds on decades of industry experience, and we've partnered with leading infrastructure and model providers to assemble what we believe is the strongest AI foundation in our end markets," said Linnea Geiss, Chief Operations Officer, PDI Technologies. "With more than 400 AI-certified engineers internally, 600 intelligent agents in active use, and a flexible skills marketplace, we're designing PDIQ for complex industries where reliability and execution matter most. Bringing this experience natively into our product suite gives customers the most natural, low-friction way to begin benefiting from AI today." PDIQ is now in its beta phase with a select group of initial customer adopters. A hands-on lab will debut at Connections Live 2026 in August, allowing conference attendees to explore how PDIQ can transform their operations. "The technology providers that are expected to lead are those that embed AI into core operations with strong governance and measurable ROI," said Deven Parekh, Managing Director at global software investor Insight Partners and PDI Board Member. "PDIQ embodies this - responsible, scalable, and purpose-built for the convenience and fuel industry." Customers can stay connected with the latest PDIQ updates by signing up online, ensuring they're among the first to see new capabilities as they roll out through 2026 and beyond. Customers can also pre-register now for Connections Live 2026 from August 23-26 at the Gaylord Texan Resort & Convention Center in Grapevine, Texas, where PDIQ will be demonstrated live. About PDI Technologies PDI Technologies, Inc. resides at the intersection of productivity and profitable growth, delivering solutions that serve as the backbone of the convenience retail and petroleum wholesale ecosystem. By "Connecting Convenience" across the globe, PDI helps businesses increase productivity and engage customers faster. Now with PDIQ, PDI is bringing embedded AI into the workflows customers already trust and helping operators work smarter, move faster, and make better decisions. From large-scale ERP and logistics to loyalty, payments, and cybersecurity, PDI simplifies the industry supply chain and delivers the intelligence needed to compete and win in a rapidly changing world. Visit the PDI Technologies website.

GlobeNewswire
Apr 1st, 2025
PDI Technologies Acquires P97 Networks

PDI Technologies has acquired P97 Networks, a mobile commerce and digital marketing platform provider. P97's platform, used by major fuel brands like CITGO, ExxonMobil, and Shell, enhances digital engagement in the convenience retail and fuel markets. PDI plans to expand P97's capabilities, including payment solutions for connected devices. This acquisition follows PDI's recent acquisitions of Nuspire and Comdata Merchant Solutions, further growing its market reach and technology suite.

Morningstar
Mar 6th, 2025
PDI Technologies Presents "The Retail Marketeers European Female Leaders in Convenience Awards"

PDI Technologies presents "The Retail Marketeers European Female Leaders in Convenience Awards"

PYMNTS
Jan 28th, 2025
Bank Account Intelligence: The New Frontier In Credit Decisioning

A bank account represents a direct connection to a business or consumer’s daily financial life. It holds all manner of data points that give lenders and payment service providers the information they need to improve their decision making, ward off would-be fraudsters and inform credit underwriting. Call it “bank account intelligence.” As ValidiFI CEO John Gordon told Karen Webster in an interview, the granular details that give insight into an account’s behavior and payment performance are critical in improving the financial ecosystem at large as enterprises are better informed about attractive customer behavior that speeds approvals — and gets to more positive outcomes in the process

INACTIVE