Position Summary

The Effectual Data Engineer builds pipelines that are used to transport date from a data source to a data warehouse. These pipelines are crucial: they are what allow us to access and analyze an organizations data and use the insights to help them make decisions. Data pipelines transport and transform data according to established business rules or a line of exploratory analysis the business wants to undertake. As a Data Engineer, you will prepare and organize the data that organizations have built in their databases and other formats.  

A Glimpse into the Daily Routine of a Data Engineer

As a Data Engineer, specializing in Confluent Kafka, you will take on big data challenges in an agile way. You will build data pipelines, utilizing Confluent Kafka, that enables our clients and their vision. You will also build data models to deliver insightful analytics while ensuring the highest standard in data integrity. You are encouraged to think out of the box and play with the latest technologies while exploring their limits. Successful candidates will have strong technical capabilities, a can-do attitude, and are highly collaborative.

Essential Duties and Responsibilities

• Design, build and launch extremely efficient and reliable data pipelines, utilizing Confluent Kafka, to move data across several platforms including Data Lakes, Data Warehouses, and real-time systems.
• Develop, construct, test and maintain data architectures from the data architect.
• Analyze organic and raw data.
• Build data systems and pipelines.
• Build the infrastructure required for extraction, transformation, and loading of data from different data sources using SQL and AWS ‘big data’ technologies.
• Write scripts for data architects, data scientists, and data quality engineers.
• Data acquisition.
• Identify ways to improve data reliability, efficiency, and quality.
• Develop dataset processes.
• Prepare data for prescriptive and predictive modeling.
• Automate the data collection and analysis processes, data releasing and reporting tools.
• Build algorithms and prototypes.
• Develop analytical tools and programs.

Qualifications

• MUST HAVE hands-on experience with Confluent Kafka including both administration and development.
• Either Confluent Certified Administrator for Apache Kafka (CCAAK) or Confluent Certified Developer (CCDAK) for Apache Kafka certificates, however, both are preferred.
• Bachelor’s or master’s degree in computer science, Engineering or a related field.
• 5+ years of experience working as a Data Engineer in an AWS professional services or consulting environment
• Proficiency in programming languages such as Python, Java, or Scala, with expertise in data processing frameworks and libraries (e.g., Spark, Hadoop, SQL).
• In-depth knowledge of database systems (relational and NoSQL), data modeling, and data warehousing concepts.
• Strong knowledge of data architectures and data modeling and data infrastructure ecosystem.
• Experience with cloud-based data platforms and services (e.g., AWS, Azure) including familiarity with relevant tools (e.g., S3, Redshift, BigQuery, etc.).
• Proficiency in designing and implementing ETL processes and data integration workflows using tools like Apache Flink, Apache Airflow, Informatica, or Talend.
• Familiarity with data governance practices, data quality frameworks, and data security principles
• Work minimal direction and turn a clients want and need into working stories, epics which can be performed upon during a sprint.
• A firm understanding of the SDLC process.
• An understanding of object-oriented programming.
• The ability to communicate cross-functionally, derive requirements and architect shared datasets; ability to synthesize, simplify and explain complex.
• The ability to thrive in a dynamic environment. That means being flexible and willing to jump in and do whatever it takes to be successful.
• Ability to travel, with strong preference to mid-west time zone or east coast.

Nice-to-Have Skills and Experience

• Knowledge of batch and streaming data architectures.
• Product mindset to understand business needs and come up with scalable engineering solutions.
• AWS Certified Cloud Practitioner
• AWS Certified Data Analytics Specialty
• AWS Certified Machine Learning Specialty
• AWS Certified Database Specialty
• SnowPro Core Certification
• Databricks Certified Data Engineer Associate

Company Offered Benefits

Full-time employees are eligible to participate in our employee benefit programs:

• Medical, dental, and vision health insurances,
• Short term disability, long term disability and life insurances,
• 401k with Company match
• Paid time off (PTO) (120 hours PTO that accrue over one year)
• Paid time off for major holidays (14 days per year)
• These and any other employee benefit offerings are subject to management’s discretion and may change at any time. 

Physical Demands and Work Environment

The work is generally performed in an office environment.  Physical demands include sitting, keyboarding, verbal communication, written communication.  Employees are occasionally required to stand; walk; reach with hands and arms; climb or balance; and stoop, kneel, crouch, or crawl. The physical demands described here are representative of those that must be met by an employee to successfully perform the essential functions of this position. Reasonable accommodation may be made to enable individuals with disabilities to perform the functions.

Salary Range for this position: $100,000-$160,000

"Salary ranges provided are for informational purposes only and may vary depending on factors such as experience, qualifications, and geographic location. The final salary offer will be determined based on the candidate’s skills and alignment with the role requirements."

This job description may not be inclusive of all assigned duties, responsibilities, or aspects of the job described, and may be amended anytime at the sole discretion of the Employer. Duties and responsibilities are subject to possible modification to reasonably accommodate individuals with disabilities. To perform this job successfully, the incumbents will possess the skills, aptitudes, and abilities to perform each duty proficiently. This document does not create an employment contract, implied or otherwise, other than an “at will” relationship. Effectual Inc. is an EEO employer and does not discriminate on the basis of any protected classification in its hiring, promoting, or any other job-related opportunity.