Data Engineer @ Shamrock Trading

INACTIVE

Full-Time

Data Engineer

Data Warehouse

Posted on 2/27/2024

Shamrock Trading

501-1,000 employees

Provides transportation, finance, and technology services

Fintech

Mid, Senior, Expert

Overland Park, KS, USA

Required Skills

Power BI

Redshift

Python

Airflow

Data Science

Data Structures & Algorithms

SQL

Postgres

AWS

Data Analysis

Requirements

Bachelor's degree in computer science, data science or related technical field, or equivalent practical experience
Experience extracting and manipulating data from a variety of sources, primarily cloud-based
Experience building and maintaining data pipelines and ETL/ELT jobs: currently utilizing Matillion, Databricks, and Airflow
Expertise in optimizing performance when loading into and extracting from relational and other data structures, including SQL tuning, indexing, etc.
5+ years of development experience with the following, and Subject Matter Expert (SME) with at least one: Strong SQL writing and tuning skills, Designing and deploying jobs within Matillion or similar GUI-based ETL/ELT tool, Expertise in designing, developing, deploying and/or supporting data pipelines using Databricks (pySpark, pySQL, DLT), Relational Databases (MS SQL, PostgreSQL, Redshift), Demonstrated analytical and problem-solving skills, Strong communication and listening skills, ability to work directly with customer teams (Business units and internal IT)
Understanding the frameworks and principles of traditional data warehousing
Experience with data processing and analytics within AWS eco-system
Experience data processing using Parquet and Avro files
Working with event based / streaming technologies to ingest and process data
Development experience delivering analytics processes using Python
Experience with front-end BI and data visualization tools (such as Power BI, Grafana, and QuickSight)

Responsibilities

Work with Architects to understand current data models, to build pipelines for data ingestion and transformation
Design and implement pipelines and/or ETL/ELT jobs in order to centralize data from internal and external sources, focusing on reliability and performance of jobs and ensuring the quality and consistency of data is maintained at all times
Follow framework and code patterns established by Architects and Data Engineers for reliable job execution
Gather, understand and fulfill technical requirements from other units and provide necessary updates throughout the project lifecycle

Description

Company Overview

Shamrock Trading Corporation is the parent company for a family of brands in transportation services, finance and technology. Headquartered in Overland Park, KS, Shamrock is frequently recognized among the “Best Places to Work” in Kansas City and Chicago and was most recently recognized as one of America’s top 100 “Most Loved Workplaces” by Newsweek. We also have offices in Atlanta, Chicago, Dallas, Ft. Lauderdale, Houston, Laredo, Nashville and Phoenix.

With an average annual revenue growth of 25% over several decades, Shamrock’s success is attributed to three key factors: hiring the best people, cultivating long-term relationships with our customers and continually evolving in the marketplace.

Responsibilities

Shamrock Trading Corporation is looking for a Data Engineer who wants to utilize their expertise in data warehousing, data pipeline creation/support and analytical skills by joining our Data Services team. This role is responsible for gathering and profiling data from many internal and external sources, building pipelines and designing a reliable cloud-focused data platform for analytics and business intelligence teams. This role requires significant understanding of data mining and analytical techniques. An ideal candidate will have strong technical capabilities, business acumen, and the ability to work effectively with cross-functional teams. Responsibilities include but are not limited to:

Work with Architects to understand current data models, to build pipelines for data ingestion and transformation
Design and implement pipelines and/or ETL/ELT jobs in order to centralize data from internal and external sources, focusing on reliability and performance of jobs and ensuring the quality and consistency of data is maintained at all times
Follow framework and code patterns established by Architects and Data Engineers for reliable job execution
Gather, understand and fulfill technical requirements from other units and provide necessary updates throughout the project lifecycle

Qualifications

Bachelor’s degree in computer science, data science or related technical field, or equivalent practical experience
Experience extracting and manipulating data from a variety of sources, primarily cloud-based
Experience building and maintaining data pipelines and ETL/ELT jobs: currently utilizing Matillion, Databricks, and Airflow
Expertise in optimizing performance when loading into and extracting from relational and other data structures, including SQL tuning, indexing, etc.
5+ years of development experience with the following, and Subject Matter Expert (SME) with at least one:

Strong SQL writing and tuning skills
Designing and deploying jobs within Matillion or similar GUI-based ETL/ELT tool
Expertise in designing, developing, deploying and/or supporting data pipelines using Databricks (pySpark, pySQL, DLT)
Relational Databases (MS SQL, PostgreSQL, Redshift)

Demonstrated analytical and problem-solving skills
Strong communication and listening skills, ability to work directly with customer teams (Business units and internal IT)

Preferred Qualifications

Understanding the frameworks and principles of traditional data warehousing
Experience with data processing and analytics within AWS eco-system
Experience data processing using Parquet and Avro files
Working with event based / streaming technologies to ingest and process data
Development experience delivering analytics processes using Python
Experience with front-end BI and data visualization tools (such as Power BI, Grafana, and QuickSight)

Benefits Package

At Shamrock we hire bright, ambitious people and give them the tools they need to be successful. By investing in training and development, we hope to become a long-term career for employees, where there are always opportunities for advancement. Shamrock also offers a premier set of benefits for employees and their families:

Medical: Fully paid healthcare, dental and vision premiums for employees and eligible dependents
Work-Life Balance: Competitive PTO and paid leave policies
Financial: Generous company 401(k) contributions and employee stock ownership after one year
Wellness: Onsite gym and discounted membership to select fitness centers. Jogging trails available at Overland Park offices

#LI-NB1 #LI-Hybrid

Shamrock Trading

View

Website

View Company Profile

Shamrock Trading Corporation offers a range of transportation, finance, and technology services through its family of brands, including trucking software and fuel/maintenance savings programs for trucking companies. The company focuses on freight brokerage, managed transportation, factoring, trade finance, and customized funding solutions for exporters.

Company Stage

N/A

Total Funding

N/A

Headquarters

Overland Park, Kansas

Founded

1986

Growth & Insights

Headcount

6 month growth

↑ 4%

1 year growth

↑ 16%

2 year growth

↑ 43%

Benefits

Medical and Insurance Benefits

401(k) and ESOP Retirement Plans

Work Life Balance Program

Wellness Program

Employee Assistance Program

Flexible Spending Accounts

Referral Bonuses

Local Discounts

INACTIVE