Full-Time
Big Data Engineer
Posted on 4/18/2024
Specializes in financial services data engineering
Consulting
Financial Services
Mid
Tampa, FL, USA
Required Skills
Apache Spark
SQL
AWS
Data Analysis
Requirements
- 3+ years of Data Management experience in an enterprise environment.
- 2+ years of experience with SQL.
- 1+ years of experience with AWS S3, Spark and Glue.
- Knowledge of metadata management, data governance, and data lineage.
Responsibilities
- Data Architecture: Design and implement scalable, robust, and efficient data architectures, including data lakes, data warehouses, and data pipelines, to support the organization's data requirements.
- Data Pipeline Development: Develop and maintain ETL (Extract, Transform, Load) processes and data pipelines to ingest, process, and transform large volumes of data from various sources, ensuring data quality and reliability.
- Data Modeling: Design and implement data models and schemas to organize and structure data for efficient storage, retrieval, and analysis, considering business requirements and data governance principles.
- Data Integration: Integrate data from diverse sources, such as databases, APIs, files, and streaming platforms, ensuring seamless data flow and interoperability across systems.
- Data Transformation and Enrichment: Implement data transformation and enrichment processes to derive insights, perform aggregations, and create derived datasets for analytics, reporting, and machine learning.
- Data Governance and Compliance: Establish and enforce data governance policies, standards, and best practices to ensure data integrity, security, privacy, and compliance with regulatory requirements.
- Performance Optimization: Optimize data pipelines and queries for performance, scalability, and efficiency, leveraging techniques such as partitioning, indexing, and query optimization.
- Collaboration: Collaborate with cross-functional teams, including data scientists, analysts, software engineers, and business stakeholders, to understand data requirements and deliver data solutions that meet business needs.
Doran Jones Inc. is a highly regarded employer in the financial services sector, focusing on data engineering and application development. The company's emphasis on leveraging agile transformation expertise, AI, and machine learning tools offers a dynamic and technologically advanced working environment. This focus not only drives the digital transformation of its clients but also positions Doran Jones Inc. as a leader in developing solutions for capital markets, risk, and regulatory compliance.
Company Stage
M&A
Total Funding
N/A
Headquarters
Charlotte, North Carolina
Founded
2010
Growth & Insights
Headcount