Senior Data Engineer

Why This Role is Exciting

AuditBoard is seeking a highly skilled Senior Cloud Data Engineer to build the data foundation to power our product and our internal analytics capabilities. This role is pivotal for architecting, building, and optimizing our advanced cloud data platform, directly contributing to the product’s success by leveraging state-of-the-art data technologies. Your work will ensure our platform’s scalability, security, and efficiency, driving AuditBoard’s growth and our customers’ success.

Key Responsibilities

• Data Platform Development: Design and implement scalable data warehouses in Snowflake, applying advanced features like Time Travel, Zero-Copy Cloning, and Secure Views to optimize performance and security.
• Pipeline Engineering: Build robust, scalable data pipelines using a suite of modern ETL/ELT tools (e.g., Apache Airflow, Matillion) and real-time data streaming technologies (Apache Kafka, Apache Flink) to facilitate efficient data flows into Snowflake, supporting both product features and analytics.
• Cloud Infrastructure: Architect and manage cloud-based data infrastructure, employing Infrastructure as Code (IaC) practices with tools such as Terraform and AWS CloudFormation for resource management across AWS, Azure, or GCP environments.
• DevOps for Data: Implement CI/CD pipelines for data using Jenkins or GitLab CI, enhancing automation and reliability in data operations.
• Product Support: Work closely with product development teams to integrate data-driven features and capabilities into the AuditBoard platform, ensuring high performance and scalability.
• Security and Compliance: Ensure the highest standards of data security and compliance within the Snowflake ecosystem, leveraging Snowflake’s security model and adhering to relevant data protection regulations.
• Data Quality & Observability: Implement processes and systems to monitor data quality, ensuring production data is always accurate and available.

Attributes for a Successful Candidate

• Expertise in Snowflake: Deep understanding of Snowflake’s architecture, security features, and optimization techniques.
• Advanced Data Engineering: Proficiency in building data pipelines and warehouses, with a strong background in programming languages (Java, Scala, Python) and SQL.
• Technology Proficiency: Hands-on experience with Apache Kafka, Apache Flink, dbt, Airflow, and Terraform. Familiarity with Docker and Kubernetes for container orchestration.
• Cloud Platforms: Experience with cloud services (AWS, Azure, GCP), focusing on building and managing cloud-native applications.
• Analytical Tools Integration: Experience integrating BI tools (Tableau, Looker, Power BI) for product analytics and internal decision-making processes.
• Security and Compliance Knowledge: Understanding of data security practices, encryption, and compliance standards in a cloud environment.