Why This Role is Exciting

As a Senior Engineer on the Data Platform team, you will play a crucial role in building and operating the data infrastructure that serves AuditBoard. Your responsibilities will include delivering high-quality results to ensure the data infrastructure’s security, compliance, optimization, and automation. You will contribute to our architecture while maintaining a high architectural standard.

Key Responsibilities

• Best practices and guardrails: Champion the best practices for using the data platform. Optimize for cost and performance
• Data architecture: Evaluate data platform architecture. Build and optimize data services to improve compliance, security, reliability, performance, and resource utilization. 
• Infrastructure automation: Increase reliability and employee productivity. Eliminate human errors through the standardization of processes and Infrastructure as Code (Iac)
• Data infrastructure monitoring: Monitor the platform’s health, reliability, cost, and usage.
• Operational support: Support the platform users for troubleshooting, configuration, version updates, and feature adoptions.
• Compliance: Build processes and tools to ensure that the data platform is compliant 
• Security: Identify infrastructure security-relevant actions, tests, and key performance indicators. Enhances the security posture of the data platform, ensuring the protection of data, systems, and assets.

Attributes for a Successful Candidate

• 8+ years of development experience with a focus on architecting cloud data platforms, setting up, managing, and automating the infrastructure of cloud data platforms, including Airflow, Databricks, Spark, and Kafka
• 8+ years of experience and proficient programming skills in Python or Java/C#. 3 years of programming experience in SQL.
• 7+ years of developer experience using Infrastructure as Code (IaC) technologies, such as AWS Cloud Formation and Terraform
• 5+ years of experience maintaining and improving OE of data platforms - service alerting, monitoring, and automation.
• 3+ years of experience with data compliance and governance. Demonstration of collaboration with the internal compliance team and external auditors in planning and execution of all phases of compliance including risk assessment & scoping. Standardize and automate the compliance process.
• B.S., M.S., or Ph.D. in Computer Science or equivalent

Nice to Have

• Subject matter expert (SME) in architecting cloud data platforms using Databricks, Snowflake, AWS Lake Formation, or other relevant technologies.
• SME in big data technologies including Spark, Presto, Airflow, Kafka
• Work in a startup-like environment building agile products and services
• Create clear and concise documentation, including diagrams, service descriptions, decisions, and runbooks.
• Deep knowledge of cloud platforms such as AWS, GCP, Azure