S

SpyCloud

Threat intelligence and identity protection platform

Data Engineer 2

Data EngineeringConfirmed live in the last 24 hours
Full-timeAustin, TX, USA$104k - $135kHybrid

About the Job

Requirements

  • At least 5 years of professional experience in data engineering, software engineering, or a closely related discipline.
  • Experience building and operating Lakehouse, Databricks, or large-scale distributed data architectures in AWS.
  • Strong understanding of distributed systems as they relate to data storage, processing, and computing.
  • Experience building and managing Elasticsearch or OpenSearch data stores.
  • Hands-on experience designing, building, and maintaining ETL and data-processing pipelines.
  • Experience preparing and managing data for machine learning and LLM workflows.
  • Strong Python development skills, particularly for building data-intensive systems.
  • Practical experience with DynamoDB and other AWS-native data services.
  • Experience with infrastructure-as-code tools such as Terraform.
  • Experience troubleshooting complex data pipeline, performance, and data quality issues in production environments.
  • Ability to independently own complex technical projects from design and development through deployment and production support.
  • Ability to evaluate technical trade-offs and contribute effectively to architectural and design decisions.
  • Bachelor’s or Master’s degree in Computer Science, Engineering, or a related STEM field is preferred, not required.

Responsibilities

  • Design, build, and improve scalable data systems with a focus on performance and reliability.
  • Use Databricks and Spark to prepare, transform, and serve high-quality data to customers and downstream systems.
  • Build, optimize, and maintain real-time and batch data pipelines using AWS technologies such as Lambda, S3, DynamoDB, ECS, EKS, and Terraform, and develop and maintain pipelines within the Databricks ecosystem.
  • Identify and implement improvements to the cost and performance of data ingestion, transformation, and storage workflows.
  • Provide on-call support to troubleshoot and remediate data pipeline failures outside standard business hours.
  • Partner with Product Management, SpyCloud Labs, and Engineering teams to understand requirements and develop effective technical solutions.
  • Evaluate and implement modern tooling for data processing, ML/AI data workflows, and observability.
  • Contribute to architectural and technical design decisions and help implement scalable solutions aligned with engineering standards.
  • Promote engineering best practices through collaboration, code reviews, documentation, and knowledge sharing.
  • Identify and resolve complex data quality, performance, security, and reliability issues across production systems.
  • Own complex data engineering initiatives independently from design and development through deployment and production support.
  • Contribute to the continued evolution of SpyCloud’s data platform as product, customer, and AI requirements grow.

Desired Qualifications

  • Familiarity with AI/ML tools and technologies such as SageMaker, LangChain, or vector databases.
  • Understanding of data governance, privacy, and compliance considerations within AI and ML environments.
  • Experience with observability and monitoring tools for large-scale data platforms.
  • Experience with Go or C programming languages.
  • Experience working with cybersecurity or security-related datasets and technologies.
  • Comfort working in fast-paced environments with evolving product and technical requirements.

About the Company

SpyCloud

Threat intelligence and identity protection platform

SpyCloud provides cyber threat intelligence and identity protection for businesses. Its Cybercrime Analytics Platform uses data recovered from the criminal underground to deliver actionable insights that help detect and prevent cyber threats such as malware, ransomware, and unauthorized access. The platform also supports de-anonymizing threat actors and linking them to crimes to aid incident response and threat attribution. Unlike many security tools that focus on one area, SpyCloud combines threat intelligence, credential monitoring, dark web monitoring, and incident response in a single platform to help enterprises protect digital assets and identities. The company's goal is to help businesses proactively prevent data breaches and other cybercrimes by continuously detecting compromised credentials, stopping unauthorized access, and monitoring the dark web for emerging threats.

Company size
201-500
Company stage
Growth Equity (Venture Capital)
Total funding
$203.5M
Headquarters
Austin, TX
Founded
2016

Recently Posted Jobs

Explore more opportunities at SpyCloud

See more jobs