Software Engineer

We’re on a mission to democratize AI by building the definitive AI data development platform. The AI landscape has gone through incredible change between 2016, when Snorkel started as a research project in the Stanford AI Lab, to the generative AI breakthroughs of today. But one thing has remained constant: the data you use to build AI is the key to achieving differentiation, high performance, and production-ready systems. We work with some of the world’s largest organizations to empower scientists, engineers, financial experts, product creators, journalists, and more to build custom AI with their data faster than ever before. Excited to help us redefine how AI is built? Apply to be the newest Snorkeler!

As an AI Platform Engineer, you will be at the forefront of designing and implementing the core AI pipelines that drive SnorkelFlow’s AI capabilities. Whether you’re an entry-level engineer eager to grow in the field or a senior engineer ready to lead high-impact projects, this role offers opportunities to work on SOTA AI models and redefine the AI stack through a data-driven primary way to solve challenging problems.

Main Responsibilities

1. AI Pipeline Development:
• Contribute to designing and maintaining AI pipelines for labeling, embeddings, training, and deploying models into production.
• Collaborate with the Compute Platform team to ensure seamless integration with orchestration tools and infrastructure.
• Define and develop AI Platform APIs for other applications to use them
2. LLM Tooling Development:
• Build and deploy LLM-based systems for production workflows, focusing on efficiency, scalability, and reproducibility.
• Implement training pipelines for LLMs, including fine-tuning, pruning, distillation, and foundational model training.
• Integrate APIs from providers such as OpenAI, Anthropic, and Gemini into SnorkelFlow’s pipelines.
3. Backend Development:
• Develop and integrate backend services to manage LLM calls and API interactions.
• Work with the Data Platform team to define data requirements and ensure smooth interoperability.
• Partner with the Application team to design and implement APIs that power workflows.
4. Observability and Monitoring:
• Implement tools and dashboards to monitor and track AI pipeline performance.
• Define metrics to ensure system health and support optimization efforts.
5. Advanced Model Development:
• Innovate on advanced training methodologies for LLMs and NLP systems, including state-of-the-art optimization techniques.
• Take ownership of end-to-end model lifecycle management, from training to production deployment.

Required Qualifications

Entry-Level Engineers:

• Bachelor’s degree in Computer Science, Machine Learning, or a related field.
• 1-2 years of experience in AI development or backend engineering.
• Strong coding skills in Python and familiarity with deep learning frameworks like PyTorch or TensorFlow.
• Passion for AI development and willingness to learn advanced techniques.

Senior Engineers:

• Bachelor’s or Master’s degree in Computer Science, Machine Learning, or a related field.
• 4-6 years of experience in A and or backend development, including hands-on work with AI pipelines in production systems.
• Strong expertise in Python and deep learning frameworks such as PyTorch.
• Proven ability to lead projects and mentor team members.
• Solid understanding of LLM architectures, fine-tuning, and deployment.
• Solid understanding of agentic workflows and tooling (Langchain, Crew.ai etc)
• Experience building scalable CI/CD pipelines for machine learning workflows.

Preferred Qualifications

• Expertise in NLP and libraries such as Hugging Face Transformers, spaCy, or XGBoost.
• Familiarity with multimodal AI concepts, including vision and audio tasks.
• Knowledge of foundational model APIs such as OpenAI, Anthropic, or Gemini.
• Familiarity with MLOps tools and practices like MLflow, Kubernetes, or Ray.
• Experience building APIs or SDKs for AI services.

What We Offer

• A leadership role with the opportunity to influence the technical direction of SnorkelFlow.
• Competitive salary and benefits tailored to your experience.
• Hybrid work environment with 3 days per week at our Redwood City HQ and SF Office.
• "No Meeting" Tuesdays and Thursdays to focus on deep work.
• The chance to work on cutting-edge infrastructure and drive impactful change in an innovative, fast-paced environment.