Software Development Engineer
Posted on 9/1/2023
Locations
Remote
Experience Level
Entry
Junior
Mid
Senior
Expert
Desired Skills
Data Analysis
Development Operations (DevOps)
Docker
Google Cloud Platform
Management
Pytorch
Natural Language Processing (NLP)
Kubernetes
Python
CategoriesNew
AI & Machine Learning
Software Engineering
Requirements
- Bachelor's or Master's degree in Computer Science, Software Engineering, or a related field
- 3+ years of software development industry experience
- Strong programming skills in Python
- Experience with cloud platforms and containerization technologies (e.g., GCP, Docker, Kubernetes)
- Excellent problem-solving skills and ability to thrive in a dynamic, fast-paced environment
- Strong communication and collaboration skills to work effectively in a team-oriented setting
Responsibilities
- Model Lifecycle Management: Design and implement systems to facilitate the end-to-end management of Large Language Models. This includes versioning, tracking changes, and ensuring proper documentation of model configurations and performance
- Model Fine-Tuning: Collaborate with NLP researchers and data scientists to implement fine-tuning pipelines that optimize model performance on specific tasks or domains. Enable efficient experimentation with different training data and hyperparameters
- Model Serving Infrastructure: Develop scalable and reliable infrastructure for serving LLMs in real-world applications. Ensure high availability, low latency, and effective resource allocation to handle production-level demands
- Application Integration: Design and build libraries and tools to help integrate LLMs into a variety of applications and services. Collaborate with application development teams to ensure smooth integration
- Continuous Integration and Deployment: Implement robust CI/CD pipelines for model deployment and updates. Automate testing and validation processes to maintain model accuracy and reliability in production environments
- Model Monitoring and Analytics: Develop monitoring solutions to track model performance, usage patterns, and potential issues. Utilize metrics and logs to identify opportunities for model improvement and optimize resource utilization
- Collaboration and Communication: Work closely with cross-functional teams, including NLP researchers, machine learning engineers, product managers, and DevOps, to align goals, exchange ideas, and deliver high-quality solutions
- Research and Innovation: Stay up-to-date with the latest advancements in NLP, model lifecycle management, and Large Language Models. Participate in research discussions and propose innovative approaches to improve model capabilities and management processes
Desired Qualifications
- Familiarity with frameworks like PyTorch, HuggingFace Transformers and DeepSpeed. Experience with large-scale model training and optimization
- Solid understanding of NLP concepts and model architectures
- Contributions to open-source NLP or machine learning projects
- Prior experience with state-of-the-art Large Language Models, and their lifecycle management