Senior Software Engineer - AI Evaluation & Benchmarking

Job Description

This opportunity is ideal for senior software engineers with strong Python expertise who enjoy writing high-quality code, reviewing technical solutions, and working on AI-related projects.

What You'll Do:

• Design and develop coding benchmarks used to evaluate frontier AI models.
• Analyze AI-generated code for correctness, reliability, efficiency, and edge cases.Build and maintain scalable data pipelines that support AI evaluation workflows.
• Create structured programming scenarios to test reasoning, debugging, and code quality.
• Work with large codebases and multi-language software environments.
• Collaborate with teams focused on improving how AI models understand, generate, and evaluate software.
• Write clean, maintainable, and well-tested Python code following software engineering best practices.

Qualifications

Requirements:

• 4+ years of professional software engineering experience (required).
• Expert-level proficiency in Python.
• Experience working at a high-growth technology company or top-tier software organization.
• Proficiency in at least one additional programming language such as JavaScript, Go, C++, or similar.
• Experience with CI/CD pipelines and automated testing frameworks such as pytest, Mocha, or JUnit.
• Strong understanding of software engineering best practices, debugging, and code quality.
• Excellent analytical and problem-solving skills.

Nice to Haves:

• Experience with AI/ML evaluation, model benchmarking, or Generative AI.
• Background in security engineering.
• Significant contributions to open-source software projects.
• Experience working with large-scale distributed systems or enterprise software platforms.

Additional Information

• Fully remote contract opportunity.
• Compensation ranges from $80–$100 USD per hour.
• Expected workload is 10–39 hours per week, depending on project needs.
• Weekly payments for approved work completed during the previous week.
• Work volume may fluctuate throughout the duration of the engagement.

Hiring Process:

• Submit your proposal.
• Complete a short qualification form provided by the client.
• Qualified candidates will receive an Upwork contract offer and onboarding instructions.
• After accepting the contract, candidates will gain access to the client's evaluation.
• Candidates who successfully pass the evaluation will be invited to a technical interview with the client.