S

Soft Tech Consulting

IT consulting delivering mission-critical solutions

AI Senior Software Engineer

AI Application EngineeringPosted on Nov 5
Full-timeWashington, DC, USA$170k - $190k /yr

About the Job

Requirements

  • A bachelor's degree is required.
  • At least 7–10 years of related experience is required.
  • Ability to iteratively tune, evaluate, and validate AI workflows, including refining prompts, agent configurations, tool-calling patterns, and retrieval strategies.
  • Experience designing and running structured tests for AI behavior, including regression tests, scenario-based evaluations, synthetic testing, and model comparisons.
  • Ability to diagnose and correct AI system failures, including hallucinations, retrieval issues, or tool-use errors, and implement guardrails or workflow adjustments.
  • Ability to document and maintain AI models, agent behaviors, experiments, and workflow designs.
  • Proficiency in data modeling and familiarity with AWS Bedrock services.
  • Proficiency in Python; familiarity with JavaScript or other programming languages is beneficial.
  • Comfort with CI/CD and Git-based development workflows.
  • Experience integrating observability tools into AI application workflows, including logging, metrics, and traces.
  • Advanced experience with AI agents and workflow integration.
  • Advanced experience with AI prompting.
  • Advanced Python software development skills.
  • Advanced experience with AWS AI services.
  • Advanced experience with AI architecture.
  • Effective verbal and written communication skills.
  • Ability to work independently with minimal oversight.
  • Ability to develop strong client relationships.
  • Must be a U.S. citizen and hold a Top Secret clearance.
  • Must work onsite in Washington, DC.

Responsibilities

  • Design, build, and optimize agentic AI systems using orchestration frameworks such as LangGraph, CrewAI, Google ADK, or AWS Bedrock AgentCore.
  • Implement multi-step, tool-enabled agentic workflows combining large language model reasoning, retrieval-augmented generation, retrieval systems, external APIs, and custom enterprise tools.
  • Use foundation models such as Claude Sonnet and Claude Opus on Amazon Bedrock to build structured generation, extraction, summarization, planning, and decision-support capabilities.
  • Tune and refine AI workflows through prompt engineering, instruction design, agent configuration, and tool-use optimization.
  • Conduct structured testing and evaluation of AI behavior, including regression tests, scenario-based evaluations, synthetic test creation, and model-to-model comparisons.
  • Diagnose hallucinations, retrieval mismatches, tool-use errors, and other unexpected agent behavior, and implement corrective strategies such as guardrails and workflow adjustments.
  • Integrate AI components with vector databases, graph databases, internal applications, and other infrastructure tools, coordinating with the engineers who manage underlying systems.
  • Document model usage patterns, agent behaviors, experimental findings, workflow designs, and best practices for reproducibility and maintainability.
  • Collaborate with cybersecurity and governance teams to ensure AI implementations comply with required policies and safe-use guidelines.

Desired Qualifications

  • A master's degree in Computer Science is preferred.
  • Familiarity with JavaScript or other programming languages is beneficial.

About the Company

Soft Tech Consulting

IT consulting delivering mission-critical solutions

Soft Tech Consulting provides IT consulting services for government and private sector organizations in the DC metro area. The company delivers enterprise-class IT solutions at good value and supports design, implementation, and ongoing maintenance of mission-critical systems. The team consists of experienced information technology professionals, and services focus on delivering continuous quality customer support to ensure successful outcomes for clients of all sizes. The firm emphasizes its status as woman- and minority-owned and aims to help government and other clients acquire effective, affordable IT solutions through a combination of skilled staff, disciplined project execution, and strong customer service. Overall goal: help customers design, implement, and maintain critical IT infrastructure while providing reliable support and value-based solutions.

Company size
51-200
Company stage
Not listed
Total funding
Not listed
Headquarters
Chantilly, VA
Founded
1996

Recently Posted Jobs

Explore more opportunities at Soft Tech Consulting

See more jobs