N

NCS

ICT consulting, digital transformation, cybersecurity services

AI/LLM Engineer

Natural Language Processing & LLMsConfirmed live in the last 24 hours
Full-timeSingaporeNo salary listedIn-person

About the Job

Requirements

  • At least 3 years of hands-on experience with large language models across prompt engineering, model integration, and evaluation.
  • Practical experience designing and optimizing prompts and prompt chains for production applications, and integrating models via APIs or gateways.
  • Strong understanding of evaluation methodologies, including accuracy, hallucination, and toxicity metrics, human-in-the-loop evaluation, and A/B testing.
  • Hands-on Python scripting ability to build and automate evaluation harnesses and integration and testing pipelines.
  • Statistical literacy to design representative test and benchmark sets and rigorously interpret results.
  • Clear, structured written communication to translate evaluation results and model recommendations into defensible reports for engineering and client audiences.
  • Working knowledge of the China AI model and technology stack, including deployment patterns, licensing, and self-hosting requirements.

Responsibilities

  • Design, test, and optimize prompts and prompt chains for production use cases while balancing accuracy, latency, and cost.
  • Integrate foundation models into applications through APIs and gateways, and advise AI Architects on model and version selection for use cases.
  • Support light fine-tuning and instruction-tuning work, including LoRA/PEFT and similar techniques, in partnership with AI Engineers.
  • Design and maintain evaluation harnesses, golden datasets, and benchmark suites measuring large language model and agentic-system accuracy, consistency, and safety.
  • Run comparative model benchmarks for accuracy, latency, and cost per query to provide technical evidence for model-selection decisions led by AI and Solution Architects.
  • Build repeatable automated regression suites for prompt, model, or pipeline changes and connect them to continuous integration.
  • Red-team AI systems through adversarial prompting, edge-case testing, and jailbreak testing to identify failure modes.
  • Detect and quantify hallucination, bias, and drift in production and pre-production systems, and produce defensible metrics.
  • Provide evaluation evidence to the Production Readiness Review Evaluation and Quality pillar, supporting engagement teams at the Scale gate.
  • Maintain evaluation and prompt-version documentation and reporting standards aligned with client compliance and audit requirements, including government AI governance requirements.
  • During Forward Deployed Engineering engagements, rapidly prototype prompts and model integrations and establish lightweight evaluation harnesses to compare candidate models and approaches during proof-of-concept and proof-of-value work.
  • During system development and maintenance engagements, own ongoing prompt and model tuning and run continuous evaluation and regression monitoring on live production systems, flagging quality degradation over time.
  • Contribute reusable prompt libraries, evaluation templates, and benchmark datasets to the shared internal asset library.
  • Provide technical benchmark evidence and integration recommendations to AI Architects and Solution Architects, who own final client-facing model recommendations and proposals.
  • Partner with AI Engineers and Testers to distinguish functional quality assurance from output-quality evaluation and hand off tuned prompts and models into production builds.

Desired Qualifications

  • Hands-on fine-tuning or instruction-tuning experience, such as LoRA/PEFT, on open-weight models.
  • Experience with large language model evaluation tools such as RAGAS, DeepEval, TruLens, or promptfoo, or with building custom evaluation frameworks.
  • Exposure to red-teaming and adversarial-testing practices for generative AI systems.
  • Familiarity with regulated-sector AI governance expectations in healthcare, government, or financial services.
  • Prior experience supporting client-facing presales or solutioning conversations with technical evidence, without owning the proposal.
  • Hands-on benchmarking or integration experience with Chinese open-weight models such as DeepSeek, Qwen, or GLM alongside Western models.

About the Company

NCS

ICT consulting, digital transformation, cybersecurity services

NCS delivers ICT services in the Asia Pacific region, including consulting, digital transformation, technology solutions, and cybersecurity for government and private sector clients. It offers advisory, implementation, and managed services, supported by three innovation centers in Singapore, Melbourne, and Shenzhen and a workforce of about 13,000 across 20 cities. The company emphasizes long-term client partnerships, regional scale, and practical solutions developed at its innovation centers. Its goal is to help clients navigate digital transformation and cybersecurity while maintaining ethical conduct and transparent governance.

Company size
10,001+
Company stage
Grant
Total funding
$780K
Headquarters
Singapore, Singapore
Founded
1981

Recently Posted Jobs

Explore more opportunities at NCS

See more jobs