K

Kiss My Apps

Head of AI

Full-TimeUpdated on 10/23/2026
No salary listed
Expert
Kyiv, Ukraine+1 moreMore locations: Warsaw, Poland
In PersonFully remote from anywhere in the world; optional office access in Kyiv or Warsaw.
Software EngineeringAI Application Engineering

About the job

Requirements
  • At least 2–3 years of hands-on AI/LLM production experience and at least 1 year managing a technical team.
  • Experience launching AI/LLM systems into production end to end, from problem framing through impact measurement.
  • Knowledge of production LLM architecture, including gateways and routers, fallbacks, caching, rate limiting, observability, retrieval-augmented generation, and agents.
  • Experience with multiple LLM providers and understanding of trade-offs among APIs, managed inference, and self-hosted inference.
  • Experience building offline and online evaluations, LLM-as-a-judge, error analysis, failure taxonomies, and drift detection.
  • Understanding of data pipelines, personally identifiable information, data quality, and data lineage.
  • Knowledge of AI security, including prompt injection, jailbreaks, data exfiltration, permissions, secrets, and human-in-the-loop controls, as well as governance frameworks including the EU AI Act and GDPR.
  • Understanding of unit economics, token costs, cost efficiency, and automation impact as part of technical decision-making.
  • Ability to select model routing and build-versus-buy-versus-fine-tune approaches based on cost and quality.
  • Experience defining business metrics and baselines and measuring impact and return on marketing investment.
  • Hands-on approach and willingness to execute technical work independently until it can be delegated effectively.
  • Ability to influence without formal authority and collaborate with stakeholders outside direct reporting lines.
  • Executive communication, business judgment, and data-driven thinking.
  • Experience building a small technical team.
Responsibilities
  • Build KMA’s internal AI operating system as a production LLM architecture with a model gateway and router, fallbacks, caching, cost caps, agent orchestration, and a skill and agent registry.
  • Develop the existing MCP Gateway and address its gaps.
  • Set up offline and online evaluations, LLM-as-a-judge, model comparisons, and controls for evaluation bias.
  • Classify common AI errors and track quality degradation.
  • Configure tracing of LLM calls, including tokens, latency, cache hits, and routing, and connect traces to quality measures such as hallucinations, groundedness, and task success.
  • Govern model and agent actions and access; protect against prompt injection and data leakage; manage access levels; block unsafe permission combinations; protect personally identifiable information and secrets; and use human-in-the-loop controls for critical actions.
  • Automate processes in Operations, Human Resources, and Finance and increase the share of tasks performed by AI without developers, in partnership with the Vice President of Engineering.
  • Choose models and approaches for each task while balancing quality and cost, including model routing and build-versus-buy-versus-fine-tune decisions.
  • Measure return on marketing investment and impact on lifetime value using control groups, baselines, and an impact ledger.
  • Form and develop the AI Integration team.
  • Work across departments to identify opportunities for AI automation and turn them into changes to processes and ways of working.
Desired Qualifications
  • Experience with internal AI, agent, or developer platforms.
  • Experience with Cloudflare, Vercel, Databricks, n8n, Claude Code, or MCP.
  • Experience with consumer or subscription products or an app holding company.
  • Knowledge of lifetime value, retention, average revenue per user, and business unit economics in business-to-consumer products.

About the company

Company Size

N/A

Company Stage

N/A

Total Funding

N/A

Headquarters

N/A

Founded

N/A