Senior/Principal Local LLM & Generative AI Platform Engineer

What you will do:

• Own the architecture and technical roadmap for a secure, reliable, and maintainable local LLM platform deployed in Parallel Wireless-controlled infrastructure.

• Partner with engineering, product, support, IT, information security, legal, and domain experts to prioritize high-value use cases and translate them into measurable product and platform requirements.

• Build a modular inference and model-gateway layer with stable APIs, model routing, streaming, concurrency controls, quotas, and the ability to change models or serving backends without rewriting every application.

• Evaluate open-weight language, code, embedding, reranking, and, where useful, multimodal models against PW-specific tasks; document model provenance, licenses, limitations, security posture, hardware needs, and total cost of ownership.

• Optimize serving across available CPU, GPU, and accelerator resources using techniques such as continuous batching, caching, parallelism, quantization, and right-sized context limits while protecting output quality.

• Design and operate RAG and enterprise-search pipelines for approved repositories, wikis, tickets, standards, design documents, test results, logs, and support content, including parsing, chunking, metadata, embeddings, hybrid retrieval, reranking, freshness, citations, and deletion.

• Enforce source-system permissions throughout ingestion and retrieval so that the platform never exposes content a user is not authorized to access; integrate with company identity, SSO, role-based access control, secrets management, and audit logging.

• Establish versioned evaluation datasets and automated offline and online evaluation for retrieval quality, groundedness, factual accuracy, citation quality, code correctness, task completion, latency, safety, and refusal behavior.

• Create release gates and reproducible regression tests for changes to models, prompts, tools, embeddings, retrieval logic, indexes, and serving configurations; support canary releases, rollback, and clear approval paths.

• Implement end-to-end observability for model and agent workflows, including traces, errors, time to first token, inter-token latency, throughput, queue time, resource utilization, saturation, availability, and user feedback.

• Design safe tool-calling and agent workflows with least-privilege access, sandboxing, input and output validation, bounded execution, human approval for consequential actions, and complete traceability.

• Integrate the platform into the tools employees already use—such as developer environments, source-control and CI workflows, knowledge systems, ticketing systems, and internal applications—through reusable SDKs, APIs, and reference implementations.

• Build the operational foundations for production use: CI/CD, configuration and model registries, backups, disaster recovery, capacity planning, dependency and vulnerability management, incident response, and lifecycle policies for models and data.

• Protect proprietary and personal information through network isolation, encryption, retention controls, redaction where appropriate, secure logging, and defenses against prompt injection, data poisoning, unsafe output handling, and model-supply-chain risks.

• Determine when prompt or retrieval improvements are sufficient and when parameter-efficient fine-tuning, distillation, or other adaptation is justified by measured quality gains.

• Make the platform usable beyond the core AI team through documentation, examples, training, office hours, and hands-on collaboration; use telemetry and structured feedback to improve adoption and effectiveness.

• Communicate architecture decisions, quality evidence, risk, capacity, and roadmap tradeoffs clearly to technical and business stakeholders.

What you bring:

• BSc or MSc in Computer Science, Computer Engineering, Electrical Engineering, Data Science, or a related field, or equivalent practical experience.

• Typically 7+ years of hands-on experience in production software, ML platform, search, data, or infrastructure engineering, including meaningful recent experience shipping LLM-powered systems; exceptional candidates with equivalent depth are welcome.

• Strong Python engineering skills and experience designing maintainable APIs, services, libraries, and data pipelines. Experience with Go, Java, or C/C++ is an advantage.

• Strong understanding of transformer-based language models and production inference, including tokenization, context management, batching, KV caching, parallelism, quantization, structured output, tool calling, and common model failure modes.

• Demonstrated experience building production RAG or enterprise-search systems using embeddings, vector and/or lexical search, metadata filtering, reranking, source attribution, and systematic retrieval evaluation.

• Experience defining task-specific LLM evaluations using representative datasets, strong baselines, domain-expert review, automated metrics, human feedback, error analysis, and regression thresholds.

• Experience deploying and operating containerized services on Linux using Docker and Kubernetes or an equivalent orchestration environment.

• Practical experience with GPU-backed model serving, performance profiling, capacity planning, monitoring, and reliability engineering.

• Strong knowledge of distributed-system fundamentals, authentication and authorization, API security, secrets handling, encryption, auditability, and data lifecycle controls.

• Experience with Git, automated testing, CI/CD, infrastructure as code, observability, and production incident response.

• Sound technical judgment about quality, security, maintainability, hardware efficiency, and total cost—not just model benchmark scores.

• Ability to lead an ambiguous, cross-functional initiative, explain complex AI behavior in plain language, and help other teams ship safely on a shared platform.

Nice to have:

• Experience operating LLMs in on-premises, private-cloud, restricted-network, or air-gapped environments.

• Hands-on experience with current inference runtimes and serving systems such as vLLM, SGLang, TensorRT-LLM, llama.cpp, Ray Serve, KServe, Triton, or equivalent technologies.

• Experience optimizing inference on NVIDIA and/or AMD GPUs using CUDA, ROCm, profiling tools, tensor parallelism, pipeline parallelism, speculative decoding, prefix/KV caching, or related techniques.

• Experience with model and experiment registries, LLM tracing and evaluation platforms, vector databases, hybrid-search engines, and production data-orchestration frameworks.

• Experience with parameter-efficient fine-tuning methods such as LoRA/QLoRA, dataset curation, synthetic-data generation, distillation, and post-training evaluation.

• Experience building code intelligence, repository-aware assistants, developer tools, or IDE and CI integrations for large C/C++ and Python codebases.

• Familiarity with Active Directory or another enterprise identity provider, fine-grained document authorization, data-loss prevention, secure software supply chains, model licensing, and AI governance.

• Experience red-teaming LLM or agent systems for prompt injection, sensitive-data disclosure, poisoned retrieval content, excessive agency, and insecure output handling.

• Knowledge of telecommunications, 3GPP, RAN/Open RAN, cloud-native network functions, or technical-support workflows.

• Experience working across heterogeneous compute platforms and making performance, energy, and TCO tradeoffs for enterprise AI workloads.

• Contributions to relevant open-source AI, search, MLOps, or infrastructure projects.