Full-Time
Senior Machine Learning Engineer
Voice AI
Together AI
201-500 employees
Open-source AI research via decentralized cloud
Compensation Overview
$200k - $260k/yr
San Francisco, CA, USA
In Person
Bachelor's, Master's
 AI & Machine Learning (1) |
|---|
Get referred to Together AI
See people who can refer or advise you
- 5+ years of experience in ML engineering, with a focus on model serving, inference optimization, or ML infrastructure
- Hands-on experience with LLM serving engines (vLLM, SGLang, TensorRT-LLM, or similar) — comfortable reading and modifying engine internals, not just using APIs
- Strong proficiency in Python and PyTorch; experience with GPU profiling and optimization (CUDA, memory management, kernel-level debugging)
- Track record of shipping ML systems to production with measurable performance improvements
- Strong product sense — you think about what developers building voice apps actually need, not just what's technically interesting
- Comfort working on a small, early-stage team where you'll wear multiple hats and move fast
- Bachelor's or Master's degree in Computer Science, Electrical Engineering, or related field, or equivalent practical experience
- Experience with speech and audio ML (ASR, TTS architectures, audio signal processing) is a strong plus but not required — you can learn this quickly if you have strong ML engineering fundamentals
- Familiarity with audio codecs and tokenization schemes (SNAC, Encodec, DAC) is a plus
- Experience training or fine-tuning speech models is a plus
- Optimize inference performance for voice models (STT, TTS, speech-to-speech) — targeting best-in-class time-to-first-byte, throughput, and GPU utilization across our curated model set
- Productionize voice models on serverless and dedicated endpoints, including batching strategies, streaming inference, and memory management tailored to audio workloads
- Build and maintain a voice model evaluation framework — measuring word error rate across accents, languages, and noise conditions for STT; naturalness, latency, and pronunciation accuracy for TTS
- Enable new model architectures in our serving stack as the field evolves, including audio-native large language models, codec-based models (SNAC), and speech-to-speech systems
- Collaborate with model partners to integrate and optimize their models (Cartesia, Deepgram, Rime, and others) running on Together's infrastructure
- Profile and debug performance across the full inference stack — from GPU kernels to framework-level bottlenecks — and ship measurable improvements
- Work with the platform engineering side of the team to ensure the serving layer meets the latency and reliability requirements of real-time voice APIs
- Contribute to voice model fine-tuning capabilities (STT and TTS) as we enable customers to build differentiated voice experiences on Together
- Lay the groundwork for multiple new products down the line
- Experience with speech and audio ML (ASR, TTS architectures, audio signal processing) is a strong plus but not required — you can learn this quickly if you have strong ML engineering fundamentals
- Familiarity with audio codecs and tokenization schemes (SNAC, Encodec, DAC) is a plus
- Experience training or fine-tuning speech models is a plus
Together AI provides open-source AI tools and decentralized cloud services to train, fine-tune, and deploy generative models for researchers, developers, and organizations. It runs tasks in the cloud where users run training jobs, manage model versions, and deploy applications via subscriptions and usage fees. It differentiates itself by prioritizing open-source, transparency, and a decentralized cloud approach instead of a proprietary stack. Its goal is to broaden access to powerful AI and build open, verifiable AI systems that benefit society through shared technology.
Company Size
201-500
Company Stage
Series C
Total Funding
$1.3B
Headquarters
Menlo Park, California
Founded
2022
Get referred to Together AI
See people who can refer or advise you
Simplify's Take
What believers are saying
- Reuters reported July 1, 2026: $800 million raised at an $8.3 billion valuation.
- Together AI said annual bookings exceeded $1.15 billion and compute commitments topped 500 megawatts.
- July 26, 2026 Kimi K3 day-zero hosting expanded demand for Together AI's inference stack.
What critics are saying
- Together Computer faces a class action complaint filed May 29, 2026, over graphics-content contracts.
- OpenRouter, TensorWave, and Upscale AI are all chasing the same open-inference budget.
- A 500-megawatt buildout raises execution risk; GPU shortages or pricing wars crush margins.
What makes Together AI unique
- Together AI hosts 200-plus open-weight models through an OpenAI-compatible API.
- ATLAS speculative decoding reached 500 tokens per second on DeepSeek-V3.1 and Kimi-K2.
- FlashAttention-4 and Blackwell kernels give Together AI a real infrastructure-performance edge.
Help us improve and share your feedback! Did you find this helpful?
Benefits
Health Insurance
Company Equity
Growth & Insights and Company News
6 month growth
↑ 0%1 year growth
↓ -2%2 year growth
↑ 0%Together AI unveils advanced autoscaling for LLM inference. Together AI introduces autoscaling features tailored for large language models, optimizing GPU use and managing latency during traffic spikes. (Read More)
Kimi K3 open weights: when self-hosting a frontier model actually makes sense. webhani · 2026-07-28 On July 26, 2026, Moonshot AI released Kimi K3 with full open weights - a day ahead of its announced July 27 target. At 2.8 trillion parameters with a 1,048,576-token (1 million) context window, it is now the largest open-weight model publicly available. The weight download is approximately 1.4TB using MXFP4 quantization. Within hours, Together AI and Modal announced day-zero hosted inference access. This is not just a release milestone; it signals a real shift in how teams evaluate where to run frontier LLM workloads. For years, the architecture decision was simpler: if you needed state-of-the-art reasoning, you used a closed API (OpenAI, Anthropic, etc.). If you self-hosted, you accepted a meaningful capability trade-off for control and data residency. Kimi K3 breaks that binary. It is a frontier model with capability metrics that compete with the best closed offerings, and it is available as downloadable weights. That opens a different decision tree for teams - especially in regulated industries, in Japan and Asia more broadly, and anywhere data sovereignty or latency isolation matter. But "it's available as open weights" does not mean "you should run it yourself." The distance between "weights available" and "running in production at scale" is measured in infrastructure complexity, operational burden, and total cost. This post walks through that distance. Moonshot has published technical claims about Kimi K3's design. Two aspects are worth isolating: Kimi Delta Attention is described as delivering up to 6.3x faster decoding at 1M token context compared to standard attention. For teams running inference workloads where latency is a hard constraint - real-time chat, live code completion, interactive search - this is material. However, this is Moonshot's own benchmark claim; it is not independently verified by third parties yet. Treat it as a signal to evaluate, not as a guaranteed specification. Attention Residuals reportedly improve training efficiency by approximately 25% at less than 2% additional computational cost. This matters if your team is fine-tuning Kimi K3 for a specialized domain. For inference-only deployment, it is less directly relevant, though the engineering rigor it signals might correlate with inference stability. The larger point is this: a 2.8T-parameter model at 1M context is not a lightweight undertaking. Running this in production requires: * Multi-GPU / multi-node infrastructure: A single GPU cannot hold the model. You need tensor parallelism or pipeline parallelism across multiple cards, often across multiple machines. This is not a "run it on a beefy server" problem; it is a distributed-systems problem. * Quantization trade-offs: The 1.4TB figure assumes MXFP4 quantization. Every quantization step trades some inference quality for memory footprint and speed. You must evaluate whether the quality loss affects your use case. * Ongoing operations: Load balancing, fault recovery, scaling to handle traffic spikes, monitoring for model drift or inference anomalies - these are not one-time setup tasks. They are continuous responsibilities. For comparison: a well-managed Llama 3.1 8B deployment (a decade older, far smaller) still requires careful infrastructure work. Kimi K3 is orders of magnitude larger. webhani inc. advise clients through a three-axis evaluation. Here is a simplified version you can adapt: This is illustrative, not prescriptive. The logic is: self-hosting Kimi K3 only makes sense if (1) you have the ops maturity to run a distributed system, (2) you actually save money after factoring in infrastructure and labor, and (3) you have a genuine constraint - data residency, latency isolation, or repeated fine-tuning - that API access cannot meet. If the cost analysis or data-residency requirement points toward self-hosting, walk through these before committing: 1. Do you have people who can build and maintain a distributed inference cluster? This is not a DevOps hire; it is a deep-ML-infrastructure hire. Autoscaling, fault recovery, load balancing across model shards - these require someone who has shipped this before. If you do not have that person, add 6-12 months and considerable cost to the timeline. 2. Have you quantified the quality drop from MXFP4 quantization? Run your critical workloads (e.g., code generation, summarization, retrieval-augmented generation) against both the full-precision and quantized versions. Measure the difference in your own metrics - not benchmark scores, but whether the output actually works for you. If quality drops 15%, and your application is latency-tolerant, that trade-off might be fine. If quality drops 40% and your application is mission-critical, it is not. 3. What is your fallback if the self-hosted cluster has a cascading failure? Stateless inference workloads can survive a node failure if you have redundancy and a good load balancer. But a 2.8T-parameter model split across 4 GPUs is not easily "redundant" - you cannot just add another replica with the flip of a switch. You need a pre-planned runbook. Often, the runbook is "fail over to a hosted API for 48 hours while we rebuild" - which means you need a contract with a hosted provider as a backup. 4. Is your codebase and workflow stack actually designed for a modular model provider? If your application is hardcoded to use Claude or OpenAI, swapping to Kimi K3 means refactoring your LLM integration layer. That is not a small task; it is architecture work. Do not underestimate it. 5. Have you tested the fine-tuning workflow if you plan to do it? Kimi K3 supports fine-tuning. The process is not the same as fine-tuning Llama 3.1. You need to work through Moonshot's fine-tuning infrastructure, validate that the resulting weights are compatible with your inference setup, and measure quality on your own data. This is a 4-8 week effort, not a weekend project. Most teams should start with hosted Kimi K3 API access - via Together AI, Modal, or directly via Kimi's API - unless they have very specific constraints: * Data residency: If your data cannot leave a specific geography or jurisdiction, self-hosting may be required. But verify that first; many hosted providers now offer regional deployments. * **Latency: **If your application requires sub-50ms end-to-end latency and network round trips to a remote API kill it, self-hosting in your own data center is justified. But quantify this carefully; most applications tolerate 200ms latency without users noticing. * Cost at massive scale: If you are processing billions of tokens per month, the math shifts. At that scale, infrastructure cost amortizes and self-hosting becomes cheaper. But at that scale, you already have the ops team to run it. * Fine-tuning at rapid iteration speed: If you are repeatedly fine-tuning Kimi K3 and pushing a new version to production daily, self-hosting lets you iterate without API latency. This is rare; most teams fine-tune quarterly or less often. For the majority of engineering teams - especially those not at trillion-token-per-month scale - hosted API access is simpler, lower-risk, and often cheaper once you factor in labor. * Kimi K3 open weights represents a real inflection: frontier-tier capability is now available outside of closed APIs. This is valuable for data sovereignty and latency-critical workloads, but does not mean "download it and run it in production." * Self-hosting Kimi K3 is a distributed-systems problem, not a model problem. You need deep infrastructure maturity, quantization evaluation, redundancy planning, and operational runbooks before committing. * The cost comparison is not "model weights are free" vs. "API is expensive." It is "self-hosting infrastructure + ongoing labor + risk of cascading failure" vs. "simple API call + predictable per-token cost." For most teams, the API wins. * If data residency or sub-50ms latency is a hard constraint, self-hosting is justified. Otherwise, start with hosted API access and migrate to self-hosting only if the token volume or iteration speed makes the cost-benefit clear. * Smaller open models (Llama 3.1 8B, Mistral 7B) are far easier to self-host and are sufficient for many workloads. Do not jump to Kimi K3 just because it exists. References: Moonshot AI's Kimi K3 announcement and technical documentation (July 26-27, 2026), public reporting on Together AI and Modal's hosted Kimi K3 offerings, Moonshot's published claims about Kimi Delta Attention and Attention Residuals efficiency.
Fireworks AI raised $1.5 billion as companies flee costly AI for open-source alternatives. The AI infrastructure startup surpassed $1 billion in annualized revenue, up fivefold year-over-year, as companies seek cheaper alternatives to frontier models Fireworks AI raised $1.505 billion in a Series D funding round on Thursday, valuing the company at $17.5 billion, as demand grows from companies looking to build and deploy customized AI models at lower cost than frontier alternatives. Atreides Management, Index Ventures, and TCV led the round. Nvidia $NVDA -1.56%, Lightspeed Venture Partners, Evantic Capital, Bessemer Venture Partners, Menlo Ventures, Insight Partners, Ontario Teachers' Pension Plan, and Lone Pine Capital also participated, the company said. The fundraise coincides with Fireworks crossing the $1 billion threshold in annualized revenue, a figure that has grown fivefold compared with the previous year. Token volume on its platform has climbed to more than 40 trillion per day from 15 trillion over the same stretch, according to Reuters. The company last raised $250 million at a $4 billion valuation in October. Fireworks co-founder and CEO Lin Qiao said the company's cost advantage over equivalent closed models is a key driver of growth. "Our cost compared with the equivalent-quality closed model is five to 10 times cheaper," Qiao told CNBC. Rising costs for the latest AI models have made finance executives increasingly uneasy, prompting them to steer employees toward open-source options. Founded in 2022 by Qiao and six co-founders, all former Meta $META -3.36% engineers, Fireworks offers infrastructure that lets enterprises fine-tune general-purpose models on their own data and serve them in production. The company's workforce stands at roughly 200, and Qiao said she plans to triple that figure before the year is out. Customers include Uber $UBER -2.02%, Shopify $SHOP -5.42%, Doximity, Elastic, GitLab, and MongoDB $MDB -1.91%, according to Reuters. Legal AI company Harvey and coding tool Cursor have also built on the platform, the company said. Fireworks competes with Together AI and Baseten in the AI inference cloud market. In March, Fireworks announced a partnership with Microsoft $MSFT -2.24% that allows Microsoft customers to access models through Fireworks' platform, which draws on computing capacity from more than 20 suppliers, according to CNBC. "We believe both frontier and open models will increasingly be used together," said Gavin Baker, CIO and managing partner at Atreides Management. Fireworks said the new capital will be used to expand compute infrastructure, grow its engineering team, and deepen partnerships with cloud providers including Microsoft and Nvidia. Daily Brief The essential business news, delivered fresh every morning. Join 500,000+ readers who start their day with Quartz.
Together AI raises $800M Led by Aramco Ventures at $8.3B valuation. Saudi Aramco's venture arm led an $800 million Series C into open-source inference provider Together AI, more than doubling its valuation to $8.3 billion as enterprise demand for open models triples. · EN · FR · AR Together AI has raised an $800 million Series C at an $8.3 billion post-money valuation, led by Aramco Ventures, the venture capital arm of Saudi Arabia's state oil company. The round, announced July 1, 2026, more than doubles the San Francisco company's valuation from the $3.3 billion it commanded at its Series B roughly sixteen months earlier, and lands as enterprise adoption of open-source models triples year over year. Key highlights. * $800 million Series C at an $8.3 billion post-money valuation, a 2.5x jump from the prior round * Led by Aramco Ventures, with NVIDIA, Vista Equity Partners, General Catalyst, and Emergence Capital participating * Total capital raised now stands at roughly $1.3 billion * Annualized bookings exceed $1.15 billion, according to the company's most recent quarter * Together AI plans to scale its infrastructure footprint roughly 50-fold over the next five years Details. Together AI operates as a "neocloud" - a GPU infrastructure and inference provider built specifically for AI workloads rather than general-purpose computing. Its platform hosts and serves open-weight models, letting companies run inference at scale without paying frontier-lab API prices. The customer roster reflects where that demand is concentrated. Coding-agent companies Cursor and Cognition both run on Together's infrastructure, as does customer-service AI startup Decagon. The company reports thousands of paying customers overall. The economics are the pitch. Together AI says customers report cost savings ranging from 6x to 60x versus closed-model pricing for equivalent or better performance on their workloads. Decagon specifically achieved a sixfold reduction in inference expenses after migrating. Beyond the lead investor, the round drew an unusually broad syndicate: NVIDIA, Vista Equity Partners, General Catalyst, Emergence Capital, March Capital, Pegatron, Salesforce Ventures, Schneider Electric's SE Ventures, and SentinelOne's S Ventures. Impact. The round is a bet that inference - not training - is where the durable business sits, and that a meaningful share of it will run on open weights rather than proprietary APIs. That bet has data behind it. Open-source model adoption tripled over the preceding twelve months as companies hunted for alternatives to premium closed-model pricing. Together AI's $1.15 billion in annualized bookings puts it in the revenue tier of established enterprise software businesses, not speculative infrastructure startups. For the MENA region, the identity of the lead investor matters as much as the round size. Aramco Ventures backing the largest independent open-source inference platform signals that Gulf capital is moving past data-center real estate and into the software layer that determines how AI is actually served. "Together has built the platform that makes open source models genuinely usable at enterprise scale," said Abhishek Shukla, Managing Director at Aramco Ventures. Background. Founded by CEO Vipul Ved Prakash, Stanford professor Percy Liang, and CTO Ce Zhang, Together AI raised $102.5 million in its 2023 Series A and $305 million in its early-2025 Series B at a $3.3 billion valuation. The company's positioning has stayed consistent across those rounds. "Intelligence is becoming a foundational resource for the modern economy, every bit as essential as electricity, bandwidth or capital," Prakash said. "Our mission is to ensure that intelligence is abundant, not expensive." He framed the open-versus-closed question in historical terms: "History shows that the biggest technology shifts are won by open ecosystems that make innovation cheaper, faster and available to everyone. The future of AI won't be owned by a few companies. It will be built by millions of developers and businesses." What's next. Together AI says the capital will fund product expansion across its inference platform and a roughly 50-fold increase in compute capacity over five years - an aggressive buildout in a market where GPU supply remains the binding constraint. The competitive field is filling in fast. Upscale AI reached a $2 billion valuation on $500 million raised, TensorWave closed a $350 million Series B at $1.55 billion, and AI gateway OpenRouter now carries a $1.3 billion valuation. Each is wagering on a different slice of the same thesis: that the inference layer, not the model layer, is where AI infrastructure spending ultimately concentrates.
Together AI offers predictable inference. Together AI introduces Provisioned Throughput, offering reserved inference capacity for open models with token-based pricing and a 99% uptime SLA. Together AI is introducing Provisioned Throughput, a new service designed to offer reserved inference capacity for open-weight frontier models. This move aims to provide businesses with predictable performance and pricing, a critical factor as AI inference costs become a significant line item for companies. Visual TL;DR. AI Inference Costs Rise leads to Inference Dilemma. Inference Dilemma solves Together AI's Solution. Together AI's Solution offers Reserved Capacity. Reserved Capacity with Predictable Pricing. Reserved Capacity ensures Reliable Performance. Reliable Performance leading to Lower Costs. * AI Inference Costs Rise: businesses face significant line item for AI inference * Inference Dilemma: choose between serverless convenience or dedicated infrastructure * Together AI's Solution: introduces Provisioned Throughput service * Reserved Capacity: guaranteed inference capacity for open models * Predictable Pricing: token-based pricing similar to proprietary models * Reliable Performance: 99% uptime SLA for production workloads * Lower Costs: up to 90% cheaper than proprietary alternatives Visual TL;DR !- /sh-diagram -> Historically, organizations have had to choose between the convenience of best-effort serverless inference or the control of dedicated, managed infrastructure. Provisioned Throughput seeks to occupy a middle ground, offering the simplicity of token-based pricing, akin to proprietary model providers, combined with guaranteed capacity and a service level agreement (SLA).