Why on-device AI is the web story of 2026
For the last few years, "adding AI" to a website meant the same architecture every time. The browser sends text to your server, your server calls a hosted model with a secret key, the model streams tokens back, and you pay for every one of them. That pattern works, and it is still the right answer for many features. But it has three costs that never go away: latency for every round trip, a bill that grows with your traffic, and user text leaving the device.
Chrome now ships a different option. A small language model, Gemini Nano, is downloaded once into the browser and shared by every site that uses it. Your page talks to it through a set of task-specific JavaScript APIs. There is no API key, no server, and no per-token charge, and once the model is on the device the text the user types never has to leave it.
This is not a preview any more. According to Chrome's built-in AI API status table, these APIs are available to web pages:
| API | What it does | Web status |
|---|---|---|
| Summarizer | Turns long text into key points, a TL;DR, a teaser or a headline | Stable from Chrome 138 |
| Language Detector | Tells you which language a piece of text is written in | Stable from Chrome 138 |
| Translator | Translates text between language pairs | Stable from Chrome 138 |
| Prompt API | Free-form prompts to the model, including structured JSON output | Listed for the web from Chrome 148 |
| Writer, Rewriter, Proofreader | Drafting and editing helpers | Developer trial |
The table is the part to bookmark, because the status of the newer APIs moves quickly. Everything in this guide that depends on a status is labelled, so you can check it against the table before you ship.
By the end you will have built three features with real code: an article summarizer with a download progress bar, a "translate this comment" button that detects the source language first, and a structured classifier built on the Prompt API. You will also wrap them in a React hook, add a server fallback for browsers that can't run the model, and see where the next step, WebMCP, fits in.
The mental model: one model, many sites, many states
Before writing code, it helps to understand three facts about how these APIs behave, because almost every bug people hit comes from one of them.
Fact 1: the model may not be there yet
The model is not bundled with Chrome. It is downloaded the first time a site needs it, and it is shared across all sites after that. So every API has an availability() method that tells you where things stand. Chrome's get started guide defines four states:
unavailable: this device, or the options you asked for, are not supported.downloadable: the model can be used, but it has to be downloaded first.downloading: a download is in progress and must finish first.available: you can create a session right now.
Your UI needs an answer for each of these, not just the happy path.
Fact 2: the user has to have interacted with the page
Starting a session can trigger a large download, so Chrome requires that "the user must meaningfully interact with your page" before create() will start one. In practice that means you call create() from a click or key handler, not on page load. You can check with navigator.userActivation.isActive.
Fact 3: many devices cannot run it
The model has real hardware requirements. From the same guide, the device needs:
- Windows 10 or 11, macOS 13 or later, Linux, or ChromeOS on Chromebook Plus devices. Chrome on Android and iOS is not supported.
- At least 22 GB of free space on the volume that holds the Chrome profile.
- A GPU with strictly more than 4 GB of VRAM, or a CPU setup with 16 GB of RAM and 4 or more cores.
- An unmetered connection for the first download.
Put those three facts together and the architecture writes itself: on-device AI is a progressive enhancement. You detect support, offer the feature when it can work, explain the download when one is needed, and fall back to a server (or to no AI at all) everywhere else. Treat it like you would treat a new CSS feature, not like a dependency you can assume.
Step 1: feature detection and availability
Each API is a global constructor. Detecting it is a one-line in check. Here is a small helper that turns the detection and the availability check into one value your UI can switch on:
// ai-support.js
export async function summarizerStatus() {
if (!('Summarizer' in self)) return 'unsupported';
try {
return await Summarizer.availability(); // 'unavailable' | 'downloadable' | 'downloading' | 'available'
} catch {
return 'unavailable';
}
}
There are two different "no" answers here, and they mean different things to the user:
'unsupported'means this browser has no such API at all. Another browser, or an older Chrome, will land here. Hide the feature, or route it to your server.'unavailable'means the API exists but this device or configuration can't use it. Same treatment for the user, but worth logging separately, because it tells you how many of your Chrome users are on hardware that can't run the model.
Call this when the page loads and use the result to decide whether to render the AI button at all. Calling availability() does not start a download, so it is safe to call early.
Step 2: create a session and show download progress
create() returns the object you actually work with. If the model still needs to be downloaded, create() starts the download and resolves when it finishes. You attach a monitor callback to receive downloadprogress events, where e.loaded is a fraction between 0 and 1.
A <progress> element is the simplest honest UI for this. It is accessible by default and needs no library:
<button id="summarize" type="button" hidden>Summarize this article</button>
<label id="download-label" for="download" hidden>Downloading the on-device model</label>
<progress id="download" max="1" value="0" hidden></progress>
<div id="summary" aria-live="polite"></div>
import { summarizerStatus } from './ai-support.js';
const button = document.querySelector('#summarize');
const bar = document.querySelector('#download');
const label = document.querySelector('#download-label');
let summarizer; // created once, reused for later clicks
async function getSummarizer() {
if (summarizer) return summarizer;
summarizer = await Summarizer.create({
type: 'key-points',
format: 'plain-text',
length: 'medium',
monitor(m) {
m.addEventListener('downloadprogress', (e) => {
bar.hidden = label.hidden = false;
bar.value = e.loaded;
if (e.loaded === 1) bar.hidden = label.hidden = true;
});
},
});
return summarizer;
}
const status = await summarizerStatus();
if (status !== 'unsupported' && status !== 'unavailable') {
button.hidden = false;
if (status !== 'available') {
button.textContent = 'Summarize this article (one-time model download)';
}
}
Two details matter here.
First, create() is called inside the click handler (you will see it in the next step), never at load time. That satisfies the user-activation rule, and it means you only download a multi-gigabyte model for people who asked for the feature.
Second, the session is cached in a module variable. Creating a session has a cost even when the model is already downloaded, and the options can't be changed after creation, so create one per configuration and reuse it.
Step 3: summarize an article
With a session in hand, summarizing is one call. The Summarizer documentation lists the options you can pass to create():
| Option | Values | Default |
|---|---|---|
type |
key-points, tldr, teaser, headline |
key-points |
format |
markdown, plain-text |
markdown |
length |
short, medium, long |
short |
sharedContext |
Background text that applies to every summary | none |
expectedInputLanguages, outputLanguage |
Language codes such as en, ja, es, de, fr |
none |
length means different things for each type. For tldr and teaser it is 1, 3 or 5 sentences; for key-points it is 3, 5 or 7 bullets; for headline it is 12, 17 or 22 words. That is useful to know when you design the space the summary will fill.
Here is the click handler:
button.addEventListener('click', async () => {
button.disabled = true;
const out = document.querySelector('#summary');
out.textContent = 'Summarizing…';
try {
const s = await getSummarizer();
// innerText gives the rendered text, without markup, scripts or hidden nodes.
const text = document.querySelector('article').innerText;
const summary = await s.summarize(text, {
context: 'A web development tutorial for working developers.',
});
out.textContent = summary;
} catch (err) {
out.textContent = 'The summary could not be created on this device.';
console.error(err);
} finally {
button.disabled = false;
}
});
Note the use of innerText rather than innerHTML. The documentation itself recommends removing unnecessary data such as markup before summarizing. Markup wastes the model's limited input space, and it adds noise that lowers the quality of the result.
sharedContext versus context
There are two places to put background information, and they are not the same:
sharedContextis set once increate()and applies to every summary that session produces. Use it for facts about your site: "These are support tickets for a payments product."contextis passed to eachsummarize()call. Use it for facts about this one input: "This ticket was written by an enterprise customer."
Keeping the stable facts in sharedContext keeps each call short and makes the output more consistent.
Streaming the summary
For long inputs, the batch call can take a few seconds. summarizeStreaming() returns an async iterable, so you can show text as it arrives:
const stream = s.summarizeStreaming(text, { context: 'For junior developers.' });
out.textContent = '';
for await (const chunk of stream) {
out.textContent += chunk;
}
One accessibility note: aria-live="polite" on the output region means screen readers announce the result. With streaming, that can mean many small announcements. If that becomes noisy, write the chunks into a visually hidden buffer and copy the final text into the live region once the stream ends.
Step 4: detect a language, then translate it
The second feature is a "Translate" button under user comments. It needs two APIs: the Language Detector, because you don't know what language a comment is in, and the Translator.
Detecting the language
The Language Detector returns a ranked list of guesses, each with a confidence between 0 and 1:
async function detectLanguage(text) {
if (!('LanguageDetector' in self)) return null;
const detector = await LanguageDetector.create();
const [best] = await detector.detect(text);
// Short inputs are unreliable: treat low confidence as "unknown".
if (!best || best.confidence < 0.6) return null;
return best.detectedLanguage; // e.g. 'es'
}
The documentation is explicit that "very short phrases and single words should be avoided, as the accuracy of the results will be low." That is why the helper returns null below a confidence threshold. Pick the threshold by testing with real comments in the languages you care about; 0.6 is a starting point, not a rule.
Translating it
The Translator is created for one language pair. You check availability for that specific pair first:
async function translateComment(text, targetLanguage = 'en') {
if (!('Translator' in self)) return null;
const sourceLanguage = await detectLanguage(text);
if (!sourceLanguage || sourceLanguage === targetLanguage) return null;
const availability = await Translator.availability({ sourceLanguage, targetLanguage });
if (availability === 'unavailable') return null;
const translator = await Translator.create({
sourceLanguage,
targetLanguage,
monitor(m) {
m.addEventListener('downloadprogress', (e) => {
console.log(`Language pack: ${Math.round(e.loaded * 100)}%`);
});
},
});
return translator.translate(text);
}
Call translateComment() from the button's click handler, for the same user-activation reason as before.
A privacy detail worth understanding
The Translator documentation notes that until a site creates a translator for a given language pair, availability() reports every pair as downloadable, even if the language pack is already on the device. This is deliberate. If sites could see exactly which language packs a browser has downloaded, that list would become a fingerprint, and it would reveal which languages the person reads. So don't build logic that depends on telling downloadable and available apart for translation; just handle both by calling create() with a progress monitor.
Step 5: structured output with the Prompt API
The task APIs are the safest place to start, because they are stable and narrow. The Prompt API is the general-purpose one: you send instructions and get text back. Chrome's status table lists it for web pages from Chrome 148 (it has been available to Chrome extensions since 138). Check the table for your users' Chrome versions before you rely on it, and keep the progressive-enhancement pattern from Step 1.
The most useful thing the Prompt API gives a web developer is structured output. You pass a JSON Schema as responseConstraint, and the model's response is constrained to match it. That turns a chatty language model into something you can safely feed into code.
Here is a classifier that labels a support message, using the Prompt API documentation shapes:
const schema = {
type: 'object',
properties: {
category: { type: 'string', enum: ['bug', 'billing', 'feature-request', 'other'] },
urgent: { type: 'boolean' },
},
required: ['category', 'urgent'],
};
async function classify(message) {
if (!('LanguageModel' in self)) return null;
if ((await LanguageModel.availability()) === 'unavailable') return null;
const session = await LanguageModel.create({
initialPrompts: [
{
role: 'system',
content: 'You label customer support messages. Answer only with the requested JSON.',
},
],
});
try {
const raw = await session.prompt(message, { responseConstraint: schema });
return JSON.parse(raw); // { category: 'billing', urgent: true }
} finally {
session.destroy();
}
}
A few things are going on here:
initialPromptswith asystemrole sets the instructions once for the session, the same way a system message works with hosted models.responseConstraintmakes the response valid JSON for your schema, soJSON.parseis safe to call. Validate the values anyway before acting on them; a constraint guarantees shape, not judgement.session.destroy()frees the session's resources. After it is destroyed, further prompts on it fail, so only destroy a session you are done with.
Watching the context window
A session has a limited context window, and every prompt and response in the session uses part of it. The session exposes how much is used:
console.log(`${session.contextUsage}/${session.contextWindow}`);
Long conversations will eventually fill it. When contextUsage gets close to contextWindow, start a new session and carry over a short summary of the conversation instead of the full history. If you want to try out how much text fits in a token budget, the LLM Token Estimator gives a quick feel for how text length turns into tokens.
Letting users cancel
Model calls can take seconds. Give users a Stop button with an AbortController:
const controller = new AbortController();
stopButton.onclick = () => controller.abort();
const stream = session.promptStreaming('Explain this error message in plain English.', {
signal: controller.signal,
});
for await (const chunk of stream) output.textContent += chunk;
Branching a conversation
If you want to ask two different follow-up questions from the same starting point, session.clone() copies a session, including its history, so the two branches don't interfere with each other. That is cheaper than replaying the whole conversation into a new session.
Step 6: wrap it in a React hook
In a React app, you want the status logic in one place and a simple interface for components. Here is a hook for the summarizer that covers every state from Step 1:
'use client';
import { useCallback, useEffect, useRef, useState } from 'react';
type Status = 'checking' | 'unsupported' | 'unavailable' | 'downloadable' | 'downloading' | 'available';
export function useOnDeviceSummarizer(options: Record<string, unknown> = { type: 'tldr' }) {
const [status, setStatus] = useState<Status>('checking');
const [progress, setProgress] = useState(0);
const sessionRef = useRef<any>(null);
useEffect(() => {
let cancelled = false;
(async () => {
if (!('Summarizer' in self)) return setStatus('unsupported');
const value = await (self as any).Summarizer.availability();
if (!cancelled) setStatus(value);
})();
return () => {
cancelled = true;
sessionRef.current?.destroy?.();
sessionRef.current = null;
};
}, []);
// Call this from an event handler, so the user-activation rule is met.
const summarize = useCallback(
async (text: string) => {
if (!sessionRef.current) {
sessionRef.current = await (self as any).Summarizer.create({
...options,
monitor(m: EventTarget) {
m.addEventListener('downloadprogress', (e: any) => {
setStatus('downloading');
setProgress(e.loaded);
});
},
});
setStatus('available');
}
return sessionRef.current.summarize(text) as Promise<string>;
},
[options],
);
return { status, progress, summarize };
}
A component then only deals with what to render:
function SummaryButton({ getText }: { getText: () => string }) {
const { status, progress, summarize } = useOnDeviceSummarizer();
const [summary, setSummary] = useState('');
if (status === 'checking' || status === 'unsupported' || status === 'unavailable') return null;
return (
<div>
<button type="button" onClick={async () => setSummary(await summarize(getText()))}>
{status === 'available' ? 'Summarize' : 'Summarize (one-time download)'}
</button>
{status === 'downloading' ? <progress max={1} value={progress} /> : null}
<p aria-live="polite">{summary}</p>
</div>
);
}
This is a client component for an obvious reason: the APIs live in the browser. If you render this from a Next.js server component, put 'use client' at the top of the file, and remember that during server rendering there is no self.Summarizer, which is why the hook does its detection inside useEffect.
The any types are a shortcut to keep the example readable. In a real project, add a small declare global block with the parts of the API you use, so the rest of your code stays typed.
Step 7: a server fallback for everyone else
Remember Fact 3: phones and lower-powered laptops can't run the model. If the feature matters, give those users the same result from a server. The trick is to keep one interface in your app and decide where it runs underneath:
export async function summarizeAnywhere(text) {
const status = await summarizerStatus();
if (status !== 'unsupported' && status !== 'unavailable') {
const s = await getSummarizer();
return { text: await s.summarize(text), source: 'device' };
}
const res = await fetch('/api/summarize', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
body: JSON.stringify({ text }),
});
if (!res.ok) throw new Error('Summary service unavailable');
return { text: (await res.json()).summary, source: 'server' };
}
The /api/summarize route calls a hosted model with your key, kept on the server. Our AI SDK guide covers streaming and tool calls for that side. If you'd rather run open models yourself, the Ollama with Next.js 16 tutorial shows a self-hosted route, and the WebLLM and React 19 tutorial shows a different on-device approach, where you ship the model weights yourself through WebGPU instead of using the one built into Chrome.
The source field is worth keeping. It lets you show a small note when text was sent to a server ("Summarized by our server") and it gives you analytics on how many requests the device handled for free.
Here is a quick way to decide which path a feature should use:
| Question | On-device built-in AI | Hosted model on your server |
|---|---|---|
| Cost per request | None | You pay per token |
| Text leaves the device | No, once the model is downloaded | Yes |
| Works on phones | No | Yes |
| Model quality | Small model, best on narrow tasks | Large models, best on open-ended tasks |
| First use | May need a large one-time download | Instant |
| Good fit | Summaries, translation, labelling, short rewrites | Long reasoning, research, code generation |
Most real products end up using both: the device for quick, frequent, privacy-sensitive tasks, and the server for the heavy ones.
Production checklist
Before you ship, walk through this list. Each item is something that is easy to miss in a demo and painful to discover in production.
- Feature-detect every API separately. Having the Summarizer doesn't mean having the Translator. Check each constructor and each
availability(). - Only call
create()from user input. Downloads need user activation, and nobody wants a surprise multi-gigabyte download on page load. - Show download progress. A
<progress>element bound todownloadprogressevents is enough. Without it, the first click looks broken. - Send text, not markup. Use
innerTextor your own plain-text source. It saves input space and improves results. - Reuse sessions. Create one per configuration and keep it, rather than one per click. Destroy Prompt API sessions you no longer need.
- Handle short input in language detection. Treat low-confidence results as unknown.
- Treat model output as untrusted. Render it with
textContent, notinnerHTML. If you summarize text that users wrote, that text can contain instructions aimed at the model, so never let model output trigger actions without validation. - Know the iframe rule. By default the APIs work only in top-level windows and same-origin iframes. A cross-origin iframe needs permission from the parent, for example
<iframe src="https://widget.example.com/" allow="summarizer"></iframe>. - Don't plan on Web Workers. The Summarizer documentation states the API isn't available in Web Workers at this time, so keep calls on the main thread and keep them async.
- Test the real states. On a supported machine,
chrome://on-device-internalsshows the model's state, and runningLanguageModel.availability()in the DevTools console should returnavailableonce the model is ready.
What's next: WebMCP and the agentic web
Everything above is about your page using a model. The next shift runs the other way: a model, usually an AI agent inside the browser, using your page.
At Google I/O 2026, Chrome's team talked about an "agentic web", where sites declare structured tools that agents can call, instead of agents clicking through the visual interface and guessing. The proposal for this is called WebMCP. It borrows its ideas from the Model Context Protocol: a page registers tools with a name, a description and a JSON Schema for the input, and an agent calls them like functions.
It is important to be precise about its status, because it is moving quickly:
- It is an origin trial, not a stable API. You need to register for the trial to use it on a live site.
- The entry point has already moved once. Early versions exposed tools through
navigator.modelContext. The specification has since moved this todocument.modelContext, and code written against the old location is already out of date. - Other browsers haven't shipped it. Treat it as Chrome-only for now.
Because of that, this guide doesn't teach a full WebMCP implementation; any code here could be wrong within a few months. What you can do today is get ready for it, and every step makes your site better for people as well:
- Make your important actions available as clean functions in your front-end code ("search the docs", "add to cart"), not only as click handlers. Those functions become tools later with a few lines of registration code.
- Describe your forms and buttons properly with labels and accessible names. Agents that can't call tools fall back to reading the page the way a screen reader does.
- Keep your structured data (JSON-LD) accurate. Agents read it.
When WebMCP stabilizes, you will feature-detect document.modelContext the same way you detected Summarizer in Step 1, and register the functions you already have.
Summary
Chrome's built-in AI APIs change the cost model of small AI features on the web. Summaries, translation and short structured prompts can now run on the user's own device, with no key, no server and no per-request cost.
The pattern that makes them work in production is the same one the web has always used for new capabilities:
- Detect the API and check
availability(). - Offer the feature when it can work, and explain the one-time download.
- Create sessions from user actions, reuse them, and show progress.
- Fall back to a server, or to no AI, everywhere else.
Start with the Summarizer, because it is stable, narrow and useful on almost any content site. Add the Language Detector and Translator for user-generated content. Reach for the Prompt API when you need structured output, and check its status for your audience first. Then keep an eye on WebMCP, which will let agents use the same functions your interface already calls.
If you are planning the bigger picture of where AI fits in a modern stack, our full-stack roadmap with React and vector databases covers the server-side half of that story.