Skip to main content

JournalAI

On-Device AI in the Browser: A Practical Guide to Chrome's Built-in AI APIs

Summarize, translate and prompt a model that runs on the user's device, with no API key and no per-token bill. A hands-on guide to Chrome's Summarizer, Language Detector, Translator and Prompt APIs, with progressive enhancement and a server fallback.

By LearnWebCraft Team21 min read

Why on-device AI is the web story of 2026

For the last few years, "adding AI" to a website meant the same architecture every time. The browser sends text to your server, your server calls a hosted model with a secret key, the model streams tokens back, and you pay for every one of them. That pattern works, and it is still the right answer for many features. But it has three costs that never go away: latency for every round trip, a bill that grows with your traffic, and user text leaving the device.

Chrome now ships a different option. A small language model, Gemini Nano, is downloaded once into the browser and shared by every site that uses it. Your page talks to it through a set of task-specific JavaScript APIs. There is no API key, no server, and no per-token charge, and once the model is on the device the text the user types never has to leave it.

This is not a preview any more. According to Chrome's built-in AI API status table, these APIs are available to web pages:

API What it does Web status
Summarizer Turns long text into key points, a TL;DR, a teaser or a headline Stable from Chrome 138
Language Detector Tells you which language a piece of text is written in Stable from Chrome 138
Translator Translates text between language pairs Stable from Chrome 138
Prompt API Free-form prompts to the model, including structured JSON output Listed for the web from Chrome 148
Writer, Rewriter, Proofreader Drafting and editing helpers Developer trial

The table is the part to bookmark, because the status of the newer APIs moves quickly. Everything in this guide that depends on a status is labelled, so you can check it against the table before you ship.

By the end you will have built three features with real code: an article summarizer with a download progress bar, a "translate this comment" button that detects the source language first, and a structured classifier built on the Prompt API. You will also wrap them in a React hook, add a server fallback for browsers that can't run the model, and see where the next step, WebMCP, fits in.

The mental model: one model, many sites, many states

Before writing code, it helps to understand three facts about how these APIs behave, because almost every bug people hit comes from one of them.

Fact 1: the model may not be there yet

The model is not bundled with Chrome. It is downloaded the first time a site needs it, and it is shared across all sites after that. So every API has an availability() method that tells you where things stand. Chrome's get started guide defines four states:

  • unavailable: this device, or the options you asked for, are not supported.
  • downloadable: the model can be used, but it has to be downloaded first.
  • downloading: a download is in progress and must finish first.
  • available: you can create a session right now.

Your UI needs an answer for each of these, not just the happy path.

Fact 2: the user has to have interacted with the page

Starting a session can trigger a large download, so Chrome requires that "the user must meaningfully interact with your page" before create() will start one. In practice that means you call create() from a click or key handler, not on page load. You can check with navigator.userActivation.isActive.

Fact 3: many devices cannot run it

The model has real hardware requirements. From the same guide, the device needs:

  • Windows 10 or 11, macOS 13 or later, Linux, or ChromeOS on Chromebook Plus devices. Chrome on Android and iOS is not supported.
  • At least 22 GB of free space on the volume that holds the Chrome profile.
  • A GPU with strictly more than 4 GB of VRAM, or a CPU setup with 16 GB of RAM and 4 or more cores.
  • An unmetered connection for the first download.

Put those three facts together and the architecture writes itself: on-device AI is a progressive enhancement. You detect support, offer the feature when it can work, explain the download when one is needed, and fall back to a server (or to no AI at all) everywhere else. Treat it like you would treat a new CSS feature, not like a dependency you can assume.

Step 1: feature detection and availability

Each API is a global constructor. Detecting it is a one-line in check. Here is a small helper that turns the detection and the availability check into one value your UI can switch on:

Fig. 1js
// ai-support.js
export async function summarizerStatus() {
  if (!('Summarizer' in self)) return 'unsupported';
  try {
    return await Summarizer.availability(); // 'unavailable' | 'downloadable' | 'downloading' | 'available'
  } catch {
    return 'unavailable';
  }
}

There are two different "no" answers here, and they mean different things to the user:

  • 'unsupported' means this browser has no such API at all. Another browser, or an older Chrome, will land here. Hide the feature, or route it to your server.
  • 'unavailable' means the API exists but this device or configuration can't use it. Same treatment for the user, but worth logging separately, because it tells you how many of your Chrome users are on hardware that can't run the model.

Call this when the page loads and use the result to decide whether to render the AI button at all. Calling availability() does not start a download, so it is safe to call early.

Step 2: create a session and show download progress

create() returns the object you actually work with. If the model still needs to be downloaded, create() starts the download and resolves when it finishes. You attach a monitor callback to receive downloadprogress events, where e.loaded is a fraction between 0 and 1.

A <progress> element is the simplest honest UI for this. It is accessible by default and needs no library:

Fig. 2html
<button id="summarize" type="button" hidden>Summarize this article</button>
<label id="download-label" for="download" hidden>Downloading the on-device model</label>
<progress id="download" max="1" value="0" hidden></progress>
<div id="summary" aria-live="polite"></div>
Fig. 3js
import { summarizerStatus } from './ai-support.js';

const button = document.querySelector('#summarize');
const bar = document.querySelector('#download');
const label = document.querySelector('#download-label');

let summarizer; // created once, reused for later clicks

async function getSummarizer() {
  if (summarizer) return summarizer;
  summarizer = await Summarizer.create({
    type: 'key-points',
    format: 'plain-text',
    length: 'medium',
    monitor(m) {
      m.addEventListener('downloadprogress', (e) => {
        bar.hidden = label.hidden = false;
        bar.value = e.loaded;
        if (e.loaded === 1) bar.hidden = label.hidden = true;
      });
    },
  });
  return summarizer;
}

const status = await summarizerStatus();
if (status !== 'unsupported' && status !== 'unavailable') {
  button.hidden = false;
  if (status !== 'available') {
    button.textContent = 'Summarize this article (one-time model download)';
  }
}

Two details matter here.

First, create() is called inside the click handler (you will see it in the next step), never at load time. That satisfies the user-activation rule, and it means you only download a multi-gigabyte model for people who asked for the feature.

Second, the session is cached in a module variable. Creating a session has a cost even when the model is already downloaded, and the options can't be changed after creation, so create one per configuration and reuse it.

Step 3: summarize an article

With a session in hand, summarizing is one call. The Summarizer documentation lists the options you can pass to create():

Option Values Default
type key-points, tldr, teaser, headline key-points
format markdown, plain-text markdown
length short, medium, long short
sharedContext Background text that applies to every summary none
expectedInputLanguages, outputLanguage Language codes such as en, ja, es, de, fr none

length means different things for each type. For tldr and teaser it is 1, 3 or 5 sentences; for key-points it is 3, 5 or 7 bullets; for headline it is 12, 17 or 22 words. That is useful to know when you design the space the summary will fill.

Here is the click handler:

Fig. 4js
button.addEventListener('click', async () => {
  button.disabled = true;
  const out = document.querySelector('#summary');
  out.textContent = 'Summarizing…';

  try {
    const s = await getSummarizer();
    // innerText gives the rendered text, without markup, scripts or hidden nodes.
    const text = document.querySelector('article').innerText;
    const summary = await s.summarize(text, {
      context: 'A web development tutorial for working developers.',
    });
    out.textContent = summary;
  } catch (err) {
    out.textContent = 'The summary could not be created on this device.';
    console.error(err);
  } finally {
    button.disabled = false;
  }
});

Note the use of innerText rather than innerHTML. The documentation itself recommends removing unnecessary data such as markup before summarizing. Markup wastes the model's limited input space, and it adds noise that lowers the quality of the result.

sharedContext versus context

There are two places to put background information, and they are not the same:

  • sharedContext is set once in create() and applies to every summary that session produces. Use it for facts about your site: "These are support tickets for a payments product."
  • context is passed to each summarize() call. Use it for facts about this one input: "This ticket was written by an enterprise customer."

Keeping the stable facts in sharedContext keeps each call short and makes the output more consistent.

Streaming the summary

For long inputs, the batch call can take a few seconds. summarizeStreaming() returns an async iterable, so you can show text as it arrives:

Fig. 5js
const stream = s.summarizeStreaming(text, { context: 'For junior developers.' });
out.textContent = '';
for await (const chunk of stream) {
  out.textContent += chunk;
}

One accessibility note: aria-live="polite" on the output region means screen readers announce the result. With streaming, that can mean many small announcements. If that becomes noisy, write the chunks into a visually hidden buffer and copy the final text into the live region once the stream ends.

Step 4: detect a language, then translate it

The second feature is a "Translate" button under user comments. It needs two APIs: the Language Detector, because you don't know what language a comment is in, and the Translator.

Detecting the language

The Language Detector returns a ranked list of guesses, each with a confidence between 0 and 1:

Fig. 6js
async function detectLanguage(text) {
  if (!('LanguageDetector' in self)) return null;
  const detector = await LanguageDetector.create();
  const [best] = await detector.detect(text);
  // Short inputs are unreliable: treat low confidence as "unknown".
  if (!best || best.confidence < 0.6) return null;
  return best.detectedLanguage; // e.g. 'es'
}

The documentation is explicit that "very short phrases and single words should be avoided, as the accuracy of the results will be low." That is why the helper returns null below a confidence threshold. Pick the threshold by testing with real comments in the languages you care about; 0.6 is a starting point, not a rule.

Translating it

The Translator is created for one language pair. You check availability for that specific pair first:

Fig. 7js
async function translateComment(text, targetLanguage = 'en') {
  if (!('Translator' in self)) return null;

  const sourceLanguage = await detectLanguage(text);
  if (!sourceLanguage || sourceLanguage === targetLanguage) return null;

  const availability = await Translator.availability({ sourceLanguage, targetLanguage });
  if (availability === 'unavailable') return null;

  const translator = await Translator.create({
    sourceLanguage,
    targetLanguage,
    monitor(m) {
      m.addEventListener('downloadprogress', (e) => {
        console.log(`Language pack: ${Math.round(e.loaded * 100)}%`);
      });
    },
  });
  return translator.translate(text);
}

Call translateComment() from the button's click handler, for the same user-activation reason as before.

A privacy detail worth understanding

The Translator documentation notes that until a site creates a translator for a given language pair, availability() reports every pair as downloadable, even if the language pack is already on the device. This is deliberate. If sites could see exactly which language packs a browser has downloaded, that list would become a fingerprint, and it would reveal which languages the person reads. So don't build logic that depends on telling downloadable and available apart for translation; just handle both by calling create() with a progress monitor.

Step 5: structured output with the Prompt API

The task APIs are the safest place to start, because they are stable and narrow. The Prompt API is the general-purpose one: you send instructions and get text back. Chrome's status table lists it for web pages from Chrome 148 (it has been available to Chrome extensions since 138). Check the table for your users' Chrome versions before you rely on it, and keep the progressive-enhancement pattern from Step 1.

The most useful thing the Prompt API gives a web developer is structured output. You pass a JSON Schema as responseConstraint, and the model's response is constrained to match it. That turns a chatty language model into something you can safely feed into code.

Here is a classifier that labels a support message, using the Prompt API documentation shapes:

Fig. 8js
const schema = {
  type: 'object',
  properties: {
    category: { type: 'string', enum: ['bug', 'billing', 'feature-request', 'other'] },
    urgent: { type: 'boolean' },
  },
  required: ['category', 'urgent'],
};

async function classify(message) {
  if (!('LanguageModel' in self)) return null;
  if ((await LanguageModel.availability()) === 'unavailable') return null;

  const session = await LanguageModel.create({
    initialPrompts: [
      {
        role: 'system',
        content: 'You label customer support messages. Answer only with the requested JSON.',
      },
    ],
  });

  try {
    const raw = await session.prompt(message, { responseConstraint: schema });
    return JSON.parse(raw); // { category: 'billing', urgent: true }
  } finally {
    session.destroy();
  }
}

A few things are going on here:

  • initialPrompts with a system role sets the instructions once for the session, the same way a system message works with hosted models.
  • responseConstraint makes the response valid JSON for your schema, so JSON.parse is safe to call. Validate the values anyway before acting on them; a constraint guarantees shape, not judgement.
  • session.destroy() frees the session's resources. After it is destroyed, further prompts on it fail, so only destroy a session you are done with.

Watching the context window

A session has a limited context window, and every prompt and response in the session uses part of it. The session exposes how much is used:

Fig. 9js
console.log(`${session.contextUsage}/${session.contextWindow}`);

Long conversations will eventually fill it. When contextUsage gets close to contextWindow, start a new session and carry over a short summary of the conversation instead of the full history. If you want to try out how much text fits in a token budget, the LLM Token Estimator gives a quick feel for how text length turns into tokens.

Letting users cancel

Model calls can take seconds. Give users a Stop button with an AbortController:

Fig. 10js
const controller = new AbortController();
stopButton.onclick = () => controller.abort();

const stream = session.promptStreaming('Explain this error message in plain English.', {
  signal: controller.signal,
});
for await (const chunk of stream) output.textContent += chunk;

Branching a conversation

If you want to ask two different follow-up questions from the same starting point, session.clone() copies a session, including its history, so the two branches don't interfere with each other. That is cheaper than replaying the whole conversation into a new session.

Step 6: wrap it in a React hook

In a React app, you want the status logic in one place and a simple interface for components. Here is a hook for the summarizer that covers every state from Step 1:

Fig. 11tsx
'use client';
import { useCallback, useEffect, useRef, useState } from 'react';

type Status = 'checking' | 'unsupported' | 'unavailable' | 'downloadable' | 'downloading' | 'available';

export function useOnDeviceSummarizer(options: Record<string, unknown> = { type: 'tldr' }) {
  const [status, setStatus] = useState<Status>('checking');
  const [progress, setProgress] = useState(0);
  const sessionRef = useRef<any>(null);

  useEffect(() => {
    let cancelled = false;
    (async () => {
      if (!('Summarizer' in self)) return setStatus('unsupported');
      const value = await (self as any).Summarizer.availability();
      if (!cancelled) setStatus(value);
    })();
    return () => {
      cancelled = true;
      sessionRef.current?.destroy?.();
      sessionRef.current = null;
    };
  }, []);

  // Call this from an event handler, so the user-activation rule is met.
  const summarize = useCallback(
    async (text: string) => {
      if (!sessionRef.current) {
        sessionRef.current = await (self as any).Summarizer.create({
          ...options,
          monitor(m: EventTarget) {
            m.addEventListener('downloadprogress', (e: any) => {
              setStatus('downloading');
              setProgress(e.loaded);
            });
          },
        });
        setStatus('available');
      }
      return sessionRef.current.summarize(text) as Promise<string>;
    },
    [options],
  );

  return { status, progress, summarize };
}

A component then only deals with what to render:

Fig. 12tsx
function SummaryButton({ getText }: { getText: () => string }) {
  const { status, progress, summarize } = useOnDeviceSummarizer();
  const [summary, setSummary] = useState('');

  if (status === 'checking' || status === 'unsupported' || status === 'unavailable') return null;

  return (
    <div>
      <button type="button" onClick={async () => setSummary(await summarize(getText()))}>
        {status === 'available' ? 'Summarize' : 'Summarize (one-time download)'}
      </button>
      {status === 'downloading' ? <progress max={1} value={progress} /> : null}
      <p aria-live="polite">{summary}</p>
    </div>
  );
}

This is a client component for an obvious reason: the APIs live in the browser. If you render this from a Next.js server component, put 'use client' at the top of the file, and remember that during server rendering there is no self.Summarizer, which is why the hook does its detection inside useEffect.

The any types are a shortcut to keep the example readable. In a real project, add a small declare global block with the parts of the API you use, so the rest of your code stays typed.

Step 7: a server fallback for everyone else

Remember Fact 3: phones and lower-powered laptops can't run the model. If the feature matters, give those users the same result from a server. The trick is to keep one interface in your app and decide where it runs underneath:

Fig. 13js
export async function summarizeAnywhere(text) {
  const status = await summarizerStatus();
  if (status !== 'unsupported' && status !== 'unavailable') {
    const s = await getSummarizer();
    return { text: await s.summarize(text), source: 'device' };
  }
  const res = await fetch('/api/summarize', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({ text }),
  });
  if (!res.ok) throw new Error('Summary service unavailable');
  return { text: (await res.json()).summary, source: 'server' };
}

The /api/summarize route calls a hosted model with your key, kept on the server. Our AI SDK guide covers streaming and tool calls for that side. If you'd rather run open models yourself, the Ollama with Next.js 16 tutorial shows a self-hosted route, and the WebLLM and React 19 tutorial shows a different on-device approach, where you ship the model weights yourself through WebGPU instead of using the one built into Chrome.

The source field is worth keeping. It lets you show a small note when text was sent to a server ("Summarized by our server") and it gives you analytics on how many requests the device handled for free.

Here is a quick way to decide which path a feature should use:

Question On-device built-in AI Hosted model on your server
Cost per request None You pay per token
Text leaves the device No, once the model is downloaded Yes
Works on phones No Yes
Model quality Small model, best on narrow tasks Large models, best on open-ended tasks
First use May need a large one-time download Instant
Good fit Summaries, translation, labelling, short rewrites Long reasoning, research, code generation

Most real products end up using both: the device for quick, frequent, privacy-sensitive tasks, and the server for the heavy ones.

Production checklist

Before you ship, walk through this list. Each item is something that is easy to miss in a demo and painful to discover in production.

  1. Feature-detect every API separately. Having the Summarizer doesn't mean having the Translator. Check each constructor and each availability().
  2. Only call create() from user input. Downloads need user activation, and nobody wants a surprise multi-gigabyte download on page load.
  3. Show download progress. A <progress> element bound to downloadprogress events is enough. Without it, the first click looks broken.
  4. Send text, not markup. Use innerText or your own plain-text source. It saves input space and improves results.
  5. Reuse sessions. Create one per configuration and keep it, rather than one per click. Destroy Prompt API sessions you no longer need.
  6. Handle short input in language detection. Treat low-confidence results as unknown.
  7. Treat model output as untrusted. Render it with textContent, not innerHTML. If you summarize text that users wrote, that text can contain instructions aimed at the model, so never let model output trigger actions without validation.
  8. Know the iframe rule. By default the APIs work only in top-level windows and same-origin iframes. A cross-origin iframe needs permission from the parent, for example <iframe src="https://widget.example.com/" allow="summarizer"></iframe>.
  9. Don't plan on Web Workers. The Summarizer documentation states the API isn't available in Web Workers at this time, so keep calls on the main thread and keep them async.
  10. Test the real states. On a supported machine, chrome://on-device-internals shows the model's state, and running LanguageModel.availability() in the DevTools console should return available once the model is ready.

What's next: WebMCP and the agentic web

Everything above is about your page using a model. The next shift runs the other way: a model, usually an AI agent inside the browser, using your page.

At Google I/O 2026, Chrome's team talked about an "agentic web", where sites declare structured tools that agents can call, instead of agents clicking through the visual interface and guessing. The proposal for this is called WebMCP. It borrows its ideas from the Model Context Protocol: a page registers tools with a name, a description and a JSON Schema for the input, and an agent calls them like functions.

It is important to be precise about its status, because it is moving quickly:

  • It is an origin trial, not a stable API. You need to register for the trial to use it on a live site.
  • The entry point has already moved once. Early versions exposed tools through navigator.modelContext. The specification has since moved this to document.modelContext, and code written against the old location is already out of date.
  • Other browsers haven't shipped it. Treat it as Chrome-only for now.

Because of that, this guide doesn't teach a full WebMCP implementation; any code here could be wrong within a few months. What you can do today is get ready for it, and every step makes your site better for people as well:

  • Make your important actions available as clean functions in your front-end code ("search the docs", "add to cart"), not only as click handlers. Those functions become tools later with a few lines of registration code.
  • Describe your forms and buttons properly with labels and accessible names. Agents that can't call tools fall back to reading the page the way a screen reader does.
  • Keep your structured data (JSON-LD) accurate. Agents read it.

When WebMCP stabilizes, you will feature-detect document.modelContext the same way you detected Summarizer in Step 1, and register the functions you already have.

Summary

Chrome's built-in AI APIs change the cost model of small AI features on the web. Summaries, translation and short structured prompts can now run on the user's own device, with no key, no server and no per-request cost.

The pattern that makes them work in production is the same one the web has always used for new capabilities:

  • Detect the API and check availability().
  • Offer the feature when it can work, and explain the one-time download.
  • Create sessions from user actions, reuse them, and show progress.
  • Fall back to a server, or to no AI, everywhere else.

Start with the Summarizer, because it is stable, narrow and useful on almost any content site. Add the Language Detector and Translator for user-generated content. Reach for the Prompt API when you need structured output, and check its status for your audience first. Then keep an eye on WebMCP, which will let agents use the same functions your interface already calls.

If you are planning the bigger picture of where AI fits in a modern stack, our full-stack roadmap with React and vector databases covers the server-side half of that story.