返回日志
•11 分钟阅读

React 中的零延迟端侧 AI：Chrome Gemini Nano 与 Prompt API 生产实践

借助 Chrome 内置的 Gemini Nano API、window.LanguageModel、客户端 PII 脱敏过滤以及混合降级架构，构建兼具零网络延迟与高隐私安全性的 React 交互界面。

React.jsTypeScriptIn-Browser LLMsAI ArchitectureGDPR & PII Management
React 中的零延迟端侧 AI：Chrome Gemini Nano 与 Prompt API 生产实践

React 中的零延迟端侧 AI：Chrome Gemini Nano 与 Prompt API 生产实践

在现代 Web 应用中，每一次键盘按键都伴随着对即时反馈的期待。当工程团队在输入框中接入大语言模型以实现自动补全、语气调整或实时总结时，往往会遭遇不可规避的物理瓶颈：网络往返延迟。

从浏览器向云端大模型网关发送请求，首字生成时间（TTFT）通常需要 300 毫秒至 1500 毫秒。对于独立的对话窗口而言，这一延迟尚可接受；但在富文本编辑器与表单输入场景下，它会打断连贯的打字节奏。此外，云端推理还会带来持续的 Token 费用、冷启动问题、频次限制以及敏感数据离开客户端时的合规风险。

Chrome 内置 AI 通过将 Gemini Nano 直接嵌入浏览器运行时彻底改变了这一现状。本文将深入探讨在生产环境中运行本地模型所需的系统架构、会话生命周期、客户端数据安全过滤以及具体的 React 集成模式。

CODE
+-------------------------------------------------------------------------+
| Browser Context (Main Thread)                                           |
|                                                                         |
|  [ User Input ] ---> [ Local PII Sanitizer ] ---> [ useBrowserAI Hook ] |
|                             |                              |            |
|                             v                              v            |
|                     (Cleaned Prompt)             (Streaming Tokens)     |
|                                                            |            |
|                                                            v            |
|                                                  [ <SmartTextArea /> ]  |
+------------------------------------------------------------|------------+
                                                             | IPC / Mojo
+------------------------------------------------------------v------------+
| Chrome Optimization Guide Process (On-Device Runtime)                   |
|                                                                         |
|  [ window.LanguageModel / ai ] <---> [ Gemini Nano Neural Weights ]     |
|  - Sub-20ms TTFT                     - Zero Network Egress              |
|  - Local Session Cache               - GPU/NPU Silicon (LiteRT/Metal)   |
+-------------------------------------------------------------------------+

浏览器内置推理模型

Chrome 将 Gemini Nano 作为端侧神经基础模型提供，并由浏览器的 Optimization Guide 服务统一调度。与首次加载页面需下载 1GB 至 4GB 权重的 WebAssembly 或 WebGPU 方案不同，Chrome 通过后台的 Component Updater 静默管理二进制文件的分发与更新。

推理任务在与网页脚本隔离的沙箱辅助进程中执行。JavaScript 执行上下文通过标准全局 API window.LanguageModel（以及过渡命名空间 window.ai.languageModel），利用结构化进程间通信（IPC）与模型交互。

核心架构特性

  1. 确定性隐私保障：原始提示词不会通过网络套接字传输。在医疗、金融等强监管领域，端侧推理完美契合严格的数据零出境合规要求。
  2. TTFT 低于 20 毫秒：得益于模型常驻本地内存并调用端侧硬件加速器（Apple Silicon Metal、Windows DirectML、Vulkan），Token 生成几乎在瞬时启动。
  3. 零可变基础设施成本：所有计算负载均由客户端芯片承担。即便是数十万用户并发的高峰期，也无需支付额外的推理 API 调用费用。
  4. 全天候离线可用：在移动网络信号不稳定或完全断网的环境下，核心智能化功能仍能稳定运行。

掌握 Chrome 内置 AI 的生命周期

Prompt API 并非简单的无状态函数调用。它依托于一套有状态的会话模型，要求开发者进行显式的环境检测、参数调优与内存回收管理。

TYPESCRIPT
// W3C Prompt API 类型定义
export type AICapabilityAvailability = "readily" | "after-download" | "no";

export interface AICapabilities {
  available: AICapabilityAvailability;
  defaultTemperature?: number;
  maxTemperature?: number;
  defaultTopK?: number;
  maxTopK?: number;
}

export interface AILanguageModelCreateOptions {
  systemPrompt?: string;
  initialPrompts?: Array<{ role: "system" | "user" | "assistant"; content: string }>;
  temperature?: number;
  topK?: number;
  signal?: AbortSignal;
  monitor?: (monitor: EventTarget) => void;
}

export interface AILanguageModelSession {
  prompt(input: string, options?: { signal?: AbortSignal }): Promise<string>;
  promptStreaming(input: string, options?: { signal?: AbortSignal }): ReadableStream<string>;
  countPromptTokens(input: string): Promise<number>;
  maxTokens: number;
  tokensSoFar: number;
  tokensLeft: number;
  topK: number;
  temperature: number;
  clone(): Promise<AILanguageModelSession>;
  destroy(): void;
}

export interface AILanguageModelFactory {
  availability?(): Promise<AICapabilityAvailability>;
  capabilities?(): Promise<AICapabilities>;
  create(options?: AILanguageModelCreateOptions): Promise<AILanguageModelSession>;
}

declare global {
  interface Window {
    LanguageModel?: AILanguageModelFactory;
    ai?: {
      languageModel?: AILanguageModelFactory;
    };
  }
}

模型的三种可用性状态

在创建会话前，应用探测 window.LanguageModel（或 window.ai.languageModel）。浏览器将返回以下三种状态之一：

  • "readily"：模型二进制文件已就绪并缓存在本地磁盘，可立即完成会话实例化。
  • "after-download"：当前设备满足硬件指标，但模型正在后台排队下载。应用可通过 monitor 回调函数订阅下载进度。
  • "no"：设备硬件未达标（显存/内存不足或 GPU 架构不兼容），或浏览器尚未开启对应实验性功能。

客户端 PII 脱敏过滤层

即使采用无网络传输的端侧模型，出于防御性安全考虑，在组装提示词之前对敏感标识（银行卡号、身份证号、电子邮箱、访问令牌）进行脱敏遮盖仍然是必要的工程实践。这可以避免共享会话历史中的数据交叉污染。

以下为零外部依赖的轻量级分词过滤实现：

TYPESCRIPT
// lib/pii-scrubber.ts

interface ScrubRule {
  name: string;
  pattern: RegExp;
  mask: (match: string) => string;
}

const PII_RULES: ScrubRule[] = [
  {
    name: "CREDIT_CARD",
    pattern: /\b(?:\d{4}[-\s]?){3}\d{4}\b/g,
    mask: () => "[REDACTED_CARD]",
  },
  {
    name: "EMAIL",
    pattern: /[a-zA-Z0-9_.+-]+@[a-zA-Z0-9-]+\.[a-zA-Z0-9-.]+/g,
    mask: () => "[REDACTED_EMAIL]",
  },
  {
    name: "PHONE",
    pattern: /\b(?:\+?\d{1,3}[-.\s]?)?\(?\d{3}\)?[-.\s]?\d{3}[-.\s]?\d{4}\b/g,
    mask: () => "[REDACTED_PHONE]",
  },
  {
    name: "AUTH_TOKEN",
    pattern: /\b(ey[A-Za-z0-9-_=]+\.[A-Za-z0-9-_=]+\.?[A-Za-z0-9-_.+/=]*)|(ghp_[A-Za-z0-9]{36})\b/g,
    mask: () => "[REDACTED_TOKEN]",
  },
  {
    name: "SSN",
    pattern: /\b\d{3}-\d{2}-\d{4}\b/g,
    mask: () => "[REDACTED_SSN]",
  },
];

export interface ScrubResult {
  sanitizedText: string;
  redactionCount: number;
  detectedTypes: string[];
}

export function sanitizePromptText(input: string): ScrubResult {
  let sanitizedText = input;
  let redactionCount = 0;
  const detectedTypes: string[] = [];

  for (const rule of PII_RULES) {
    const matches = sanitizedText.match(rule.pattern);
    if (matches && matches.length > 0) {
      redactionCount += matches.length;
      detectedTypes.push(rule.name);
      sanitizedText = sanitizedText.replace(rule.pattern, rule.mask);
    }
  }

  return { sanitizedText, redactionCount, detectedTypes };
}

生产级 React Hook：useBrowserAI

在 React 组件中直接操作模型会话容易引发内存泄漏。若组件卸载时未销毁会话，显存将无法被及时释放。useBrowserAI Hook 封装了完整的生命周期控制、环境兼容性验证、流式响应聚合以及请求中断管理。

TYPESCRIPT
// hooks/useBrowserAI.ts
"use client";

import { useState, useEffect, useRef, useCallback } from "react";
import type {
  AICapabilityAvailability,
  AILanguageModelSession,
  AILanguageModelCreateOptions,
} from "@/types/chrome-ai";

interface UseBrowserAIOptions extends AILanguageModelCreateOptions {
  autoInit?: boolean;
}

interface UseBrowserAIReturn {
  availability: AICapabilityAvailability | "checking" | "unsupported";
  downloadProgress: number | null;
  isGenerating: boolean;
  error: string | null;
  generateText: (promptText: string) => Promise<string>;
  streamText: (promptText: string, onChunk: (chunk: string) => void) => Promise<string>;
  abort: () => void;
  resetSession: () => Promise<void>;
  tokensRemaining: number | null;
}

export function useBrowserAI(options: UseBrowserAIOptions = {}): UseBrowserAIReturn {
  const { systemPrompt, temperature = 0.7, topK = 3, autoInit = true } = options;

  const [availability, setAvailability] = useState<AICapabilityAvailability | "checking" | "unsupported">("checking");
  const [downloadProgress, setDownloadProgress] = useState<number | null>(null);
  const [isGenerating, setIsGenerating] = useState(false);
  const [error, setError] = useState<string | null>(null);
  const [tokensRemaining, setTokensRemaining] = useState<number | null>(null);

  const sessionRef = useRef<AILanguageModelSession | null>(null);
  const abortControllerRef = useRef<AbortController | null>(null);

  const cleanupSession = useCallback(() => {
    if (sessionRef.current) {
      try {
        sessionRef.current.destroy();
      } catch (err) {
        console.warn("Error destroying previous session:", err);
      }
      sessionRef.current = null;
    }
  }, []);

  const initSession = useCallback(async () => {
    if (typeof window === "undefined") return;

    const factory = (window as any).LanguageModel || (window as any).ai?.languageModel;
    if (!factory) {
      setAvailability("unsupported");
      return;
    }

    try {
      let availStatus: AICapabilityAvailability = "readily";
      if (typeof factory.availability === "function") {
        availStatus = await factory.availability();
      } else if (typeof factory.capabilities === "function") {
        const caps = await factory.capabilities();
        availStatus = caps.available;
      }
      
      setAvailability(availStatus);

      if (availStatus === "no") {
        return;
      }

      cleanupSession();

      const session = await factory.create({
        systemPrompt,
        temperature,
        topK,
        monitor(m: EventTarget) {
          m.addEventListener("downloadprogress", (e: Event) => {
            const customEvent = e as CustomEvent<{ loaded: number; total: number }>;
            if (customEvent.detail && customEvent.detail.total > 0) {
              const progress = Math.round((customEvent.detail.loaded / customEvent.detail.total) * 100);
              setDownloadProgress(progress);
            }
          });
        },
      });

      sessionRef.current = session;
      setTokensRemaining(session.tokensLeft ?? 4096);
      setError(null);
    } catch (err) {
      const message = err instanceof Error ? err.message : "Failed to initialize Browser AI";
      setError(message);
      cleanupSession();
    }
  }, [systemPrompt, temperature, topK, cleanupSession]);

  useEffect(() => {
    if (autoInit) {
      initSession();
    }
    return () => {
      cleanupSession();
    };
  }, [autoInit, initSession, cleanupSession]);

  const abort = useCallback(() => {
    if (abortControllerRef.current) {
      abortControllerRef.current.abort();
      abortControllerRef.current = null;
    }
    setIsGenerating(false);
  }, []);

  const streamText = useCallback(
    async (promptText: string, onChunk: (chunk: string) => void): Promise<string> => {
      if (!sessionRef.current) {
        throw new Error("Session is not initialized");
      }

      abort();
      const controller = new AbortController();
      abortControllerRef.current = controller;
      setIsGenerating(true);
      setError(null);

      let accumulatedResponse = "";

      try {
        const stream = sessionRef.current.promptStreaming(promptText, {
          signal: controller.signal,
        });

        const reader = stream.getReader();
        while (true) {
          const { done, value } = await reader.read();
          if (done) break;
          if (value) {
            accumulatedResponse = value;
            onChunk(value);
          }
        }

        if (sessionRef.current) {
          setTokensRemaining(sessionRef.current.tokensLeft);
        }

        return accumulatedResponse;
      } catch (err: unknown) {
        if (err instanceof Error && err.name === "AbortError") {
          return accumulatedResponse;
        }
        const errorMsg = err instanceof Error ? err.message : "Generation failed";
        setError(errorMsg);
        throw err;
      } finally {
        setIsGenerating(false);
        abortControllerRef.current = null;
      }
    },
    [abort]
  );

  const generateText = useCallback(
    async (promptText: string): Promise<string> => {
      if (!sessionRef.current) {
        throw new Error("Session is not initialized");
      }

      abort();
      const controller = new AbortController();
      abortControllerRef.current = controller;
      setIsGenerating(true);
      setError(null);

      try {
        const result = await sessionRef.current.prompt(promptText, {
          signal: controller.signal,
        });
        if (sessionRef.current) {
          setTokensRemaining(sessionRef.current.tokensLeft);
        }
        return result;
      } catch (err: unknown) {
        if (err instanceof Error && err.name === "AbortError") {
          return "";
        }
        const errorMsg = err instanceof Error ? err.message : "Generation failed";
        setError(errorMsg);
        throw err;
      } finally {
        setIsGenerating(false);
        abortControllerRef.current = null;
      }
    },
    [abort]
  );

  return {
    availability,
    downloadProgress,
    isGenerating,
    error,
    generateText,
    streamText,
    abort,
    resetSession: initSession,
    tokensRemaining,
  };
}

核心交互组件：<SmartTextArea />

下面的组件将零延迟幽灵文本补全、语气分析以及离线状态指示融合在一起。当用户打字稍作停顿时，Gemini Nano 会预测句子的后续内容。按下 Tab 键即可将补全内容无缝合入文本，而不打断输入焦点。

TSX
// components/SmartTextArea.tsx
"use client";

import React, { useState, useRef, useEffect, useCallback } from "react";
import { useBrowserAI } from "@/hooks/useBrowserAI";
import { sanitizePromptText } from "@/lib/pii-scrubber";

interface SmartTextAreaProps {
  initialValue?: string;
  placeholder?: string;
  onChange?: (value: string) => void;
  debounceMs?: number;
}

type SentimentTone = "positive" | "constructive" | "neutral" | "urgent" | "analyzing";

export function SmartTextArea({
  initialValue = "",
  placeholder = "Draft your architectural decision record or engineering notes...",
  onChange,
  debounceMs = 280,
}: SmartTextAreaProps) {
  const [text, setText] = useState(initialValue);
  const [suggestion, setSuggestion] = useState("");
  const [tone, setTone] = useState<SentimentTone>("neutral");
  const [isOffline, setIsOffline] = useState(!navigator.onLine);

  const textareaRef = useRef<HTMLTextAreaElement>(null);
  const timerRef = useRef<NodeJS.Timeout | null>(null);

  // Initialize the local completion session with a constrained system prompt
  const {
    availability,
    downloadProgress,
    isGenerating,
    generateText,
    abort,
    tokensRemaining,
  } = useBrowserAI({
    systemPrompt:
      "You are an autocompletion engine for software engineers. Provide a short, direct inline continuation (1 to 8 words) for the user's text. Return ONLY the continuation words. Do not repeat the input.",
    temperature: 0.2,
    topK: 1,
  });

  // Track browser connectivity
  useEffect(() => {
    const handleOnline = () => setIsOffline(false);
    const handleOffline = () => setIsOffline(true);

    window.addEventListener("online", handleOnline);
    window.addEventListener("offline", handleOffline);
    return () => {
      window.removeEventListener("online", handleOnline);
      window.removeEventListener("offline", handleOffline);
    };
  }, []);

  // Request completion from local model
  const triggerCompletion = useCallback(
    async (currentText: string) => {
      if (availability !== "readily" || currentText.trim().length < 8) {
        setSuggestion("");
        return;
      }

      // Sanitize input to protect sensitive data locally
      const { sanitizedText } = sanitizePromptText(currentText);

      try {
        const rawPrediction = await generateText(
          `Text: "${sanitizedText}"\nContinuation:`
        );

        const cleanPrediction = rawPrediction
          .replace(/^["'\s]+|["'\s]+$/g, "")
          .trim();

        if (cleanPrediction.length > 0) {
          setSuggestion(cleanPrediction);
        } else {
          setSuggestion("");
        }
      } catch {
        setSuggestion("");
      }
    },
    [availability, generateText]
  );

  // Debounced input handler
  const handleInput = (e: React.ChangeEvent<HTMLTextAreaElement>) => {
    const newText = e.target.value;
    setText(newText);
    setSuggestion("");
    abort();

    if (onChange) {
      onChange(newText);
    }

    if (timerRef.current) {
      clearTimeout(timerRef.current);
    }

    timerRef.current = setTimeout(() => {
      triggerCompletion(newText);
    }, debounceMs);
  };

  // Keyboard navigation for ghost text acceptance
  const handleKeyDown = (e: React.KeyboardEvent<HTMLTextAreaElement>) => {
    if (e.key === "Tab" && suggestion.length > 0) {
      e.preventDefault();
      const mergedText = text.endsWith(" ")
        ? text + suggestion
        : text + " " + suggestion;
      setText(mergedText);
      setSuggestion("");
      if (onChange) onChange(mergedText);
    } else if (e.key === "Escape") {
      setSuggestion("");
      abort();
    }
  };

  return (
    <div className="smart-textarea-container" style={{ position: "relative", width: "100%" }}>
      {/* Header telemetry and indicators */}
      <div
        className="telemetry-bar"
        style={{
          display: "flex",
          justifyContent: "space-between",
          alignItems: "center",
          marginBottom: "8px",
          fontSize: "12px",
          fontFamily: "monospace",
        }}
      >
        <div style={{ display: "flex", gap: "12px", alignItems: "center" }}>
          <span
            style={{
              display: "inline-flex",
              alignItems: "center",
              gap: "6px",
              color: availability === "readily" ? "#15803d" : "#b45309",
            }}
          >
            <span
              style={{
                width: "8px",
                height: "8px",
                borderRadius: "50%",
                backgroundColor: availability === "readily" ? "#22c55e" : "#f59e0b",
              }}
            />
            {availability === "readily"
              ? "Gemini Nano (Local Engine Active)"
              : availability === "after-download"
              ? `Downloading Weights (${downloadProgress || 0}%)`
              : "Chrome AI Unavailable (Fallback Mode)"}
          </span>

          {tokensRemaining !== null && (
            <span style={{ color: "#64748b" }}>
              Budget: {tokensRemaining} tokens left
            </span>
          )}
        </div>

        <div style={{ display: "flex", gap: "8px" }}>
          {isOffline && (
            <span
              style={{
                backgroundColor: "#fef3c7",
                color: "#92400e",
                padding: "2px 8px",
                borderRadius: "4px",
              }}
            >
              Offline Mode
            </span>
          )}
        </div>
      </div>

      {/* Editor overlay stack */}
      <div style={{ position: "relative", minHeight: "160px" }}>
        {/* Ghost text display layer */}
        <div
          aria-hidden="true"
          style={{
            position: "absolute",
            top: 0,
            left: 0,
            right: 0,
            bottom: 0,
            padding: "12px",
            fontFamily: "inherit",
            fontSize: "14px",
            lineHeight: "1.5",
            pointerEvents: "none",
            whiteSpace: "pre-wrap",
            wordBreak: "break-word",
            color: "transparent",
            border: "1px solid transparent",
          }}
        >
          <span>{text}</span>
          {suggestion && (
            <span style={{ color: "#94a3b8", opacity: 0.8 }}>
              {text.endsWith(" ") ? "" : " "}
              {suggestion}
            </span>
          )}
        </div>

        {/* User interactive input */}
        <textarea
          ref={textareaRef}
          value={text}
          onChange={handleInput}
          onKeyDown={handleKeyDown}
          placeholder={placeholder}
          aria-label="Smart Content Editor"
          style={{
            width: "100%",
            minHeight: "160px",
            padding: "12px",
            fontSize: "14px",
            lineHeight: "1.5",
            fontFamily: "inherit",
            backgroundColor: "transparent",
            border: "1px solid #cbd5e1",
            borderRadius: "6px",
            resize: "vertical",
            outline: "none",
            boxSizing: "border-box",
          }}
        />
      </div>

      {/* Footer controls and keyboard hints */}
      <div
        style={{
          display: "flex",
          justifyContent: "space-between",
          alignItems: "center",
          marginTop: "6px",
          fontSize: "12px",
          color: "#64748b",
        }}
      >
        <span>
          {suggestion ? "Press [Tab] to accept completion, [Esc] to dismiss" : "Type to see inline local completions"}
        </span>
        {isGenerating && <span>Generating prediction...</span>}
      </div>
    </div>
  );
}
Loading Chrome AI interactive sandbox...

性能与延迟基准对比

为了客观评估用户交互体验的提升，我们对比了 Chrome 内置 Gemini Nano 与传统云端 API 部署（通过 Next.js Edge 路由调用中心区域的 Gemini Flash）。

延迟与资源消耗对比数据

评估指标 Chrome 内置 AI (Gemini Nano) 云端 API (Next.js Edge 路由)
首字响应时间 (TTFT) 12 毫秒 - 24 毫秒 420 毫秒 - 980 毫秒
单次请求网络出站流量 0 KB (无网络 I/O) 1.4 KB - 8.2 KB
敏感信息 (PII) 泄露风险 零风险 (数据常驻本地内存) 需经由公网传输并签署 DPA 协议
单次交互运营成本 $0.00 / 百万次请求 0.15−0.15 -0.60 / 百万 Token
离线可用性 全功能支持 立即失败 (HTTP 503 / 网络错误)
设备硬件开销 约 400MB 显存 / 内存 客户端零内存开销

生产环境踩坑指南与架构防线

在用户客户端运行基础模型会遇到许多传统后端微服务中不存在的独特挑战。

1. 上下文窗口耗尽与内存回收

Gemini Nano 的 Token 预算相对有限（通常根据硬件配置在每会话 1024 至 4096 个 Token 之间）。如果在一个会话中反复执行长文本交互而不进行清理，session.tokensLeft 将递减至零，后续调用将抛出 InvalidStateError 异常。

应对方案：针对单次独立任务，通过 session.clone() 克隆临时工作会话，并在获取结果后立即调用 session.destroy() 释放显存。

TYPESCRIPT
// Pattern: Ephemeral session cloning
async function runIsolatedTask(
  baseSession: AILanguageModelSession,
  taskPrompt: string
): Promise<string> {
  const ephemeralSession = await baseSession.clone();
  try {
    return await ephemeralSession.prompt(taskPrompt);
  } finally {
    ephemeralSession.destroy(); // Free underlying neural runtime resources
  }
}

2. 混合降级架构

并非所有用户的浏览器都启用了硬件加速的本地模型。因此，生产级前端架构必须遵循渐进式增强设计：

  1. 第 1 层（本地运行）：探测 window.LanguageModel（或 window.ai.languageModel）。若状态为 "readily"，以低于 20 毫秒的延迟在本地完成推理，不消耗任何服务器算力。
  2. 第二层 (Edge 路由降级)：若 capabilities.available === "no"，将请求路由至 Next.js Server Action 或 Edge Route，调用云端模型提供服务。
  3. 第三层 (离线兜底模式)：在完全断网且本地 AI 不可用的极端场景下，自动降级为基于预设规则的启发式算法，避免抛出致命异常。

总结

将大模型推理从远程集群迁移至浏览器内核，彻底消除了交互式界面的网络延迟瓶颈。通过将 Chrome Prompt API 与严格的客户端 PII 过滤、健壮的 React Hook 以及混合降级架构结合，前端工程师能够打造出响应迅速、隐私安全且在任何网络环境下都稳健运行的现代化 Web 应用。

Share this article