Introduction

Run AI on your users' devices

Natif is an SDK that runs language, voice and image models on the user's own device (browser today, React Native next). It decides, for every call, which model the device can handle, runs it in a Web Worker, validates the output against your schema, and falls back to a cloud model only when the device can't and the user agreed.

Natif is in early access. APIs can change before 1.0.

Install

npm i natif zod

Requirements: a browser with WebGPU (Chrome/Edge 113+, Safari with WebGPU enabled) for on-device text and image models. Voice runs on WebAssembly and works without a GPU.

Quickstart

import { createClient } from "natif";
import { z } from "zod";

const ai = createClient({
  tasks: {
    "daily-checkin": { systemPrompt: "Score the user's wellbeing check-in from 1 to 10." },
  },
  onDownloadProgress: (p) => console.log(p.phase, Math.round(p.progress * 100) + "%"),
});

const CheckIn = z.object({
  mood: z.number().int().min(1).max(10),
  stress: z.number().int().min(1).max(10),
});

const { data, meta } = await ai.extractWithMeta({
  task: "daily-checkin",
  schema: CheckIn,
  input: { text: "Slept badly, lots of anxiety about work." },
});
// data → { mood: 3, stress: 8 }
// meta → { runtime: "device", modelId: "qwen3-4b", latencyMs: 2010, estimatedSavingsUsd: 0.00004 }

The first call downloads the model (~2 GB), stores it in the browser cache and measures the device speed. Later calls load from cache in a few seconds.

Use with your AI assistant

The fastest way to add benatif: let your coding assistant do it.

The MCP server gives the assistant five read-only tools: get_started, get_docs, list_models, recommend_model (which model fits a device) and design_task (generates a zod schema, task and typed function). The same guide is plain text at /llms-full.txt.

claude mcp add Natif -- npx -y @natif/mcp

React Native

npm i @natif/react-native zod llama.rn @dr.pogodin/react-native-fs \
  @react-native-async-storage/async-storage react-native-device-info
cd ios && pod install

Same API as the web: createClient, extract, chat. Models run with llama.cpp (Metal on iPhone) as GGUF files downloaded once into the app's Documents folder. Expo needs a development build: add "llama.rn" to plugins and iOS deploymentTarget ≥ 16.4.

Option
availableMemoryMBYour reading of the RAM the app can use now (iOS os_proc_available_memory, Android availMem). Best signal for choosing a model that won't get the app killed. Without it, the GPU working-set limit or total RAM is used.
unloadOnBackgroundDefault true: frees the model when the app goes to the background; it reloads from disk in seconds.
llama{ nCtx, nGpuLayers }. Default 4096 tokens of context, all layers on the GPU.
filesCustom model storage (e.g. expo-file-system).

Ask before downloading on cellular data: models are 1–2.5 GB.

Tasks

A task is the prompt and settings for one job. Define tasks in code (above) or publish them from the dashboard so you can change prompts without shipping a new app version. Published tasks are cached with an ETag and keep working offline.

FieldDefault
systemPromptrequiredInstructions, ideally with an explicit rubric for consistent scores.
temperature0On-device only. 0 gives repeatable results.
maxTokens256Max output tokens.
minDeviceTokPerSec8Below this speed the device is "too slow" and the next model (or the cloud) is used.
recommendedModel—Try this catalog model first if it fits.
deviceOnlyfalseNever send this task's input to the cloud.

Model selection

Each call picks the best model from a curated catalog that fits the device. Only licenses that allow commercial use without extra terms for your users are included.

ModelTierMemoryCapabilitiesLicense
Qwen3-4Bhigh3.4 GBtextApache-2.0
Gemma 4 E2Bhigh3.4 GBtext, visionApache-2.0
Qwen3-1.7Blow2.0 GBtextApache-2.0

Order of checks: WebGPU available → fits the memory budget → enough storage (skipped if already downloaded) → local benchmark fast enough → anonymous benchmarks from similar devices. If the chosen model fails to load or is too slow, the next one is tried.

Memory budget

Natif budgets on memory the app can use, not on what the machine has installed, so it never slows down the user's computer. A 64 GB laptop with 10 GB free gets a 6 GB budget.

SourceBudget
memoryBudgetMB you passexactly that
Available RAM reported by the OS (React Native)60%
Browsers (they only expose total RAM, if anything)50% of total, never more than 4 GB

Browsers can't read free memory, so on the web set memoryBudgetMB if you know your audience. Models are freed after idleUnloadMs without use (default 2 minutes) and stay on disk; call ai.unload() to free them right away.

Cloud fallback

When the device can't (no WebGPU, out of memory, too slow, invalid output after one repair, timeout), the same call can run in the cloud with the same schema. It happens only if:

  • the user consented: consent: { cloudFallback: true },
  • the project and the task are not marked device-only (enforced on the server),
  • the project has budget left this month.

Otherwise you get a typed DeviceUnavailableError with the reason. meta.runtime and meta.fallbackReason tell you where each call ran.

Loading UX

The first download is the slow part (about 1–2 minutes on fast Wi-Fi). Treat it like a game's loading screen: call ai.prepare() during onboarding and show onDownloadProgress. On a metered connection, ask first.

await ai.prepare({ task: "daily-checkin" }); // download + benchmark ahead of time
const caps = await ai.capabilities(); // what this device can do, without downloading

createClient(options)

OptionDefault
apiKey—Public project key. Needed for dashboard tasks, telemetry and the cloud fallback.
baseUrl—Your Natif API URL.
tasks—Tasks defined in code.
capabilities["text"]What your app needs.
preload"lazy""eager" downloads at init.
runtime"auto""device-only" or "cloud-only".
consentall false{ cloudFallback, telemetry }. Change later with setConsent.
memoryBudgetMBderivedMax memory a model may use.
idleUnloadMs120000Free the model after this long unused. 0 = never.
timeoutMs60000Abandon a device run after this long.
onDownloadProgress—{ modelId, progress, phase }

extract / extractWithMeta

ai.extract({ task, schema, input: { text }, signal? }): Promise<z.infer<typeof schema>>
ai.extractWithMeta(…): Promise<{ data, meta }>

meta: modelId, runtime ("device" | "cloud"), taskVersion, latencyMs, tokensIn, tokensOut, estimatedSavingsUsd, fallbackReason?, repaired?.

Output is generated under your JSON Schema (constrained decoding), parsed, and validated with your zod schema. If validation fails, the model gets one repair attempt.

chat

let history = []; // your app owns it (state, storage, DB…)

const r = await ai.chat({
  system: "You are a friendly barista.", // keep it identical across turns
  history,
  message: userText,
  onToken: (d) => out.textContent += d, // streaming
  signal, // AbortController to stop
});
history = r.history; // history + the new message + the reply

meta includes decodeTokPerSec, ttftMs, cachedTokens (when the engine reports it) and droppedMessages. Chat runs on the device only for now; without a usable GPU it throws DeviceUnavailableError.

Performance is shared work. The SDK reuses the engine's cache when the new prompt starts like the previous one, trims history in blocks (maxHistoryMessages, default 20) so that cached prefix survives several turns, and frees the model when idle. Your app should keep the system prompt identical, keep history append-only (don't edit old turns), keep maxTokens small, summarize very long chats into a fresh history, and call ai.prepare() early so the first message isn't the slow one.

Errors

ErrorWhen
DeviceUnavailableErrorThe device can't run it and the cloud isn't allowed. Has reason.
SchemaValidationErrorOutput didn't match the schema after the repair attempt.
CloudFallbackErrorThe cloud call failed (rate limit, budget, upstream). Has status.
TaskNotFoundErrorUnknown or unpublished task.

HTTP API

Endpoint
GET /v1/tasks/:slugPublished task config (ETag) plus a recommendation for this device class.
POST /v1/fallback/extractCloud fallback. Uses the published prompt only; rate-limited and budgeted per project.
POST /v1/telemetryBatched anonymous metrics (sendBeacon). Unknown fields are dropped.

Every request carries the public key in x-api-key. Keys are restricted to allowed origins (web) or app ids (mobile), and can require Firebase App Check.

Privacy

DataLeaves the device?
User input and resultsNo. Only in a consented cloud fallback; not stored by Natif. The model provider applies its own retention policy.
TelemetryOnly with consent. Closed list: model, latency, tokens, outcome, device class, SDK version. Kept 90 days.
Device classHash of coarse traits (browser major, GPU family, memory bucket). Used only with ≥ 20 devices per class.

Browsers

BrowserText / imageVoice / video
Chrome, Edge (desktop)Yes (WebGPU)Yes
Safari (macOS)With WebGPU enabledYes
FirefoxPartial (WebGPU rolling out)Yes
PhonesSmaller models; memory limits per tabYes