# JOIN_Earth — research verdict: is free local AI for everyone real?

**Source:** 5-agent web-research workflow (wf_b869da0a), 2026-07-02, sourced + honest-state.
**Answers:** Robert's "can it run on a shitty Nokia?" and "why can't we make a 1GB gemma4?"

## TOP — the verdict in one line
**The literal Nokia/feature-phone dream is dead (512MB can't run any useful LLM). The honest version is
REAL today: a ~$60–150 Android phone (3–4GB RAM, no NPU, no GPU) runs a 0.5–1B model (~1GB) at 8–14
tok/s, or BitNet b1.58 2B4T ternary at a 0.4GB footprint that BEATS the 1B FP models on quality.**

## Your 1GB question, answered with numbers

- A **0.5–1B model at Q4 ≈ 1GB** and runs on cheap Android: first-hand test = Qwen2.5-0.5B Q6 at 8–10
  tok/s on a $150/4GB phone using **~740MB**. Llama-3.2-1B / Gemma-3-1B Q4 are the "daily-driver" sweet spot.
- **BitNet b1.58 2B4T** (native ternary, 1.58-bit) is the star: **0.4GB** non-embedding footprint,
  CPU-only, and it **beats Llama-3.2-1B and Gemma-3-1B** (MMLU 53.2, GSM8K 58.4). This is the real path
  to a sub-1GB brain — but only because it's *trained* ternary. You still cannot crush gemma4-8B to 1GB;
  you use a small sibling or a natively-ternary model.
- ARM kernels make it CPU-friendly: bitnet.cpp = 1.37–5.07× speedup + 55–70% energy cut on ARM.

## The three hard ceilings (physics — "overnight" doesn't beat them)

1. **Memory bandwidth (binding).** Decode speed = bandwidth ÷ model-bytes. Entry LPDDR4X ≈ 17 GB/s →
   a 1GB model caps at ~10–14 tok/s. Overnight batching accumulates output but does NOT raise per-token
   speed for one interactive user. More TOPS on the box can't beat the bandwidth wall.
2. **Thermal throttle.** Sustained inference on a passively-cooled phone: 8 tok/s observed collapsing to
   **1.2 tok/s after 90s**. Plugging in fixes battery, not heat.
3. **OS process killers.** Android Low Memory Killer + Android 12+ Phantom Process Killer terminate
   long-running CPU-heavy processes — hostile to overnight batch. Opening WhatsApp can kill your run.

The floor is a **cheap real smartphone (~3GB+ RAM), not a feature phone.** "Useful" = narrow tasks
(short Q&A, summarize, autocomplete, simple assistant), not a do-everything frontier-quality bot.

## The recommended architecture — a 3-tier composite behind ONE `/v1` seam

Everything speaks OpenAI-compatible `/v1`, so tier selection is *routing, not a fork*:

- **TIER 0 — on-device SLM ($0 default, ~80–90% of use):** device-adaptive — BitNet-2B/0.5–1B Q4 for
  cheap 3–4GB Android; 3–4B Q4 for 8GB+; WebLLM ~8B in-browser. Free, offline, private.
- **TIER 0.5 — personal "you bot" via RETRIEVAL (ship now):** EmbeddingGemma-308M + sqlite-vec +
  Mem0-style fact extraction. **RAG beats fine-tuning +14.92% vs +1.07%**, avoids catastrophic
  forgetting, wins cold-start. NEVER fine-tune. Validated at scale (Apple Intelligence, Gemini memory).
  **This is exactly what `tools/evolution/convo_distill.py` already does — the research confirms the
  design.**
- **TIER 1 — donor mesh (heavy-tail fallback, explicitly NON-private):** Petals/Parallax run 70B–405B
  on volunteer GPUs, ~8–10 tok/s IF regional + speculative decoding. Four ceilings keep it a fallback,
  never default: latency (must stay regional — speed-of-light floor), **privacy (a donor reconstructs
  the prompt from activations >90% — never send sensitive data)**, trust (verification unsolved),
  economics (volunteer supply decays). Opus's donate-compute path is real but bounded.
- **TIER 2 — BYOK frontier power-up:** one settings field for the user's own key, explicit
  "this leaves your device" notice at escalation. Keeps "free forever" true (frontier is user-funded)
  and sovereignty honest.

**Free-forever holds** because Tier 0/0.5 are $0-marginal and the frontier tier is user-funded (BYOK),
never vendor-subsidized. Local-default, sovereign-by-default.

## Build path (all "now")
1. Ship the OpenAI-compatible `/v1` client seam (local-vs-anything = base_url + key swap, not a fork).
2. Bundle Tier 0 device-adaptive SLM (BitNet-2B / 0.5–1B Q4 / 3–4B / WebLLM).
3. Android anti-kill + thermal survival: persistent foreground notification, chunked generation with
   thermal backoff, charge-aware overnight batching.
4. Tier 0.5 you-bot as pure retrieval (EmbeddingGemma + sqlite-vec + Mem0) — extend `convo_distill.py`.
5. Tier 2 BYOK field with the always-there "make a key" link + boundary notice.

## Honest caveats (no hype)
Below ~1–3B params there's a genuine quality cliff — a 1GB bot hallucinates more and is narrow, not a
frontier peer. "Knows you AND as smart as a frontier model, fully offline" is NOT solved — expect
personalized-but-modest. The personal index must be *curated, not dumped in* (small models can't
recover from a messy multi-hop index). Truly universal reach needs a sub-1B fallback that feels weaker
on the cheapest phones.