Key Takeaways:
A 27-billion-parameter AI model that fits inside a smartphone's memory budget has drawn Apple into early-stage talks with a Caltech spinout, potentially reshaping how the iPhone maker balances on-device intelligence against cloud costs.
PrismML on July 14 released Bonsai 27B, a compressed build of Alibaba's Qwen3.6 27B that reduces each model weight from 16 bits to a single binary value — either +1 or -1 — or to one of three values in its ternary variant. The 1-bit version occupies 3.9GB of memory and runs at 11 tokens per second on an iPhone 17 Pro, making it the first 27-billion-parameter model capable of operating entirely on a consumer smartphone, according to the company.
"This closes the gap between what's possible in the cloud and what can run privately on a device," Babak Hassibi, PrismML's chief executive, told CNBC. "Apple and other companies are evaluating our technology right now."
The compression technique, built on proprietary Caltech intellectual property and backed by $16.25 million in seed funding from Khosla Ventures and Cerberus Capital, applies end-to-end low-bit quantization across every layer of the model — embeddings, attention mechanisms, multi-layer perceptron blocks, and the language model head — with no higher-precision escape hatches. Each group of 128 weights retains a 16-bit scaling factor to anchor output quality, a method the company calls FP16 group-wise scaling. Across 15 benchmarks, the 1-bit variant retains more than 90% of the full-precision model's performance, while the ternary version at 5.9GB retains more than 95%, per PrismML's self-reported results.
Why Apple is paying attention
The timing aligns with Apple's most consequential AI push in years. The company opened the first public beta of iOS 27 on July 13 — one day before PrismML's release — built around a rebuilt Siri assistant that depends on a hybrid of on-device and cloud infrastructure. Apple's own on-device model, AFM 3 Core Advanced announced at WWDC 2026, uses a different approach: a 20-billion-parameter model stored in flash memory that loads only 1 billion to 4 billion parameters into active DRAM per request, rather than keeping all weights active simultaneously.
The engineering tradeoff is stark. Apple's sparse-activation architecture avoids quantization accuracy loss but creates a flash-to-memory bandwidth bottleneck. PrismML's dense-compression approach eliminates that bottleneck entirely — every parameter is always active in DRAM — but introduces an accuracy penalty and may require native chip support to realize full speed gains. Standard CPUs and GPUs must dequantize ternary weights back to higher precision before using their multiply-accumulate hardware, which can negate the computational benefit. Whether Apple's Neural Engine can execute 1-bit operations natively without this overhead remains an open technical question.
Multiple independent reports, citing anonymous sources, indicate that Apple encountered significant performance degradation when it attempted internally to compress its own models to run on iPhones — a failure that reportedly opened the door to PrismML's evaluation.
The financial stakes of on-device AI
The cost incentive for Apple is quantifiable. Bank of America analysts estimated that Apple's most advanced cloud-based model, AFM 3 Cloud Pro, could account for roughly 5% of all Siri AI queries but up to 67% of weighted cloud-computing costs, because of the higher processing intensity required for complex reasoning and image-generation tasks. That model runs on Nvidia GPU hardware inside Google's cloud infrastructure — a dependency that sits uneasily with Apple's privacy marketing and its competitive relationship with Google.
Morgan Stanley has projected that Apple's memory costs could climb sharply in its 2027 financial year, with the bank expecting the company to raise iPhone prices to protect margins. A capable on-device model that reduces the fraction of queries flowing to expensive cloud infrastructure would address both the cost and privacy equations simultaneously.
PrismML's Bonsai 27B is available now under an Apache 2.0 open-source license on Hugging Face, compatible with Apple's MLX framework and Nvidia's CUDA. Hassibi said a compressed version of Google's Gemma model is next in the pipeline, followed by work on models above 27 billion parameters. "Imagine, maybe three years from now, 95% of the intelligence you need is available locally — on your phone, laptop, home appliances — and only that last 5% of high-end needs actually requires the cloud," he told CNBC.
For investors, the implications extend beyond Apple. If on-device inference at this capability level scales, it could reduce demand for cloud GPU infrastructure in consumer-facing AI workloads — currently among the highest-growth segments in the AI data center market. Decentralized compute networks and cloud inference providers would face a shrinking addressable market in the use cases easiest to capture on-device first: private, latency-sensitive, frequently repeated queries. Apple shares, which hit an all-time high on July 14, have gained roughly 20% year-to-date as the market prices in the Siri AI upgrade cycle, but the path to margin expansion may depend on solving exactly the problem PrismML is targeting.
Analysts have urged caution on the unverified claims. Tarun Pathak, research director at Counterpoint Research, said the real test will be "millions of queries, thousands of device combinations and robust testing at scale." Phil Solis of IDC flagged battery life as the potentially defining variable: a model capable enough to be invoked frequently could drain a phone's battery even if individual inferences are efficient. Independent third-party benchmarks have not yet been published.
This article is for informational purposes only and does not constitute investment advice.
