Local LLMs

Community reports say the Strata engine can run a 125B Qwen model on a single RTX 4090, with speeds ranging from about 62 to 166 tokens per second depending on setup. Users praise the cost savings, but independent measurements and quality tests show the headline numbers are contested.

From the stories

Everything You Know About Local AI is OUTDATEDMaddy Zhang

Strata puts a 125B Qwen on a single RTX 4090

The Show HN for Strata pitched Qwen 3.8 Flash Next (125B) at 100 tokens/sec on consumer hardware and pulled 940 points and 420 comments on Hacker News. The submitter, snehesht, reported 124 tok/s on a 4090 with 128GB DDR5. Commenters pushed back: Jackson__ measured a median 154.8-pixel error on a 50-image vision test, and a11r said he stays wary of going below 4-bit quants. Over on r/LocalLLM, u/_JohnMcAfee said Strata got him to buy an old x299 workstation with "16gb vram total" and "it has replaced deepseek api for me" (21 upvotes). The spread of reported numbers is wide: a darkerow benchmark logged 166 tok/s on a 4090 with IQ2_XS quantization but the actual coding test, building a hill-climbing game, failed to produce a playable result. A separate open-lowmem launcher repo measured 62-94 tok/s across 8 runs on a 4090D with 32GB RAM, a median around 77, well under the headline 100 tok/s. llamaperf's Strata page aggregates these community reports directly, and its broader RTX 4090 page shows one user running Qwen3.8-Flash-Next at around 122 tok/s in regular chat at 64K context while weighing a $700 RAM upgrade against a $3,100 Strix Halo box. A Kunal Ganglani walkthrough frames the gap as a KV-cache tuning problem: the same setup can hit ~100 tok/s or OOM depending on those settings. A Korean-language teardown traces one viral 94 tok/s claim back to a GPU swap, the headline cited an RTX 4090 but the original source measured a different card. On Hugging Face Forums, AIJUUD's Juud_engine fork of Strata v0.1.39 ran its own comparison against pinned Strata on a single 4090, generating 256 tokens per prompt across four code contexts up to 4K.

One r/LocalLLM commenter says RAM he bought for $120 now costs $550

That line lands in the "Is local AI becoming a luxury hobby" thread (45 points, 150 comments). The counter-thread on r/LocalLLaMA, "You don't need that expensive of a build," hit 449 points, though replies grumbled "A 9060 xt 16gb is $500+. That's expensive!" (45 upvotes). In the big-purchase thread, the top advice is "64 GB Mac Studio owner here. Get the 128GB" (98 upvotes). Maddy Zhang's 218,893-view video adds the time cost: if a local model takes 20 minutes on a task a cloud model finishes in 20 seconds, it "just cost you your entire afternoon." A Presenc AI cost comparison tries to put numbers on this debate, modeling total cost of ownership for DGX Spark, Mac Studio, and consumer GPU setups against per-token cloud pricing and finding the breakeven point shifts heavily with model size and utilization. Hardware comparisons back up the "it depends" framing: llamaperf's DGX Spark vs RTX 4090 page shows the Spark at 59.1 tok/s versus the 4090 at 149.6 tok/s on Gemma 4 26B, while its RTX 4080 Super vs 4090 comparison shows a closer 59.0 versus 67.0 tok/s gap on Qwen3.8 27B at 2-bit, all flagged as community reports rather than controlled benchmarks.

"I think local AI cost me a job" hits 719 points on r/LocalLLaMA

The thread (440 comments) turned into a debate about how local AI reads to employers. The top reply was blunt: "Should've just said local AI as a hobby, then opus 5.5 in some complicated harness, because that's all they want to hear" (862 upvotes). u/NoFunk offered a pitch built around "I use the right tool for the right job" (133 upvotes). The title does appear on job boards: Parallel Wireless has a Senior/Principal Local LLM & Generative AI Platform Engineer listing on Lever, dated September 9.

Janas claims an 80B model at 37 tok/s on a 32GB laptop

That github.signals clip drew 29,286 views and 1,238 likes. The same account pitched FreeToken running 290B-parameter models on a gaming PC, which skip_ci frames as a README claim. raymundoojeda1 posted DeepSeek V4.1 Flash at 102 tok/s on two DGX Sparks (46,133 views), and morningcoffeebyte showed a ternary 30B MoE fitting a 12GB GPU at 172 tok/s (17,292 views). All of these numbers are self-reported by the posters.

Llmspeed lets you feel tokens/sec before you buy, and HN argues with its numbers

The Llmspeed Show HN got 331 points and 86 comments on October 17. kgeist said the site lists 46 tok/s for Qwen3.6-27b 4-bit on an RTX 5090, while he gets 60-70 on small contexts and about 40 on large ones. BenchLM's Best Local LLMs ranking, as of October 25, puts Qwen3.6-27B first at 48.53. @thesummaryseries made the same point in a 65-view promo for the book Local LLMs in Production: "A public benchmark cannot tell you whether a local model will work in your system."

oMLX by @jundotkim ships as an engine inside LM Studio and Bionic

@yagilb announced it on October 29 (51 likes, 2,471 views) and tagged @lmstudio as a partner. An r/macbookpro post claims an M5 Max runs Qwen 35B at 346 tok/s and a 180B model at 108 tok/s (23 upvotes). Apple's MLX distributed inference on Mac Studio page also surfaced on HN. On the cross-platform side, llama.cpp sits at 133,262 stars and cut release b12074 on November 3.

chiefgyk3d runs Gemma 4 12B locally under a Claude Opus orchestrator

His Instagram reel (21,461 views, 875 likes) says routing work to a local server with 128k context "cut down on my Claude usage dramatically." The limits showed up on HN too: in the XDA "I'm not paying $20" thread (52 points), mongrelion wrote that "16K of context is nothing for agentic coding tasks." themergelog pitched OpenCode as a coding agent that works with 75+ providers, including local models. @observerai posted a tutorial on running local LLMs on a phone.