To be fair, it's "possible" to run such setup with llama.cpp with ssd offload. I... | Hacker News

Hacker Newsnew | past | comments | ask | show | jobs | submit

3abiton 3 months ago | parent | context | favorite | on: iPhone 17 Pro Demonstrated Running a 400B LLM

To be fair, it's "possible" to run such setup with llama.cpp with ssd offload. It's just abysmal TG speeds. But it's possible.

Consider applying for YC's Fall 2026 batch! Applications are open till July 27.
Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact