A 120B LLM like gpt-oss-120b would require a minimum of 60GB to 80GB of a combination of VRAM and system memory to run at decent performance, meaning that it’s absolutely necessary to have pricey hardware in your possession to achieve this feat. However, one ambitious person managed to accomplish this task with a combination of low-end machines and an Android smartphone, showing that even on a smaller budget, there’s no limit to where your creativity can take you. Despite achieving the impossible, a cluster of devices running the 120B LLM will deliver insanely slow speeds, making it an impractical experiment […]
Read full article at https://wccftech.com/120b-llm-phone-macs-windows-pcs-distributed-setup/