A 20B AI model running on an RTX 5070 Ti with 16GB GDDR7 VRAM was no problem, but one user decided that this LLM was too limiting for his daily workflow. Instead, he opted to switch to a 35B one, and to help offload the model without experiencing performance issues, he leveraged the use of 128GB of DDR4 ECC memory from a workstation. Unfortunately, just 90 minutes in, one DDR4 stick mysteriously dies, with the user claiming that thermals didn’t kill the RAM. Surprisingly, the user only mentions CPU and GPU temperatures, as a second DDR4 stick started showing errors […]
Read full article at https://wccftech.com/35b-llm-database-workload-destroys-ddr4-ram-stick-90-minutes/