I recently got hold of two RTX 3090 GPUs specifically for LLM inference and training. Everything seems to work well and I can finally fit a 70B model into the VRAM with 4 bit quantization. I am wondering if …

I knew the 3090 would win, but I was expecting the 3060 to probably have about one-fifth the speed of a 3090; instead, it had half the speed! The 3060 is completely usable for small models.

Jul 7, 2023 · People seem to consider them both as about equal for the price / performance. I know 4090 doesn't have any more vram over 3090, but in terms of tensor compute according to the specs …

Jun 19, 2023 · 3090 is faster than 4070, especially at the 4K, plus more vram which matters at 4k. The only downside is the TDP, 3090 is a power hag. Reply reply kou07 •

Dec 15, 2022 · Upgrade from RTX 3090 Hey guys, hope you are doing fine So, I bought a 3090 for a very good price a month ago to my new build with an i7-13700kf, the doubt is, I mainly play the new …

Jan 4, 2022 · I have a PNY 3090 Revel and I was never happy with the temps so I looked into undervolting. Didn’t realize how easy it is. I managed to settle with 1890mhz/850mv with a -270 core …

Sep 18, 2023 · I was reading on another subreddit about a gent (presumably) who added another 8GB chip to his EVGA 3070, to bring it up to 16GB VRAM. In the comments, people were discussing the …

Jun 20, 2023 · I was looking at all the benchmarks and although most of the 4xxx series cards are marginal improvements on 3xxx series, the 4090 looks to be anywhere from 50% to 80% …

Power and case capacity is fairly easy for dual 3090's or 4090's as well. Also, the high end models tend to be tuned around having 48GB VRAM. You can fit a 70b 5bpw with 32k context in that or go with a …

Sep 24, 2020 · The real RTX 3090, meanwhile, gaming focussed first and foremost, ought to arrive with 12GB of memory and a $999 price point, thereby offering a compelling upgrade without resorting to …