Do not buy a 4 GB card for this. Mine is an RTX 3050 Laptop, I picked it because it
is boring hardware that many people already have.
If you are buying, buy VRAM. At 0.5B where I could measure both, resident training
was 1.43x faster than streaming. And if you do stream, system RAM matters more, the
base sits there and has to page-lock. 16 GB is about the floor for 8B.
alt Hacker News
Do not buy a 4 GB card for this. Mine is an RTX 3050 Laptop, I picked it because it is boring hardware that many people already have. If you are buying, buy VRAM. At 0.5B where I could measure both, resident training was 1.43x faster than streaming. And if you do stream, system RAM matters more, the base sits there and has to page-lock. 16 GB is about the floor for 8B.