alt Hacker News1B vectors is nothing. You don’t need to index them. You can hold them in VRAM on a single node and run queries with perfect accuracy in milliseconds
Replies
adastra22 • yesterday at 9:35 PM
1B x 4096 = 4T scalars.
That doesn't fit in anyone's video ram.
➕ show 1 reply
I guess for 2D vectors that would work?
For 1024 dimensions even with 8 bit quantization you are looking at a terrabyte of data. Lets make it binary vectors, it is still 128GB of VRAM.
WAT?