i'm not good with paper work, in fact, i'm horrible with anything that's paperwork related. for the past few days, i ran this model on my rtx 4090 + rtx 3070 and told it to check all the bills, invoices, contracts for me and my small company. i used pi with llama and the pi-llama plugin. oh, boy - i hooked it to my email, told it to download all of the invoices and bills i had for both me and my company and organize them by company/date/ and then merge them with the ones i have locally. it did ocr, wrote scripts, organized everything neatly. i am now the most organized i've ever been in my life. Next: RAG on all the documents and bills i have. if you connect staan-search (there is a pi plugin for that) and ctx7 to this it almost does miracles. the downside is i have to sit next to my noisy threadripper as the magic happens and pay for the electricity, but that's about it, i'll gladly do that. and as i finished this paragraph, it also finished organizing all my personal documents on my san. i don't use the expression "game changer" easily, but it's hard to resist in this case. out of all the models i've used locally qwen3.8:27b blows everything out of the water.

my setup

# Logical CUDA0 = RTX 4090, logical CUDA1 = RTX 3070 export CUDA_VISIBLE_DEVICES=0,1

cd ~/projects/misc/llama.cpp/

exec ./build/bin/llama-server -hf ggml-org/Qwen3.8-27B-GGUF:Q4_K_M --mmproj /xx/xx/xx/xx/xx/mmproj-Qwen3.8-27B-Q8_0.gguf --host 0.0.0.0 --port 8080 --jinja --parallel 1 --split-mode layer --tensor-split 6,1 --fit on -fa on -c 98304 -ctk q8_0 -ctv q8_0 --image-min-tokens 1024

i load more on the 4090 because it's faster.

usually the temp stays around 65 for both. utilization for 4090: 70-90% 3070: 30-50%. I get around 30-40 tk/s. if i offload more to the 4090 the tk/s goes up, but i stress the card too much and that thing now is worth its weight in gold.

note: the pi-llama plugin needs a patch for pi to send the model vision capabilities, seems it doesn't work out of the box.