Seems like the most practical path is some kind of distributed peer to peer contraption where you can allocate some storage and optionally participate in crawling.

I imagine if set some constraints you could get index size down quite a bit but I still suspect it'd be hard keeping up with content churn.

Edit: Looks like enwiki bz2 is coming in around 46Gi which isn't too bad considering the amount of content it contains.