alt Hacker NewsI was considering paying someone to build something like this at some point. With two jobs, I eventually had no time to even organize what I find. It's just piles of links in text files.
Can I give your software a huge list of URL's to index? Or do I need to use browser automation to open them a few at a time with it caching and indexing them?
I accept donations ;)
Hister has a built in crawler with standard HTTP lib and browser based backends, you can feed your link collection to it. Also, Hister supports importing your existing browser history automatically using either of the mentioned backends.