alt Hacker NewsOK, so, I too have built a local index tool called slopdex [1].
It's very simple, parses the repo code and docs with treesitter, indexes it, vectorizes everything, then allows you do things like create a map of the codebase or do a semantic search.
One cool trick that does work well is the cross-search command, which allows you to do a duplicate code search using the vector index. This gets rid of a blind spot that lots of LLMs have that make it create lots of dupliclate helper functions. This works quite well in practice.
It has a few evals that I'm still working on, showing only modest improvements in token usage when using the map command, maybe around ~10% less. I think this can depend a lot on the type of task though.
Would be happy to hear any feedback.
Replies
Curious about this implementation. How does the vector search work with a versioned code base?
If I'm on commit A, then save the work and now I'm on commit B, how does cross-search ensure it doesn't return results for code JUST in commit A (i.e. code that is no longer in the codebase).
I am interested to know your thoughts about publishing with the repository a natural-language implementation-agnostic specification that would allow other coding agents to create and mutate their own parallel implementations.
Are you modeling the actual API costs? If you can cut round trips at the end of a long context window you can end up reducing usage even if the total token count is the same.
Nice tool. Does it support local embedding providers such as qdrant https://qdrant.tech/ ? Why only support paid, cloud-based embeddings provider ?
ChatGPT says it could work without updating the code, but since README doesn't advertise it, I was curious as to the reason behind this choice.
It says: "Can Slopdex run without paid embedding APIs?
Potentially yes, without modifying its code. [...] Slopdex supports a custom embeddingBaseUrl, allowing it to call a locally hosted, OpenAI-compatible embedding server. For example, with a local model producing 768-dimensional vectors: { "provider": "openai", "model": "your-local-embedding-model", "dimensions": 768, "symbolDimensions": 768, "embeddingBaseUrl": "http://localhost:1234/v1" }
Its code still requires an OPENAI_API_KEY value, but a dummy key should work if your local server doesn't enforce authentication.
The server must implement /v1/embeddings and accept Slopdex's request format, including the dimensions parameter. The relevant code is in vector_openai.rs."