alt Hacker NewsShow HN: TERMy – A fast terminal assistant that does not use LLMs
I love research and development, you may have heard of me because of PJON (Padded Jittering Operative Network). It is a network protocol I started developing in 2010, which was recently implemented in silicon by the ETH Zurich university thanks to the research of Pius Sieber.
I am excited to share with you TERMy, a terminal assistant built on top of the NPC-Forge framework. Unlike everything else being built today, TERMy does not use embeddings, machine-learning or LLMs. It runs on the CPU (even on a Raspberry Pi Zero) both in the terminal or client-side in a browser tab and responds in milliseconds. It is a cynical but very knowledgeable Linux terminal assistant that translates your natural language into shell commands without relying on a single artificial neuron.
I had a chance to focus for 2 months on my personal projects since early July, during the strange times of AI price hikes and the end of subsidized tokenmaxing. I was curious to see if I could develop from scratch a terminal assistant capable of handling simple natural language requests. I have a bad memory and got used to ask to copilot "activate the virtual environment" or similar trivial operations spending a non negligible sum every month. I started thinking, maybe I can do something to make my workflow more efficient? Do I really need trillions of parameters to accomplish those tasks?
How it Works
When you type a prompt, it goes through a lightweight NLU pipeline written in ~1000 lines of Python that implement the following steps:
1. Strip expletives, interjections, encouraging, discouraging and thanking words (remove noise)
2. Sentiment analysis
3. Exact Match (very fast)
4. Template Match (slower)
5. Probabilistic Match (even slower)
Step 5 relies on:
1. IDF (Inverse Document Frequency) to identify rare words.
2. BOW (Bag Of Words) to accommodate word inversions.
3. IDF weighted Levenshtein to safely handle typos.
Permission gating is hardcoded into the dataset and enforced for all potentially destructive commands, so it's inherently safer than letting an unpredictable LLM run wild on your machine.
- TERMy in operation: https://www.youtube.com/watch?v=qeIp0xePLBg
- Variance and typo tolerance: https://www.youtube.com/watch?v=tQvGDk6fkk0
- Copilot integration: https://www.youtube.com/watch?v=Wzzouhq2a8A
- Advanced features: https://www.youtube.com/watch?v=qeIp0xePLBg
- Source Code: https://github.com/gioblu/NPC-Forge
Comments
This is a cool project; I think it's wonderful that traditional NLP methods are being used, rather than reaching straight for an LLM.
One significant advantage of not using a local LLM is the significantly simplified dependency stack.
If you could get Termy to code, you'd be a rich man
It would make sense to have this integrate with a self-learning routine for an agent: e.g. at night it looks through what it did and writes NPC-Forge recipes. Tomorrow it can answer queries (which he turned added to NPC-Forge) without an LLM. Of course this implies a branching where a query is either processed by NPC-Forge or an LLM, depending on some measure of confidence that NPC-Forge can answer it well.
Hi, I am the creator, feel free to ask any questions :)
What do you think about it?
It's kind of antithetical to the tool's deterministic positioning, but have you considered making TERMy leverage an LLM for unseen or low-confidence queries, and then generate the config and update itself to make future similar queries deterministic?
Very interesting project, I like it. Just wanted to clarify though the sentiment analysis is just the count of stripped words and used to tag things with the emoji? I was initially expecting it to be a part of the actual command construction process (even though I couldn't figure out how that would be relevant) given how it was listed.