alt Hacker NewsI’ve been wondering if chat is the wrong interface for slower local models (and some projects) and maybe something like a ticket system is a better fit. I just decided how I would test this idea on my available hardware before I go drop money on a Mac Studio or GPUs. I’ll probably have a POC this week. There is nothing novel here, just need to spend the time to get it working for me.
Replies
So I’ve been thinking about this problem a lot, specifically as it relates to running LLMs at home, and I’ve been using GLM-5.2 to make an SMTP/IMAP-to-LLM gateway.
I’ve been wondering about something similar - a system that enforces (or does the heavy lifting) of dividing a large task into smaller sub-tasks so that it’s easy to run/check/test each one independently - even on a fresh model instance if needed.
This is based on the observation that the medium-sized open weight models (~20-35b) are very able to one-shot smaller discrete tasks but seem to lose their way project managing themselves through larger tasks that have multiple steps.
This is actually really smart. It would be like working with a team of humans.
I have a 3 Mac Studio set up and built an IDE / harness (propelcode.app) and would be interested in contributing if you’re open to collaboration
Time to make EmailGPT
Use the ticket system built into mininote.ink 's mcp server. Works perfect right out of the box. Also great notetaking app.
Docs:
Having a thin python/ts orchestrator and workers that pick up tasks from the directories like events and decide whether to make deterministic calls and wait is pretty standard albeit custom way of doing things in this space where you're bottlenecked by the concurrent call your workers/agents can make.
The hard thing is always keeping complexity low and being ZeroOps.