alt Hacker NewsIt seems to me that the model struggles to have enough general intelligence, knowledge, or reasoning capacity for arbitrary prompted tool calling. At this size, not surprising.
I am VERY interested in seeing how it could perform with some fine-tuning for a specific family of tools/tasks. That would be a great addition to the demo.
Replies
electroglyph • today at 10:36 AM
i would assume a model this size would require finetuning tbh. even functiongemma recommends that.
It also seems to have far more tokens per second than needed for general "close the blinds" "tool_call(blinds, CLOSED)".
I do wonder if more smartness could be had by using sparser experts.... And possibly even having some kind of expert switching penalty to try to reduce the amount of data read from read only flash memory by encouraging subsequent tokens to use already loaded experts.