logoalt Hacker News

kevinqitoday at 6:11 PM5 repliesview on HN

I agree distillation isn't illegal; I also think Moonshot/Kimi is very impressive. But the more interesting question is whether labs like Moonshot can be a real competitor to OpenAI/Anthropic. If you can only play catchup (however quickly you do that), then you're never going to be at the frontier - I think that's why distillation matters.


Replies

mring33621today at 6:16 PM

People that think the Chinese are only able to copy western tech are in for a wakeup call.

Actually, that has already happened in many domains, it's just that most western people (USA especially) won't admit it.

show 1 reply
overgardtoday at 8:56 PM

I think it depends on where you think we are on the S curve of intelligence growth. (Yes, I think it's an S curve, not an unbounded exponential). If you think we're near the peak than playing catch up (especially if you can play catch up quickly) is very rational.

I know this isn't exactly a scientific test, but I had a local Qwen 3.6 27B model implement a fairly sizable feature today. There were a couple of bugs, mostly around me not giving sufficient specifications, but they were ironed out quickly when I pointed it out. I was able to ask the model to create instructions so next time it doesn't fall into the same pitfalls, and it did a great job. 27B local model! (And it was super fast too).

I ran Fable 5 as a code review and it didn't really have any significant corrections.

I guess my point here is that, for most work the frontier models are probably overkill anyway, and improving on overkill in a way that raises prices significantly is probably not a winning strategy.

The only place I can think of where the super high powered models are "required" is if you want to do a ridiculous token burn like GasTown where you just have it run un-monitored on very long tasks. To me though, that's an experiment, not a real workflow. And the way these labs are like "oh we made this (broken) thing in a week using just agents!" always also follows with "and it cost $100,000+ in tokens!". Like, ok, I get it if you're doing research but that's the salary of an entire person.. that can actually learn and improve.

overfeedtoday at 10:09 PM

> But the more interesting question is whether labs like Moonshot can be a real competitor to OpenAI/Anthropic.

The answer depends on whether you think the AI researchers at Chinese labs are (or can be) as smart, motivated, and as good at math as those working at US labs - a not-insignificant proportion of whom are Chinese nationals.

nylonstrungtoday at 7:14 PM

So many of the breakthroughs and architecture that make LLMs powerful in general today came from China, especially ones related to sparsity and MoE that have made inference and training substantially cheaper.

Let's not forget how much people talked about "prompt engineering" before Deepseek mainstreamed the idea of thinking mode which is now universal

himata4113today at 6:49 PM

My entire point was that this was not achieved purely from distillation and claiming that is slander against open research.