Is Luna MoE? Had no idea OpenAi went that direction
We don’t know but I think we assume that it is MoE or some almost equivalent sparse activation technology.
alt Hacker News
We don’t know but I think we assume that it is MoE or some almost equivalent sparse activation technology.