logoalt Hacker News

programjames • yesterday at 8:29 PM • 0 replies • view on HN

Most people who publish papers in ML are alchemists, but please do not assume all of us are. The original authors of the RoPE paper knowingly designed their embedding to be translation invariant. I do not know if they were aware it was a group representation, but what they were trying to design was in essence a group representation.