They basically just added DeltaNet hypernetworks to existing LLMs.
Nothing super novel or groundbreaking, but a moderately interesting read.