Can you reverse engineer our neural network?

I worked on a puzzle like this roughly 2 years ago from Anthropic. I did the first half, the easier part of the CTF, and my friend did the second half, the more technical ML stuff. We both got interviews at Anthropic, which was cool - I wasn't anywhere close to nailing an interview at Anthropic but it gave me a lot of confidence to end up going all in on tech, which paid off greatly. My friend's short write up: https://x.com/samlakig/status/1797464904703910084

I was curious to see if I could crack the MD5 hash so I managed to write the following python code to extract the expected hash from the model:

https://gist.github.com/alexspurling/598366d5a5cf5565043b8cd...

Knowing the input text was two words separated by a space, I was able to use hashcat and the unix wordlist (/usr/share/dict/words) to find the solution almost immediately. It's a shame that Alex didn't find it this way on his first attempt as the two words are fairly common.

I'm really curious what were the magic words.

> Alex had actually tried to brute force the hash earlier, but had downloaded a list of the top 10,000 most popular words to do it, which turned out not to be big enough to find it. Once he had a big enough word list, he got the answer.

They don't reveal the answer.

This is pretty cool, I wasn’t aware of these types of challenges. How does one even approach this?

Feels to me like it’s similar to dumping a binary with an image, the format being entirely custom.

And/or trying to decode a language or cipher, trying to recognize patterns.

Model interpretability is going to be the final frontier of software. You used to need to debug the code. Now you'll need to debug the AI.

Ah dang. When I did this I also thought the length bug was intentional but I didn't figure it out before I started my new job, so I dropped the puzzle.

Another classic Jane Street puzzle. Boy this was a good one. Sometimes I look back at my childhood and how quick I was to solve some difficult integrals and so on and now I’d struggle at that. This is far beyond that but the leaps of intuition required here sort of have that property that they need you to stay in the game. Step away a few years and try to come back and there’s just a wall.

I don’t think I’m close to making progress on stuff like this. Interesting to note. Glad they wrote out this behind the scenes thing.

Give me unlimited API access maybe I can distill it

[stub for offtopicness]

Seems like a thinly-veiled recruiting ad...