Sorting doesn't need any branches. For FHE you need to think like a circuit designer without indexing operations. In that situation your default sort is a sorting network, made out of nlogn conditional swaps.

The flip is also true: LLM inference is very nearly maximally optimal for FHE. Thee costly bit, multiplication of a bunch of compressed floating point numbers, translates 1:1. The only bit I’m not sure about is the softmax sampling at the end, but that’s just once per token.

On the other hand, each FHE step is a LOT more costly (e.g. elliptic curve exponentiation) than a vectorized BF8 multiply on GPU.

Unlike the sorting number case, it is probably same complexity. But no hardware support AND a massive slow down constant even if you were comparing apples to apples on hardware.

Disclaimer: I spent a decade working on crypto systems but I am not familiar with TFA’s research.

I found [1] which appears to offer state-of-the-art performance of ~1000s latency for a FHE GPT-2 transformer block, equating I think to 3.33h inter-token latency (0.00008 token/s) for GPT-2(small) which has 12 transformer blocks. This result is using optimised packed arithmetic operations on a GPU as well--so seemingly is unlikely to have much performance upside from further optimisation.

I'm not sure I've interpreted [1] correctly though, and would appreciate correction if necessary.

[1] https://arxiv.org/pdf/2604.04783 -- GPU Acceleration of TFHE-Based High-Precision Nonlinear Layers for Encrypted LLM Inference -- Guoci Chen, Xiurui Pan, Qiao Li, Bo Mao, Congming Gao, Chengying Huan, Mingzhe Zhang, Jie Zhang -- Apr 2026

I’m genuinely not an expert, but isn’t the beauty of MoE models the fact that we explicitly don’t evaluate every parameter on inference? We evaluate exactly the subset that are needed to evaluate a prompt. Seems like this will bring back data-dependent branches again.