Here is an article by Andrej Bauer answering the exact question, "What is algebraic about algebraic effects and handlers?": https://arxiv.org/abs/1807.05923v2

Contrary to the claim from the comment you are replying to, according to Andrej Bauer, "algebraic" does not refer to the composition of different algebraic effects via composing their handlers.

When you see "algebra," think "algebraic structure," i.e., a set of operations and a set of equational laws on those operations.

An algebraic effect consists of a set of effectful operations. In that sense, each algebraic effect (reader, state, etc.) defines its own algebraic structure. In theory, the operations of an effect should also be related to each other by laws. Here is an example for the state effect from Andrej Bauer's paper:

    lookup(ℓ,λs.lookup(ℓ,λt.κst)) = lookup(ℓ,λs.κss)
    lookup(ℓ,λs.update((ℓ,s),κ)) = κ()
    update((ℓ,s),λ_.lookup(ℓ,κ)) = update((ℓ,s),λ_.κs)
    update((ℓ,s),λ_.update((ℓ,t),κ)) = update((ℓ,t),κ)

However, monads that cannot be defined in this equational style cannot be translated to algebraic effects. An example is the continuation monad: https://old.reddit.com/r/haskell/comments/44q2xr/is_it_possi... (I think you are involved in the Reddit comment chain that I'm linking to...)

AFAIK, another example is the "exception" monad, because the way that you interact with it is through the handler itself. I once saw a thread on r/Haskell discussing this, but can't find it.

This is my favorite one, short and understandable: https://philipnilsson.github.io/Badness10k/posts/2017-05-07-...

I show it when I teach Haskell, and it's what usually makes it "click" for students. Probably because motivating examples are in normal imperative pseudocode.

I'm no expert, but perhaps https://arxiv.org/abs/2302.01415 (especially section 2.2, "Non-Algebraic Effects") will help.