Rust + Axum has sort of supported this with a middleware (https://github.com/lazear/axum-aws-lambda). Essentially you write your API as normal, and then when you build for lambda you enable a feature that does the special handler for you.

This + Function URL + Cloudfront, and you end up with an API entirely hosted in lambda. I have a backend that does is hosted like this that does pretty complex work (image/video uploads, image conversions, on top of the standard auth, database connections, etc). It even renders a frontend with HTMX/Alpine.

Even with all the bot traffic I receive, at almost 15 million requests per month the cost is under $10/mo (median latency is roughly 90ms). Cold starts are a non-issue with the Rust runtime.