It depends what you're doing it.

My app is fairly simple node process with some side car worker processes. k8s enables me to deploy it 30 times for 30 PRs, trivially, in a standard way, with standard cleanup.

Can I do that without k8s? Yes. To the same standard with the same amount of effort? Probably not. Here, I'd argue the k8s APIs and interfaces are better than trying to do this on AWS ( or your preferred cloud provider ).

Where things get complicated is k8s itself is borderline cloud provider software. So teams who were previously good using a managed service are now owning more of the stack, and these random devops heros aren't necessarily making good decisions everywhere.

So you really have three obvious use cases:

a) You're doing something interesting with the k8s APIs, that aren't easy to do on a cloud provider. Essentially, you're a power user. b) You want a cloud abstraction layer because you're multi-cloud or you want a lock-in bargaining chip. c) You want cloud semantics without being on a cloud provider.

However, if you're a single developer with a single machine, or a very small team and you're happy working through contended static environments, you can pretty much just put a process on a box and call it done. k8s is overkill here, though not as much as people claim until the devops heros start their work.

Depends. For personal projects, yeah definitely. But at work? Typically the “Platform” team can only afford to support 1 (maybe 2) ways of deployment, and k8s is quite versatile, so even if you need 1 small service, you’ll go with the self-service-k8s approach your Platform team offers. Because the alternative is for you (or your team) to own the whole infrastructure stack for your new deloyment model (ecs? lambda? Whatever): so you need to setup service accounts, secret paths, firewalls, security, pipelines, registries, and a large etc. And most likely, no one will give you access rights for all of that , and your PM won’t accept the overhead either.

So having everyone use the same deployment model (and that’s typically k8s) saves effort. I don’t like it for sure

I totally agree, but that's not what happens in reality: the average devops knows k8s and will slap it onto anything they see (if only so they can put in on their resume). The average manager hears about k8s, gets convinced they need and hires beforementioned devops to build it.

In some sense, Kubernetes is just a portable platform for running Linux services, even on a single node using something like K3s. I almost see it as being an extension of the Linux OS layer.

I know that "resume-driven development" exists, where the tradeoffs between approaches aren't about the technical fit of the solution but the career trajectory. I've seen people making plain workstation preparation scripts using Rust, only to have something to flex about in interviews.

I'm not surprised even in the slightest that DevOps workers will slap k8s on everything, to show "real industry experience" in a job market where the resume matches the tools.

> People use Kubernetes for way too small things, and it sounds like you don't have the scale for actually running Kubernetes.

This is a problem I've run into enterprise deployments. K8s is often the lowest common denominator semi small platform engineering teams arrive on. At my current employer, a platform managed K8s namespace is the only thing we got in terms of PaaS offering, so it is what we use. Is it overpowered? Yes. Is it overly complex for our usecase? Definitely. Could we basically get by hosting our services on a few cheap mini computers with no performance penalty? Also yes.

Doing Kubernetes like doing Agile is mandatory nowadays. I've been asked to package a 20 line worth of bash script as docker image so it can be delivered via CI/CD pipeline via Kubernetes pods in cloud.

Value is not that I got job done at a day's notice. It is black mark that I couldn't package it as per industry best practices.

Not doing would mean out of job/work. Whether it is happening correctly is not something decision makers care as long it is getting done anyhow.

We have a hobby web based app that consists of multiple containers. It runs in docker compose. Serves 1000 users right now (runs 24/7). Single VM.

No Kubernetes whatsoever.

I agree with you.

> it sounds like you don't have the scale for actually running Kubernetes.

You don't set up k8s because your current load can't be handled, you do for future growth. Sometimes that growth doesn't pan out and now you're left with a complex infrastructure that is expensive to maintain and not getting any of the benefit.

k8s is useful when you have services that must spin up and down together, and you want to swap out services and deploy all/some/one.

and then also package this so that you and other developers can get the infrastructure running locally or on other machines.

They use it for inflating their resume for career progression rather than actually evaluating if they need it in the first place.

This is why you get many folks over-thinking the solution and picking the most hyped technologies and using them to solve the wrong problems without thinking about what they are selling.

You don't need K8s + AWS EC2 + S3 just to host a web app. That tells me they like lighting money on fire and bankrupting the company and moving to the next one.

Even if using just one VM, I'll probably slap k3s on it and manage my application using manifests. It's just so much easier than dealing with puppet or chef or vanilla cloud-init. Docker compose works too, but at that point it's just easier to stick with k3s and then I can have nice things like background jobs, a straightforward path to HA, access to an ecosystem of existing software, and a nicer CLI.

yeah it's like wanting to drive to the mall in the Space Shuttle and then complaining how its too complicated

The problem with Kubernetes is that it doesn't scale down to small deployments very well, but it sure as shit doesn't scale up to large ones either. Large shared multi-tenant clusters have massive problems even when running parts of the same application with the same incentives, it falls apart completely when the tenants are diverse.

Nomad has neither of these problems.

I have nom doubt that there are legit use cases for something like k8s at Google or other multi-billion companies.

But if its use was confined to this use case, pretty much nobody would be using it (unless as a customer of the organization's infra) and barely would be talking about it (like how there isn't too much talk about Borg).

The reason k8s is a thing in the first place is because it's being used by way too many people for their own goods. (Most people having worked in startups have met too many architecture astronauts in our lives).

If I had to bet, I'd wager that 99% of k8s users are in the “spin a few containers to run your web app” category (for the simple reason that for one billion-dollar tech business using it for legit reasons, there's many thousands early startups who do not).