alt Hacker News> Yes, starting with tests without market fit can also be fatal. But calling anything done without tests is just a slower poison.
I think we are pretty close to agreement here. I'd be interested in what you have experienced in the realm of front-end testing though - whether you think things are just as cut and dried in that realm (that's another discussion though).
And I'll also accept the point about skill in test writing that improves the cost-benefit analysis. I'll also cop to not having that kind of practiced ability at testing to the level I would personally like. But it's chicken / egg. A lot of folks get their start at scrappy start ups that can't attract the best talent. And just can't afford to let their devs invest in their skills in this way. Hell - even established companies just grind their devs without letting them learn the shit they need to learn.
I feel a victim of this to some degree - and am combating it with time off work which I can afford at the moment. One of the things I'm working on is just understanding testing better - y'know, so I can in the future write a SKILL.md file that tells Claude what sort of tests it should write. lol...
Testing is hard. No, testing is fucking hard. I've had more mentors in testing than any other two disciplines combined. And I still look at my own tests and make faces. But to a man everyone who has claimed testing is not hard has written tests that made me want to push them into traffic.
Every problem is easy if you oversimplify it.
I send people who come to me struggling with their tests away with permission to fail at them but not permission to give up on getting better. You're gonna write broken tests. Just don't keep writing the same broken tests.
If anyone looking for a PhD or book idea is reading along with this, my suspicion is that it's so difficult because we are either 1) fundamentally doing it wrong (in which case there's room for at least 2 more revolutions in testing method) 2) someone will prove mathematically that it's an intractable problem, Gödel-style, and then someone will apply SAT solvers or similar to the problem and call it a day. Property based testing already pretty much does Monte Carlo simulation...
For backend tests, the penalty at each level of the testing pyramid is about 8x cost for a single test (and IME, moving a test down one level takes 5x as many tests for equivalent coverage, so moving a test down 2 layers reduces the CPUtime by half but also allows parallel execution).
For frontend I think that cost is closer to 10x. So you want to push hard as you can to do component testing and extract a Functional Core for unit tests even harder than you do for backend code. Karma is not awesome but is loads better than Selenium, particularly once you have to debug a failing test. I've been on long projects for a minute so I can't really opine on Puppeteer, but Selenium is hot flaky garbage. I wouldn't tolerate more than 100 E2E tests in it, even on a half million line project. Basically a smoke test situation if you use it like that, but you won't have constant red builds on the same eight tests on shuffle.
I want to say we had 47 E2E tests on a project I thought was going swimmingly from a SDLC perspective. But it might have been 65.