But you didn’t spend hundreds of hours on it, so when it did happen to be useful it seemed like an outsized benefit.
I would wager that for most people, most data about themselves will be useless and not worth collecting.
Of course you can’t know what data will be useless or not, so unless the cost of collecting it is minimal or nil (wearing a smart watch, writing down your weight each day/week), it’s probably not worth it.
Spending hundreds of hours to build a solution to capture all data about yourself to find interesting patterns has a huge assumption baked into it: that there are interesting patterns to find.
Like anything else, I think it comes down to having a good use case.
I've gotten deep into weightlifting/bodybuilding over the past couple of years, and that's the kind of hobby where micro-optimizations and data tracking can have a pretty big impact on results (and sort of necessary, you can't fly blind with things like diet, especially)
E.g. I track and weigh everything I eat, take body measuraments on a weekly basis, Dexa scans every few months, etc - for me it's worth it because I know what I want to do with the data. If I didn't have a goal, all that tracking would clearly be overkill.
Right, but you don't necessarily need to spend hundreds of hours to capture most of this; the data is already out there. If there were a tool that could collect it all in one place and give you insights with minimal effort that would be pretty neat.
The tricky part is the maintenance drag and subtle risk that you'll start making decisions to optimize your life by proxy metrics just because you have them.
About 9 years ago I had a run-in with stomach cancer. After a few months of chemo and a 7 hour operation I was eventually declared cancer free and have been ever since, but still have to live with the consequences of the treatment and be vigilant for any signs of it returning.
I still suffer intermittent stomach aches, especially in the early hours of the morning, and had a terrible time trying to decide if they were getting better or worse over time.
Our narrative voice is awful at detecting long term trends and tends to overcompensate for particularly good or bad patches so it was impossible for me to judge and I started keeping records of how bad the aches were each day.
Long story short, the average severity was mostly decreasing over time and the average time between bad aches was slowly increasing but it would have been impossible to tell if this was happening without keeping detailed records because it wasn't consistent - some months were much worse than others and completely skewed my perception of long term trends.
While most people hopefully won't ever need to do something like this, it did make me realise just how bad we are at picking up on long term trends so I can definitely see keeping daily records of, for instance, average daily happiness being eye-opening.