(source, 20+ years experience working in traffic planning software industry). Such data do exist. But they won't avail you (much). You have similar problems as when you optimise the energy sector, or economy in general. The "actual" problem you will be optimising on/for, has a lot of conflicting needs and interests, and choosing your trade-offs resemble the 'trolley problem'. I will just enumerate a random list of things that affect and collide with your optimisation. - your vehicles and vehicle fleets (and road networks, with congestion) provide one part of the problem. - your vehicle drivers etc., have other needs and union rules; they need to sleep, have breaks, go to the bathroom, eat; swap places on the road, they need to arrive and get home. - once you are on the road 'you might as well continue driving': There is a huge startup cost each time you pull into traffic. - you have two peak hours (morning, evening); what should your drivers and buses do in the lull in the middle of the day? (spoiler: they reasonably hate sitting idle for 3 hours, with or without pay). - lots of politics related to how it is funded and votes, who is paid for operating it. These are the shameful parents for things like frankenstein bus-routes, where 2-3 cancelled budget-cut bus-routes are merged into a single meandering remaining bus-route that makes all its passengers miserable.

- a lot of it is probablility-based/spontaneous; only some traffic patterns are regular (children to school); to service the probability-need, you need a regular service that may run empty/low half of the time, until people need a hair-cut.

The whole thing is handled on a sliding-scale/spectrum, ordered by how short/far into the future, the travel is/can be planned. On one end, you have taxi/uber, which is on-the-spot travel. The other end is planned year-service, e.g. a summer schedule of a bus. In between, with computers we have a lot of "urgent-but-planned" flexi-flexible traffic, where people book on the day or the week, and a number of bookings are then weaved to be driven like a pearl string (so e.g. a number of people will share a ride in a medium sized bus, that does pre-planned pseudo-taxi-service.)

Pretty much all large public transit authorities already collect all sorts of data about requirements and usage, and have scientists/engineers on payroll to optimize the network within the economic and political constraints these systems operate in.

Not much. People plan for the lowest common denominator. If service levels surge during certain days or hours that's great, but people don't want to get burned by a period of thin service. Imagine if Fedex said overnight mail could no longer be delivered on Thursdays in certain towns. You'd have to remember to check if today's Wednesday and if the recipient's town is a low service zone. Or you could just switch to a more consistently available network like UPS.

The interesting thing with transit is that optimal may not be the most optimal. E.g. your model might suggest half mile stop frequencies. Seems good, until the public comment period and all the old folks who take the line show up and ask their particular stop not be displaced.

There is a reason why a lot of american bus networks basically follow the same old streetcar routes that were always there. People become used to existing patterns and build lives around them. New transit is a good opportunity to apply this model, but reworking old transit is tricky because some of your existing ridership will end up with a worse offering by definition, and they will protest. While the people who might benefit will not show up with equivocal advocacy in the public comment meeting.

I live in Helsinki, where there is a good tram and bus network. Every now and again a major road will be dug up, in the city center, and trams often get rerouted for a few months to work around that.

I've noticed that there are some diversions which are definitely beneficial to me. There are busses from my house to the city center, as well as a tram. If the tram diverts to a different destination that essentially doubles my available "travel destinations" in a way that's really useful.

So no doubt, for me personally, there are changes that could be useful. But perhaps they would be less useful for others.

The best answer for many problems, is to avoid the action in the first place. Just because you can do it doesnt mean you should do it, and public transport is mostly thought to serve commuting.

A lot of people commute for hours to sit at a computer...which means they could work at home.

You might explore "gray" semi-formal bus networks. Where the on-the-table gap is sufficient to fund businesses. And where the efforts to shut them down, provide insights into mechanisms, incentives, and interests, around that inefficiency.

due to how hard it is to change the networks, I think there's a lot of inefficient route design. if you look at SEPTA in Philadelphia, they're in the middle of realigning a lot of their bus routes for the first time in 60 yrs and ridership has increased noticeably since they started.

Well, there is some of that. It's not near as much as you might guess. Predictable transit is very helpful because people plan their life around it. What the data you're looking at will show where the bottlenecks are, and yes, you can remove a few bottlenecks and make people's life a little bit better, but it's not going to be a major change because of major change will not show up in the data until well after it's made.

I don't remember the terms used, but I've seen data sets of where people live and work in Manhattan, and you do see people who live on subway lines take jobs on that same line, and they don't take jobs on a different subway line that while possible is overly inconvenient.

people have already arranged their life around the existing transportation that exists around them. They are not going to change their life because it is already in place. They want bottlenecks fixed but otherwise don't really need a change to their life. Even if a change would greatly benefit them, they often can't see that and wouldn't imagine themselves making the change to their comfortable life.

The real value of transportation experts is they can look at a map and all your data and say, if we make these changes, over 20 years, the city will redevelop in this way, and then we can ask questions. "Is that how we want the city to develop?" however, the data is never going to show that because the data shows what people are doing today, not what they're going to do in 20 years. Don't get me wrong, experts get this wrong too when they make predictions. However, there is a lot of things we can use to inform our predictions. even though we're not perfect, we can make predictions that are usually right.

Varying the route timing is the absolute worst thing you can do. People need to depend on your transport system. If they have to look at a schedule that's going to change by day to figure out when they can actually get somewhere, they're going to get frustrated and start looking up for an alternative that they control. If you vary your timing at all, it's only because you're changing between a train every minute to train every five minutes. You don't get to go any farther apart than that without losing writers because they no longer can trust that the system will get them there when they want to be there. Physics puts a lower limit on time. I believe the current record is a train every 90 seconds, and I'm not sure how much closer together you can get. The limit for a bus is three minutes apart and even that is extremely hard.

Transit systems that are running worse than 5 minute frequencies have large drops in ridership and as they increase frequency: ridership jumps because people start to use, they can always just go out when they're ready to go. If this wasn't so expensive, everyone in the world would be running 5-minute frequencies all day.

"Efficency" in what domain? Travel time? Cost? Convenience/comfort?