alt Hacker NewsIt's easy to underestimate just how slow digital systems are in many real time scenarios compared to analog systems.
Consider it in the context of camera based self-driving cars, it's tangential to this discussion but it's an easy to visualize metaphor:
- A car traveling 60mph is traveling at 88 feet per second
- Assuming a 60hz camera, there would be a 16.67 ms gap between each frame
- The car is traveling 1.5 feet between each frame interval
- A certain amount of exposure time is necessary for the camera to generate even 1 frame or it will be blurry
- High framerate cameras often work around this by staggering/interlacing multiple sensors, but doing this implicitly increases the latency of each frame
- A 120hz camera might deliver double the frames per second, but each frame could be arriving 4 frames late in exchange
- 4 frames would be imperceptible to humans, it would be 3 feet for the car
- You haven't even processed any of these frames yet.
- Your off the shelf library introduces a random 1 second delay for some reason and costs you 88 miles in processing time
- The car can drive as fast as 120mph
All digital sensors implicitly have a sampling frequency, and the fundamental disconnect is always high sampling frequency =/= low latency. People constantly make this mistake over and over, and by the time you notice you are already too deep into development to make a change.
Decreasing latency is expensive, and requires specialized knowledge. Often you get expert software engineers who end up bottlenecked by the hardware limitations they can't even comprehend or the reverse, hardware guys bottlenecked by the software they can't introspect. The latency is only truly understood when you get to integration testing.
Nearly every step of the way you discover you need specialized hardware, software, operating systems, sensors. Every part of the chain each costing you more latency. It's like it's own ecosystem where almost everyone writes everything from scratch and doesn't share anything. It's gotten better in recent years though.
Full disclosure: I work in medtech and don't actually deal with cars, but it's a very similar problem space. We often use the same hardware/software cars use for this reason.
Replies
Honestly I'm just a dumb web dev and even though many of our performance problems can be solved by horizontal scaling, there are a minority of cases where latency is important and that's when things get really fun :-) No more downloading random halfbaked libraries off the web, architecture starts to matter, vm settings start to matter, gateways/load balancers in the middle start to matter, even code optimization starts to matter again just like in good old days.
I can’t quite understand your comment.
> Assuming a 60hz camera, there would be a 16.67 ms gap between each frame. The car is traveling 1.5 feet between each frame interval.
Ok? So? You are just stating this as if we should understand the implications. I do in fact work with self-driving cars. What you say is true, but it is not a big deal. Why do you feel this maters? Or what is your point?
> A certain amount of exposure time is necessary for the camera to generate even 1 frame or it will be blurry
This is a confused statement. A certain amount of exposure is necessary for the camera to collect light. If you don’t have long enough exposure the picture will be dark, not blurry. Your statement makes it sound as if avoiding blur is why we need exposure time, which is a complete nonsense.
In reality none of this is a problem. There are automotive grade cameras which can collect enough light fast enough that the images are not blurry in practice. Yes, these cameras have a non-zero exposure time. Yes, this adds latency. No, this is not a problem.
> Your off the shelf library introduces a random 1 second delay for some reason and costs you 88 miles in processing time
You mean 88 feet. If my off the shelf library introduces a random 1 second delay i will chuck it in the bin post haste. Use stuff whose performance characteristics are well understood by you and is not terrible.
> Nearly every step of the way you discover you need specialized hardware, software, operating systems, sensors.
I do not recognise the world you describe.
Awesome post, illuminating metaphor, +1.
Tangent: why do so many people (even smart articulate ones like you) in our field (which involves precision with syntax and grammar) get apostrophes wrong?
"It's [IT IS] like its [ITS/HIS/HER/MY/YOUR/THEIR] own ecosystem..."
I anticipate downvotes for picking nits (let alone mentioning karma points), but FTR my intent is to help non-native English speakers (and mostly-literate English language-natives). Getting this "it's : its" distinction wrong is increasingly common and sometimes leads to signal loss.
Yesco, rereading your comment makes me slightly ashamed of this apostrophe rant, I hope others comment on the substance here. / end tangent
As someone who spent years doing web performance optimization for a living, your observations resonate. Beyond obvious low-hanging fruit, latency gains are rarely simple to achieve in practice; tradeoffs abound.
88ft not 88 miles. I don't understand your 120hz camera line, I also have a degree in imaging tech and that part I do not understand, would you be so kind as to expand upon it??