> I've also kind of wondered if GPS might be able to give you a time less accurately but quicker. Sometimes I look at the NMEA GPS output, and it sometimes doesn't have a position fix, but it does give out a time + week number

No. Accurate time is one of the prerequisites for acquiring a position fix, and it's the one that's the easiest and fastest to obtain from the sky. Typically a receiver should be able to get time within seconds from cold start if there's enough sky visibility, and once it does it is as accurate as it gets regardless of position fix status (unless you care about sub-nanosecond precision, but you need to be able to somehow consume that anyway and you won't with a Pi).

That said, NMEA output will usually contain the time from the module's internal clock that gets synchronized with GPS time when available rather than GPS time directly. Some modules can be configured to report when they have synced their time though, and you could pay attention to the time jumping to infer it too.