C++ has to be approached as a grab bag of useful parts, not a coherent whole. It's somewhat akin to Python if Python modules were C++ language features. It's said that nobody knows all of C++, even its creators.
Do you know C? Everything that works in C works in C++, modulo a few minor syntactic changes (void* doesn't implicitly convert). You can get acquainted to C++ features on an as-needed basis. Other people will surely disagree with me on this.
With pointer types in particular, I'd say use C-style pointers long enough to know how frustrating they are but also how they work. Then you'll know what unique_ptr and shared_ptr are doing to help you. (And those are the only two you need 99% if the time. weak_ptr is a spin-off of shared_ptr that you'll know if you need it. observer_ptr is just stupid ivory-tower stuff.)
It's probably nice to know WHY you might need weak_ptr.