Yes, neural networks are famously poor at generalising.
They are poor at generalising from a small number of examples; this is why the real generalisation power is achieved in pre-training.
alt Hacker News
They are poor at generalising from a small number of examples; this is why the real generalisation power is achieved in pre-training.