It's hard. Particle physics faces the problem that in order to dig down to ever smaller scales, ironically, ever larger experiments are needed. We've pretty much built large enough colliders for our current understanding. No one really knows how much more energy would be needed to expose something new - it might be incremental, within current technical reach, or it might be many orders of magnitude beyond our current capabilities. The experiments have become expensive enough that there isn't a lot of appetite to build giant new systems without some really good reason. The hard part is coming up with a theory to justify the outlay, if you can't generate compelling data from existing systems.

Physics advances have been generally driven by observation, obtained through better and better instrumentation. We might be entering a long period of technology development, waiting for the moment our measurements can access (either through greater energy or precision) some new physics.