You can definitely do that, and it's really not too bad.

Grab SDL or Qt (underrated; gets you a nice menu and HUD layer) for windowing/input, basic event loop, etc.

Write a renderer, e.g. on top of wgpu or bgfx depending on how much scaffolding you want to have to write yourself.

OpenAL Soft for audio.

Jolt or Bullet for physics.

A good scene/world model as the backbone, and ways to efficiently mutate and propagate state. You can pick up an ECS lib for this, or just go custom and hand-wring your data structures, mutation journals, caches, what have you. Your scene model feeds into (and interacts with) collision/physics, audio (listener/sound sources), your renderer for viz, etc..

For gameplay a nice approach is some fundamentals in native code (e.g. triggers and actions) and then a scripting bridge.

The problem is basically that doing a good job requires a substantial amount of experience on several levels of being a dev: Good architectural knowledge (incl. state of the art, historical examples, trade offs), lots of domain-specific techniques (rendering, stepping, etc.), solid systems engineering (good platform/shell abstraction, OS/platform integration bits, debugging/logging/tracing infra), being handg with algorithmic work, performance/optimization-minded work, and so on and so forth. It takes probably at least a solid decade before you can knock this out without tripping up or needing a lot of endurance.

Oh, and on top of all of that don't forget to design an actually fun game.