I'm a physicalist as well. A commitment to physicalism doesn't force you to reject the hard problem. The hard problem doesn't entail that phenomenal consciousness is not grounded in physical structure and function. The hard problem is about what is needed to explain consciousness. Science typically involves defining some phenomena precisely in terms of structure and function, then giving a precise story about how some observed behavior captures the structure and function that defines the phenomena under question. For example, we define temperature by the height of mercury in a thermometer, then we mathematically derive the height of mercury given the average speed of molecules in the environment. Thus we successfully reduce temperature to average kinetic energy of molecules.

But the process of reduction starts by precisely defining the phenomena in terms of structure and function. If we are unable to give a precise definition that uncontroversially captures the target phenomena, then we cannot in principle give a scientific explanation of said phenomena. This is where we stand with consciousness. There is an in principle barrier to a transparent structural description of phenomenal consciousness. But this is an explanatory limit only. It doesn't necessitate some non-physical phenomena is involved. What we need are new concepts that can connect the phenomenal to the physical. But conceptual innovation is not something you get from more measurements and more data. This is what makes consciousness a philosophical problem.