I do this too - "thee actual" vs "thuh real" and such. I do expect it's to do with having a vowel sound of a certain kind at the beginning of the next work making you move your tongue in a certain way.
with "the actual" I imagine you're also putting in a kind of intrusive "y" sound so it flows better into the vowel
with "the actual" I imagine you're also putting in a kind of intrusive "y" sound so it flows better into the vowel