That is really poorly worded by Apple, because if I understand it correctly, the "Files & Folders" list is just a list of apps that have requested Full Disk Access/FDA (or other locations).

It's really confusing that some of those settings can be toggled on/off, while the Full Disk Access is greyed out and can only be toggled under "Privacy & Security".

To add to the confusion, toggling FDA off just protects a few selected folders that Apple decided are extra sensitive, like:

  Messages                     ~/Library/Messages
  Safari browsing history      ~/Library/Safari
  Cookies                      ~/Library/Cookies
  Identity services            ~/Library/IdentityServices
  Spotlight data               ~/Library/Metadata/CoreSpotlight
  Phone call history           ~/Library/Application Support/CallHistoryDB
  Facetime data                ~/Library/Application Support/Facetime
  TCC database                 ~/Library/Application Support/com.apple.TCC.db

"Normal" files and folders on your disk (including Desktop, Documents, Downloads, network volumes, and removable volumes) can always be accessed (even with FDA permission revoked!) after a simple prompt. [1]

[1] https://support.apple.com/guide/security/controlling-app-acc...