I wonder if "Security Labels" can be configured and set for Windchill "Folders" as well? And if yes, how can that be configured?
No, security labels cannot be set for folders.
The only object types Windchill currently exposes security labelling for are the ones listed in the <Windchill_Home>\Windchill\conf\exposedSecurityLabelObjects.xml file.
You could post a product idea, as the article mentions, for Product Management to consider.