OK. Macros.dtd is part of the Arbortext infrastructure, used for managing recorded macros. I guess the next thing I'd try is work through the catalog files in the install structure to make sure the catalog that defines the actual location of this file (usually $ARBORTEXT/doctypes/macros/macros.dtd) is accessible, and that the file itself exists.
Also, check any catalog files in your custom directory structure to make sure you aren't inadvertently redirecting the public ID to the wrong location. The public ID is "-//Arbortext//DTD Macros XML 1.0//EN".
You don't say whether you have successfully run 5.4 M080 on Windows 7 (not in a VM). I have been using 5.4 M080 on Win7 for several months without any problems (other than ones of my own making...).
On my Win7 system there is a directory "C:\Users\Public\Public Desktop" which contains just the shortcuts to start Editor and Architect. APTCUSTOM is set as a user environment variable and points to the server directory where all our doctypes and custom files are located (on our network users must map the required share to a specific drive letter before starting Editor). I don't set APTPATH or APTCATPATH.
What VM are you using?
David S. Taylor
Project Manager, Structured Information Institute for Research in Construction National Research Council Canada Bldg. M-23A, Room 239 1200 Montreal Road, Ottawa, ON K1A 0R6