Best practice is usually to use the "custom" folder. Think of your shell DTD as any other DTD, and follow the usual Arbortext process.
It goes something like this: 1. Create folder inside custom/doctypes named "myDoctype" 2. Copy in the "myDoctype.dtd" 3. Setup catalog file to link the PUBLIC ID for your doctype to "myDoctype.dtd" (good idea to setup your .ent and .mod here too) 4. Setup DCF
With step #4 I recently discovered you can inherit from the base Arbortext DITA stuff. Have a look at $ARBORTEXT$\application\com.arbortext.sma\doctypes\servicemanual\servicemanual.dcf for pointers.
I think by default Arbortext will recognise your specialisation and apply the out-of-the-box DITA styling but you can also make your own stylesheet(s) too.
You can distribute your custom folder as a folder or (since 6.0?) as a ZIP. The client will use the APTCUSTOM environment variable to locate your customisations.