When creating a saved report and using the 'New Data Monitor' function to create a scheduled report the only file formats available are: .html, .csv, .pdf, or .xml.
XLS or XLSX are not an options and do not look to be configurable.
Setting a Data Monitor to deliver .csv and then converting to .xls or .xlsx does not keep formatting, and does not work as expected for multiple tab reports or formatting. For example if you have a report column that has a value with a comma in the output the .csv format for the Data Monitor splits the value into multiple cells which is unacceptable.
I think this depends on how you parse the .csv.
From what I've seen, if a value has commas in it, the data monitor export will enclose the value in quotes. Excel will automatically parse this correctly, so that the string with the comma doesn't get split into two cells. If you are parsing some other way, you will need to add logic for addressing that.
I haven't seen how to add formatting or multiple tabs to a data monitor.