When checking in a generic and an instance of a family table, the other instances CANNOT be BOTH checked out AND unmodified. If they are, the checkout fails and, according to the documentation, the instances will have their checkouts automatically undone for you (without notice) and you can try again, though this doesn't always happen.
The resolution from PTC is to undo the unmodified checkouts before checking in the modified pieces, or select all of the family table and use Set for Checkin. The second one would be a LOT easier if the "Collect Instances" collector button was present, but this is the one collector that doesn't have it.
Best outcome: Allow check-in of family table instances while others are checked out and unmodified.
Acceptable outcome: Add the "Collect Instances" and "Collect Generic" buttons that we see on every other collector.