Recipient Email Addresses

How can we hide the recipient tab which reveals addresses to specific groups or users?  This is a potential security breach that we would like to address in our company due to external users from different companies sharing objects.