Access Control issue

I have 2 users user1 and user2 who have access to doc1.I have denied access to user1 by applying Deny acl on doc1;but when I use user2 and send the document to user1 for review/approve;user1 has access to this document from his task page.How do I prevent user1 from getting access to this doc at all.