Securing Directory Assistance

I have Directory Assistance setup and running right now, and have a couple of address books which I need shared to a high percentage of users. The problem I am having is making it secure to those users only — I have the default and anonymous users as no access, selected groups having reader, a few people having update authority. How can I prevent other people (who shouldn’t have access to the address book) access?