OOO and Internet e-mail

Recently I enabled my out of office notification and checked the “do not automatically reply to internet e-mail addresses”. It seems, however, it promptly did just that. Any ideas why? I have rights to run the agent, it was enabled on the correct server, etc etc…