You might consider having some sort of gateway device connected to the internet…with it forwarding email to your Notes environment.
There are special purpose gateway appliances that filter all kinds of nasty stuff before it has a chance to get into your network.
At the very least, you could consider creating your own gateway email server using a cheap Linux box and something like Exim (or Sendmail). They give you the opportunity to have rather fine-grained control over SMTP email.
I’ve never been fond of having a full-function email environment tied directly to the internet.
I have a gateway device connected which controls Virus and Spam. I don’t think there’s much more we can do other than restricting the abusing IP address.
I guess if it were possible to create a rule that blocks incoming SMTP connections for 30 minutes if:
More than 50 connections from the same IP are attempted within 10 minutes.