IMAP port closes connection when downloading email

Has anyone else had a IMAP port close when downloading an email? I have the case where I can open a port then when I try to download the email the port closes.