We ran into that problem just last week. If the document is in edit mode and the person clicks on one of the attachments to view it in the browser, then when they click the back button on the browser, the document saves a second instance of the attachment on the form. We also found out that even though it looked the same, it was creating a whole new response document with the same values but a different unique ID and the added attachment. We only needed one response document, so we did a lookup to see if a response existed for the parent document. If it did, then we set the SaveOption to “0”.
Are you getting it at the bottom of the document or to the side. Make sure you have not computed RT field into which the attachment is re-appearing as duplicate, after you save the document.
Check if there is any wrong code while creating response or saving the document or at any form event, which grabs the content of this RTfield and append it to the same, in which case you may have duplicates as you mentioned.