MIME-CD conversion looses "attachments"

HCL Domino Domino Forum
,
1

Hello,

at one of my customers, I observe a problem and I need to cover some basics. Hopefully someone can help.

The questions:

Is there some documentation/redbooks/whitepapers/blogs about how Notes treats MIME and Conversion? (did not find anything helpful)

When exactly is mime content of a rich text field supposed to be (permanently) converted to notes rich text?

Obviously the mime content of a document is converted to rich text, when opened for editing and then reconverted back to mime, when saved. (this behaviour seems to be controllable with the session property "convertmime=true/false"). Is there some local temporary data stored on disk that may be intercepted by security software?

In which way are are mimeparts and the respective "$file" items linked together? Can this link break somehow so that the $file items are considere obsolete and therefore discarded?

The problem:

There is a custom Notes application - lets call it mailpool - which receives E-Mails and starts a workflow on them. When those E-Mails are from outside (SMTP -> MIME) they loose content as soon as they are being saved with edits.

MIME content of type binary (e.g. inline images) or with content disposition attachment is saved in a NotesItem called "$file". Those items just disappear, when I open the e-mail, set it in edit mode and save it afterwards. There is no change to the body (, there have only some items been added to the document.

Further info:

Notes 9.0.1 FP10 SHF315

Windows 10 1909/Server 2016

Mail body item is of type rich text, the property "save content as HTML" is NOT checked.

Session-Property "convertmime" is default (true), but it also happens with "false"

This happens for two years now, but until 2 weeks ago it was intermittently maybe once a month. Now I can reproduce it.

I know, that is a lot to read. Thanks to anyone who does. And the more for any helpful thought.

/Manuel

2

Please log reproducible bugs with HCL support, so they can ask development to solve it. Development loves reproducible bugs, since having a known testcase makes finding the source of the problem a lot easier.

3

As Lars says above, please report this problem to HCL support with as much accompanying details and data as possible. (E.g., if possible, provide the app and an example database.)

"When exactly is mime content of a rich text field supposed to be (permanently) converted to notes rich text?"

It depends (of course). There are many scenarios in which MIME is converted to Notes Rich Text or Composite Document (CD) format. For example, on the server-side the Router converts a MIME format message to CD when delivering to a recipient who 'Prefers Notes Rich Text.'

The Note client will save CD when the user opens & edits a MIME format message (if save content as HTML" is not checked, as you noted above).

If your app is LotusScript or Java, setting the convert MIME session property to false prevents an 'automatic' conversion to CD when the app opens the message. If the app then simply edits the message -- for example, altering the text in a Body item or adding an item -- and then saves the message, there should not be any loss.

However, if the convert MIME session property is true, the MIME->CD conversion will be done and it is not a full fidelity conversion.

The Domino SMTP server stores an incoming Internet message as a Notes document in the Router's mail box. Each MIME part is stored to a Body item that is TYPE_MIME_PART, not TYPE_COMPOSITE (CD). If the MIME part's Content-Disposition is 'attachment', then the part's data is always redirected to a database object (i.e., file attachment), the corresponding $FILE item is created and the part's Content headers are stored in the Body item, followed by a blank line, followed by the attachment filename. In this case the $FILE type is HOST_MSDOS -- a regular file attachment.

If a MIME part's Content-Disposition is 'inline' and the part's size exceeds about 50KB, then as noted for attachments above, the part's data is redirected to a database object. However, in this case the $FILE type is set to HOST_STREAM. (HOST_STREAM is used for large binary objects.)

When the Notes client opens a MIME message which has file attachments which are HOST_STREAM, the client renders the data inline if it can (e.g., if the binary data is an image) and it does not render a file attachment icon.

It's possible that the reason you don't see a file attachment when you open the message is that the attachment was actually a large inline image.

4

Hallo Paul,

that is the most educative thing I have read about that topic ever. Thank you very much for that!

There is a small misunderstanding: it is not that I just do not see the attachment icon. I do not see an icon AND the $file is just gone (looking with scanEZ). If it had been an inline image (even smaller then 50kb), I see the red rectangle instead of the image (no $file anywhere).

Is content of type "text/html" with disposition "inline" supposed to be redirected to "tkmime01" and why? (does no happen very often)

Headers: "Content-transfer-encoding: quoted-printable
Content-type: text/html; charset=ISO-8859-1
Content-Disposition: inline

"

Body: “tkmime01”

Finally, of course I have a case with HCL, but so far the response is not very helpful. For one they never answer questions to the "why" of something. Of course the support personnel would be out of work, if I could really understand the product and fix my problems on my own ;)

On the other hand the problem occurs within a custom application and they do not do debugging of custom applications - apparently not even if everything points to a Notes-Bug.

The result is an SPR with no timeline, a link to a KB-Article I do not understand the relevance of, because its about C-API and a workaround that does not work around (convertmime=false).

With high regards,

Manuel

5

Hi Manuel,

What is the support problem or case number? (E.g., the 'problem number' is something like 'PRBnnnnnnn' where 'nnnnnnn' is a random 7 digit integer while the case number is of the form, 'CSnnnnnnn'.) If you post the support number, I can take a look at the case and also at any associated data that you've supplied.

The SPR number is probably noted in the case, but please post that as well and I'll also look at the SPR.

regards,
pwl

6

Hi Paul,

that is a very kind offer from you. Thank you very much.

The case number ist CS0223454.

SPR # MAVABXLQL8

Some other update:

the mentioned tkime01 is the result from our mail security plugin (IQ.Suite). I disabled it for my testmails and the behavior slightly changed, but it did not resolve the problem. (wanted to do this test before answering you)

You said earlier that inline content > 50kb is redirected to a database object. The IQ.Suite always redirects it to a database object, which worsens the problem, but is not the reason.

Thanks again for your kind support,

Manuel

7

I was now able to recreate the problem with a very small modification of the standard mailfile:

Add a protected section to the memo-mask which contains the body field.

Protect the section e.g. with value " - protected". (see attachment)

Send a MIME-Mail with embedded image to that mailfile, open, change it (subject), save it.

Behold the missing image.

I created a case, but I thought, you should know :)

Best regards,

Manuel