TIdMessage.LoadFromStream: Missing fields

[rlebeau]

I then call  IMAP.UIDRetrieve(IntToStr(IMAP.MailBox.SearchResult[i]), MSG);
I can now read all fields like From, Subject etc. fine.

Internally, TIdIMAP4.UIDRetrieve() downloads the raw bytes of the email into a TMemoryStream, and then calls TIdMessage.LoadFromStream() with the AUsesDotTransparency parameter set to False (its default is True instead).

Then the entire email is loaded into a Stream using
IMAP.UIDRetrieveNoDecodeToStream(IntToStr(IMAP.MailBox.SearchResult[i]), AStream);

Internally, TIdIMAP4.UIDRetrieveNoDecodeToStream() creates a temporary TIdMessage and then downloads the email using the same logic as TIdIMAP4.UIDRetrieve(), except that it sets both TIdMessage.NoDecode and TIdMessage.NoEncode to True. Then, it calls TIdMessage.SaveToStream() with the AUsesDotTransparency parameter set to True.

You really need AUsesDotTransparency to be set to False instead in this situation.  There is a TODO comment about this in the source code for TIdIMAP4.UIDRetrieveNoDecodeToStream() (and all of the other TIdIMAP4.(UID)RetrieveNoDecodeTo...() methods):

Code:

{RLebeau 12/09/2012: NOT currently using the same workaround here that
is being used in AppendMsg() to avoid SMTP dot transparent output from
TIdMessage.SaveToStream().  The reason for this is because I don't
know how this method is being used and I don't want to break anything
that may be depending on that transparent output being generated...}
...
{TODO: add an optional parameter to specify whether dot transparency
should be used or not, and then pass that to SaveToStream(). Or better,
just deprecate this method and implement a replacement that downloads
the message directly to the file without dot transparency, since it
has no meaning in IMAP. InternalRetrieve() uses an internal stream
anyway to receive the data, so let's just cut out TIdMessage here...}

This Stream is then saved to file, as an .eml file.
I can load this file with an email program fine, all data complete.

Most other programs are not expecting an .EML file to have its data escaped with dot-transparency.  That is a detail of the POP3 and SMTP protocols only, IMAP doesn't use dot-transparency at all.  But TIdMessage does by default, so watch out for that!

The same Stream is internally kept.
But when I afterwards use an IdMessage.LoadFromStream, all fields like From, Subject, etc. are all empty.
The LoadFromStream does not seem to initialize the fields.

Are you resetting the stream's Position back to 0 before you load the stream?  TIdMessage.LoadFromStream() will not do that for you.

Also, make sure that you always call TIdMessage.LoadFromStream() using the same AUsesDotTransparency parameter value that was used when TIdMessage.SaveToStream() was called.

It seems only the IMAP does this, during the call to IMAP.UIDRetrieve.

TIdIMAP4.UIDRetrieve() and TIdIMAP4.UIDRetrieveNoDecodeToStream() are using TIdMessage in different ways.