Need a little help SMTP email send program syntax, terminating string.

[Jester]

I’m trying to resurrect an Arduino 8266 based email sender program that was working fine in 2023. Since then I changed email provider service and now with the new email, this program fails at step 32 of 33.
The program successfully:
  Logs into local wifi,
  Logs into email providers server,
  Passes authentication,
  Provides to and from email addresses,
Appears to send body text of email, and then the server times out at step 32 of 33

I think the problem is with the terminating string that is supposed to convey the end of the text in the email. In the 2023 version the text was terminated with a simple “.” On a new line, it appears that the 2024 version wants a <CR><LF>.<CR><LF>

Using the 2023 terminating string yields 550 5.7.1 Message Rejected (104) [F3EC52000E/1721829661-832712]

Using the 2024 terminating string appears to try and send the email , however it times out after about 20 seconds trying.

I have attached both the program (extension changed from .ino to .c) as well as a line by line comparison of the SMTP messages for the working and non working versions.

[Ian.M]

Your '2024' version sends

Code: [Select]

<.> preceded and followed by blank lines at the end of the message data.  This is not a valid RFC2821 end of mail data indicator (end of message body), so of course it hangs till timeout - its expecting more lines of data (message body).

The SMTP response code 550 is "Requested action not taken: mailbox unavailable (e.g., mailbox not found, no access, or command rejected for policy reasons)".  The fact you received it indicates the mail data block (message body) was correctly terminated,  The 5.7.1 *should* indicate why the message was rejected, but it seems to be a 'portmanteau'  code used for 'Unable to Relay', and various detections of spam or unacceptable content.  Obviously mail server admins don't want to elaborate!  Its also used for excessive message length.

[Jester]

Your '2024' version sends

Code: [Select]

<.> preceded and followed by blank lines at the end of the message data.  This is not a valid RFC2821 end of mail data indicator (end of message body), so of course it hangs till timeout - its expecting more lines of data (message body).

The SMTP response code 550 is "Requested action not taken: mailbox unavailable (e.g., mailbox not found, no access, or command rejected for policy reasons)".  The fact you received it indicates the mail data block (message body) was correctly terminated,  The 5.7.1 *should* indicate why the message was rejected, but it seems to be a 'portmanteau'  code used for 'Unable to Relay', and various detections of spam or unacceptable content.  Obviously mail server admins don't want to elaborate!  Its also used for excessive message length.

I appreciate your help.........

I was using the following termination based on receiving  -> 354 End data with <CR><LF>.<CR><LF>  from server just after sending the body text of the email  [step 30]

Code: [Select]

  espClient.println(F(""));
  espClient.println(F("<.>"));                                                             
  espClient.println(F(""));       

If I instead use:

Code: [Select]

  espClient.println(F(""));
  espClient.println(F("."));
  espClient.println(F(""));

The server responds with:
550 5.7.1 Message Rejected (104) [D5B2E20010/1721861964-932527]
-> 500 5.5.2 Error: bad syntax

So I think your saying the second termination version is actually correct, even though it's giving a 500 5.5.2 Error: bad syntax message?

I'm still lost. How do I determine what is causing the 5.5.2 bad syntax error message?

[madires]

From https://datatracker.ietf.org/doc/html/rfc5321#section-4.1.1.4:

   The mail data are terminated by a line containing only a period, that
   is, the character sequence "<CRLF>.<CRLF>", where the first <CRLF> is
   actually the terminator of the previous line (see Section 4.5.2).
   This is the end of mail data indication.  The first <CRLF> of this
   terminating sequence is also the <CRLF> that ends the final line of
   the data (message text) or, if there was no mail data, ends the DATA
   command itself (the "no mail data" case does not conform to this
   specification since it would require that neither the trace header
   fields required by this specification nor the message header section
   required by RFC 5322 [4] be transmitted).  An extra <CRLF> MUST NOT
   be added, as that would cause an empty line to be added to the
   message.  The only exception to this rule would arise if the message
   body were passed to the originating SMTP-sender with a final "line"
   that did not end in <CRLF>; in that case, the originating SMTP system
   MUST either reject the message as invalid or add <CRLF> in order to
   have the receiving SMTP server recognize the "end of data" condition.

[Ian.M]

You are sending an extra blank line after the ".".  The .println() method includes a terminating EOL sequence, which on your system is presumably <CR><LF> or it wouldn't work at all!
The extra <CR><LF> is known to cause 500 5.5.2 Error: bad syntax.

[Jester]

You are sending an extra blank line after the ".".  The .println() method includes a terminating EOL sequence, which on your system is presumably <CR><LF> or it wouldn't work at all!
The extra <CR><LF> is known to cause 500 5.5.2 Error: bad syntax.

Okay, I commented out both the preceding and post <CR><LF>.  The previous body text is sent with a espClient.println(), so that should serve as the preceding

Code: [Select]

//    espClient.println(F("")); 
     espClient.println(F("."));                                                              // and this should be the final  <CR><LF> after the '.'
//    espClient.println(F(""));     

When I run this, I no longer get the bad syntax error, so I guess that's progress and I'm probably correctly terminating the email, now I get the following error:

550 5.7.1 Message Rejected (104) [2396E20026/1721933938-342398]

Any suggestions?

[rich t]

Have you tried just talking to the SMTP server interactively using telnet?  You should be able to send commands by hand and see what works.

rtestardi@rich-pc3:/mnt/c/Users/rtest$ telnet smtp.hostedemail.com 587
Trying 216.40.42.128...
Connected to mail.hostedemail.com.
Escape character is '^]'.
220 smtp.hostedemail.com ESMTP

[Jester]

Have you tried just talking to the SMTP server interactively using telnet?  You should be able to send commands by hand and see what works.

rtestardi@rich-pc3:/mnt/c/Users/rtest$ telnet smtp.hostedemail.com 587
Trying 216.40.42.128...
Connected to mail.hostedemail.com.
Escape character is '^]'.
220 smtp.hostedemail.com ESMTP

I actually have tried, the time allowed before it kicks me out is quite short, so unless I type very quickly and accurately I get kicked out. This combined with having to enter base64 coded information makes it all but impossible. Keep in mind that it's failing at the last step.

[WatchfulEye]

550 5.7.1 normally means message blocked for security or content policy reasons (e.g. Suspected spam).

Try using a normal mail client on the same account to send the same message. If that doesn't work contact your mail providers tech support.

[westfw]

In the 2023 version the text was terminated with a simple “.” On a new line, it appears that the 2024 version wants a <CR><LF>.<CR><LF>

The termination has always been "a dot on a line by itself", where "lines" are delimited by CRLF (both of them.  "NVT ASCII" per the RFCs.)
using Arduino or similar, do ESPCLient.print("\r\n.\r\n");

[shapirus]

What's in the SMTP server logs?

If you have no access to those, try setting up a local one for testing and try to reproduce the problem with it using the same client.

[Jester]

What's in the SMTP server logs?

If you have no access to those, try setting up a local one for testing and try to reproduce the problem with it using the same client.

How would I go about doing that (I'm more of a hardware vs. Software guy). I did install WireShark (first time usage) on my Win7 PC yesterday and it only catches a few packets from the Arduino board. I guess this makes sense that it can only see the PC's WiFi traffic  I think there may be a way to have WireShark use the PC's WiFi to snoop on all WiFi traffic?

In the capture below the sender fails early when using port 465 so less packets, and runs to the end with port 587 so more packets, so my assumption is that WireShark is only capturing the initial broadcast from the sender. If I could see the entire sequence, that might provide some insight.

[zilp]

What's in the SMTP server logs?

If you have no access to those, try setting up a local one for testing and try to reproduce the problem with it using the same client.

How would I go about doing that (I'm more of a hardware vs. Software guy).

First, get yourself a Linux (or other Unix) system. Then install a random MTA, like exim or postfix.

I mean, I guess there are MTAs for Windows, too, but it's not really the environment such systems usually live in, and if you want to get more into building internet connected stuff, it's probably valuable to understand such environments anyway.

I did install WireShark (first time usage) on my Win7 PC yesterday and it only catches a few packets from the Arduino board. I guess this makes sense that it can only see the PC's WiFi traffic  I think there may be a way to have WireShark use the PC's WiFi to snoop on all WiFi traffic?

In the capture below the sender fails early when using port 465 so less packets, and runs to the end with port 587 so more packets, so my assumption is that WireShark is only capturing the initial broadcast from the sender. If I could see the entire sequence, that might provide some insight.

What you are seeing in that screenshot there has nothing to do with the SMTP connection itself, that's just your device discovering its IPv4 address (DHCP) and the MAC address of the IPv4 default gateway (ARP).

But your analysis is correct that if the interface that you are sniffing on is not on the path between the device and the target, then you'll generally only see broadcast packets and no actual payload traffic between the two.

On Linux, I would just configure the machine as a router or switch that sits between the two sides so that I could sniff the traffic. I have no idea whether that's a thing that Windows can do. If you have a switch that can do port mirroring, that could also be a way to get a look at the traffic, but your standard unmanaged office switch won't be able to do that.

Another approach that might work in this particular case could be to set up a simple TCP proxy, i.e., a thing that listens on some TCP port, and then establishes a second connection to a configurable target for every incoming connection and forwards all data between the two. On Linux, xinetd would be a piece of software that could do that. You could also write a simple verison of that yourself, of course. Then, you could have your device connect to that proxy, and that proxy connect to the target server. Then you wouldn't see the exact packets between the two, but still the same payload, so, if it's a payload syntax problem, that might be sufficient to diagnose the problem.

[Jester]

From https://datatracker.ietf.org/doc/html/rfc5321#section-4.1.1.4:

   The mail data are terminated by a line containing only a period, that
   is, the character sequence "<CRLF>.<CRLF>", where the first <CRLF> is
   actually the terminator of the previous line (see Section 4.5.2).
   This is the end of mail data indication.  The first <CRLF> of this
   terminating sequence is also the <CRLF> that ends the final line of
   the data (message text) or, if there was no mail data, ends the DATA
   command itself (the "no mail data" case does not conform to this
   specification since it would require that neither the trace header
   fields required by this specification nor the message header section
   required by RFC 5322 [4] be transmitted).  An extra <CRLF> MUST NOT
   be added, as that would cause an empty line to be added to the
   message.  The only exception to this rule would arise if the message
   body were passed to the originating SMTP-sender with a final "line"
   that did not end in <CRLF>; in that case, the originating SMTP system
   MUST either reject the message as invalid or add <CRLF> in order to
   have the receiving SMTP server recognize the "end of data" condition.

Thanks and great explanation

In the 2023 version the text was terminated with a simple “.” On a new line, it appears that the 2024 version wants a <CR><LF>.<CR><LF>

The termination has always been "a dot on a line by itself", where "lines" are delimited by CRLF (both of them.  "NVT ASCII" per the RFCs.)
using Arduino or similar, do ESPCLient.print("\r\n.\r\n");

Thanks for that. I think the incorrect termination is now resolved. Ian.M brought to my attention that the version I tried with

Code: [Select]

espClient.println(F(".")); actually allowed the sender to progress to the 550 message so it appears that the entire email did make it to the email server.  When I used the incorrect termination

Code: [Select]

espClient.println(F("<.>")); it appeared to be progressing further (took longer to provide the error message), however the last error message in that case before timeout is 354 indicating it was just waiting for the termination and never received it.

Using

Code: [Select]

]ESPCLient.print("\r\n.\r\n")  yields the same result as

Code: [Select]

espClient.println(F(".")); , both of these appear to correctly convey end of message. 

550 5.7.1 normally means message blocked for security or content policy reasons (e.g. Suspected spam).

Try using a normal mail client on the same account to send the same message. If that doesn't work contact your mail providers tech support.

So I think I have at least progressed to what hopefully will be the last issue to resolve, why is the email provider not letting the email in?
10:01:13.055 -> 550 5.7.1 Message Rejected (104) [F3EC52000E/1721829661-832712]

I spoke with email provider tech. support and they seem a bit lost. they asked me to send the identical email from Thunderbird on my PC as well as via their webmail and both of those work.

If anyone has further suggestions I'm all ears.

Thanks for helping me sort out the termination issue.

[WatchfulEye]

Try using wireshark to capture exactly what thunderbird sends and then program the arduino to exactly duplicate it.

[agehall]

Don’t ever rely on println()-like functions to do network related stuff. Always use the non-ln() versions and add your own line termination. This way you don’t get bitten by extra <cr><lf> being inserted or the wrong (just <cr> or <lf>) being sent.

[shapirus]

Another approach that might work in this particular case could be to set up a simple TCP proxy, i.e., a thing that listens on some TCP port, and then establishes a second connection to a configurable target for every incoming connection and forwards all data between the two. On Linux, xinetd would be a piece of software that could do that. You could also write a simple verison of that yourself, of course.

xinetd has an unnecessary learning curve for this case.

I would advise simpleproxy instead: https://packages.debian.org/bookworm/simpleproxy

Its name represents exactly what it is.

[Nominal Animal]

No, the sending code does not look right.  The EHLO domain is not right, the code does not check for DATA response, and there should be an empty line between message headers and the message body.

If it has worked at some point, it was only because the server was lax and allowed slightly malformed and/or unsafe messages through.

The emailResp() function is completely bogus. 

There are two types of responses from the SMTP server:
    NNN DASH TEXTSTRING CR LF
    NNN SPACE TEXTSTRING CR LF
where NNN is the three decimal digit status code, TEXTSTRING consists of zero or more ASCII codes 9 or 32-126 inclusive, DASH is ASCII code 45, SPACE is ASCII code 32, CR is ASCII code 13, and LF is ASCII code 10.

For each response (with the exact same NNN), the final line will be of the latter (space) form.  If there is more than that final line, the preceding lines will use the former (dash) form (so you can simply ignore the dash form lines).  Each emailResp() call should receive exactly one response, consisting of zero or more dash form lines, and exactly one space form.  It should also return the decimal value of NNN from that last line.  (You can use negative codes and codes 0-99 for errors, so that 100 and above are SMPT response codes.)

If you are sending a message to recipient@example.com from account@domain, the client should:

• Connect to the SMTP server using the appropriate port, and wait for a response.  The response should be 220.
• Send "EHLO domain\r\n", and wait for a response.  The response should be 220.
• Send "AUTH LOGIN\r\n", and wait for a response.  The response should be 334.
• Send "username\r\n" (where username is Base64-encoded form of the username), and wait for a response.  The response should be 334.
• Send "password\r\n" (where password is Base64-encoded form of the password), and wait for a response.  The response should be 235.
• Send "MAIL FROM:<account@domain>\r\n", and wait for a response.  The response should be 250.
• Send "RCPT TO:<recipient@example.com>\r\n", and wait for a response.  The response should be 250.
• Send "DATA\r\n", and wait for a response.  The response should be 354.
• Send each header line, for example "Subject: This is the message subject\r\n".  (Every header line must end with "\r\n".)
• Send empty line between headers and message body, "\r\n".
• Send message body.  If the body is not empty, it must end with "\r\n".
• Send ".\r\n", and wait for a response.  The response should be 250.
• Send "QUIT\r\n", and wait for a response.  The response should be 221.

Also note that each line must not exceed 998 characters (plus CR LF), but limiting to 78 characters (plus CR LF) would be even better.  Servers will check domain against the list of domains the specified username is allowed to send, and will reject the MAIL FROM command if they do not match.  In the case where the server knows about domain users, it may even check that username and account match.  So, you cannot just use whatever domain in a EHLO greeting!

[NorthGuy]

I spoke with email provider tech. support and they seem a bit lost. they asked me to send the identical email from Thunderbird on my PC as well as via their webmail and both of those work.

This is most likely about the message body, not about the SMTP protocol. May be you're missing some fields, or your "from" address does not match the user being authenticated. Or your headers are malformed in some way. Or they may use spam filters. Their spam filters might be very complex, they definitely can tell if the mail is composed by a well-known mail client or not.

The error code they show most likely encodes the cause of failure or contain a reference to a log record, but nowadays you can't expect tech support personnel to be able to decipher that.

[westfw]

Using Code:

Code: [Select]

ESPCLient.print("\r\n.\r\n")yields the same result as

Code: [Select]

espClient.println(F("."));both of these appear to correctly convey end of message.

Lucky.
Some of those println() type functions will produce alternate line endings.
(actually, "\n" isn't very safe either, in "print", since it will sometimes be expanded to a "full eol as appropriate for the device" or something.  Using espClient.write() would be better.  (write() should send raw bytes.  print() sometimes does interpretation.))

[HwAoRrDk]

I agree with others, if the server is responding with "550 5.7.1 Message Rejected", then the SMTP transaction is completing, but the message is being rejected because the server doesn't like the recipient or contents.

The "5.7.1" part after the "550" is what's known as an 'Enhanced Status Code' (that the server says it supports by responding with "250-ENHANCEDSTATUSCODES" in response to "EHLO") that decodes into a more specific meaning. In this case, it refers to a permanent "delivery not authorized, message refused" status ('permanent' meaning not likely to be resolved by resending the message in the current form), meaning "the sender is not authorized to send to the destination". See RFC3463 for details.

It may mean the user you're logging in with isn't allowed to send to the given destination address (unlikely, given it appears you're sending to yourself), or there's something incorrect about the recipient address. But it could also mean that there's something about the message contents itself that is incorrect or missing that means the server can't figure out how to deliver it.

I note from the posted code you are only sending a single header in the message: "Subject". You aren't including the usual "From" and "To", nor others like "Date" or "Content-Type". I would advise trying to add "From" and "To" and see if that helps. Make sure they match the "MAIL FROM" and "RCPT TO" being given to the server. For example:

Code: [Select]

  espClient.println(F("DATA"));
  espClient.println(F("To: <info@*****.com>"));
  espClient.println(F("From: <info@*****.com>"));
  espClient.println(F("Content-Type: text/plain; charset=UTF-8"));
  espClient.println(F("Subject: 8266 sender message 2024"));
  espClient.println();
  espClient.println(F("8266 sender message 2024"));

Obviously, you can't include a "Date" header if you have no clock or calendar facility on the Arduino, so that can be omitted. And of course, as others have mentioned, you need a blank line between headers and body of the message. (BTW, if memory serves, I think println() can be used without arguments to emit just line break characters - no need to pass an empty string.) I won't go into the mess that is other content types and MIME message formatting.

[westfw]

it could also mean that there's something about the message contents itself that is incorrect

I'll point out that SMTP mailers (even ESMTP) have been getting more and more paranoid careful over time, due to the widespread forgery of email for nefarious purposes.  So "something incorrect" could mean "The From address in your message doesn't exactly match the "Mail From" command in the SMTP dialog, or "the hostname used in one of those addresses doesn't exactly match the name lookup of your IP address."

[zilp]

Code: [Select]

  espClient.println(F("DATA"));
  espClient.println(F("To: <info@*****.com>"));
  espClient.println(F("From: <info@*****.com>"));
  espClient.println(F("Content-Type: text/plain; charset=UTF-8"));
  espClient.println(F("Subject: 8266 sender message 2024"));
  espClient.println();
  espClient.println(F("8266 sender message 2024"));

Obviously, you can't include a "Date" header if you have no clock or calendar facility on the Arduino, so that can be omitted. And of course, as others have mentioned, you need a blank line between headers and body of the message. (BTW, if memory serves, I think println() can be used without arguments to emit just line break characters - no need to pass an empty string.) I won't go into the mess that is other content types and MIME message formatting.

Note that the Content-Type header in the syntax used here is a MIME-Header, and as such, if you use it, you also have to add a MIME-Version header:

https://datatracker.ietf.org/doc/html/rfc2045#section-4

Also, you have to include a Date header, it is mandatory in RFC5322:

https://datatracker.ietf.org/doc/html/rfc5322#section-3.6

[zilp]

So "something incorrect" could mean "The From address in your message doesn't exactly match the "Mail From" command in the SMTP dialog, or "the hostname used in one of those addresses doesn't exactly match the name lookup of your IP address."

The mail domain never has to match the host name of the server (or else, no email provider could operate), but also, and more relevant here, checks of host names or IP addresses only apply to server-to-server connections, not to submission connections, where the client authenticates to the outbound server of their own domain in order to submit an email for delivery, as it is expected that email client will connect from just about anywhere, and also, it would serve no purpose, as the SMTP AUTH credentials do already reliably establish the identity of the client.

For server-to-server, you might also want to setup SPF, DKIM, and DMARC, but none of that is relevant for submission connections.

[Jester]

I agree with others, if the server is responding with "550 5.7.1 Message Rejected", then the SMTP transaction is completing, but the message is being rejected because the server doesn't like the recipient or contents.

The "5.7.1" part after the "550" is what's known as an 'Enhanced Status Code' (that the server says it supports by responding with "250-ENHANCEDSTATUSCODES" in response to "EHLO") that decodes into a more specific meaning. In this case, it refers to a permanent "delivery not authorized, message refused" status ('permanent' meaning not likely to be resolved by resending the message in the current form), meaning "the sender is not authorized to send to the destination". See RFC3463 for details.

It may mean the user you're logging in with isn't allowed to send to the given destination address (unlikely, given it appears you're sending to yourself), or there's something incorrect about the recipient address. But it could also mean that there's something about the message contents itself that is incorrect or missing that means the server can't figure out how to deliver it.

I note from the posted code you are only sending a single header in the message: "Subject". You aren't including the usual "From" and "To", nor others like "Date" or "Content-Type". I would advise trying to add "From" and "To" and see if that helps. Make sure they match the "MAIL FROM" and "RCPT TO" being given to the server. For example:

Code: [Select]

  espClient.println(F("DATA"));
  espClient.println(F("To: <info@*****.com>"));
  espClient.println(F("From: <info@*****.com>"));
  espClient.println(F("Content-Type: text/plain; charset=UTF-8"));
  espClient.println(F("Subject: 8266 sender message 2024"));
  espClient.println();
  espClient.println(F("8266 sender message 2024"));

Obviously, you can't include a "Date" header if you have no clock or calendar facility on the Arduino, so that can be omitted. And of course, as others have mentioned, you need a blank line between headers and body of the message. (BTW, if memory serves, I think println() can be used without arguments to emit just line break characters - no need to pass an empty string.) I won't go into the mess that is other content types and MIME message formatting.

YES.... that was it


The missing to from was the problem.

Super Duper big thank you to everyone that helped out.

[zilp]

If you are sending a message to recipient@example.com from account@domain, the client should:

• Connect to the SMTP server using the appropriate port, and wait for a response.  The response should be 220.
• Send "EHLO domain\r\n", and wait for a response.  The response should be 220.

Servers will check domain against the list of domains the specified username is allowed to send, and will reject the MAIL FROM command if they do not match.  In the case where the server knows about domain users, it may even check that username and account match.  So, you cannot just use whatever domain in a EHLO greeting!

This is not correct. The name specified in the EHLO has absolutely nothing to do with the email addresses of the submitted email, it is supposed to be the host name of the system submitting the email, or an IP literal in case the host name is unknown:

https://datatracker.ietf.org/doc/html/rfc5321#section-4.1.1.1

In practice, for submission connections, servers will accept anything that is syntactically valid, because in practice, most SMTP submission clients nowadays can't really send anything particularly meaningful anyway (end user devices usually don't have DNS names, and often are behind NAT and thus at best know their RFC1918 address on the random LAN that the happen to be connected to).

Servers often will check that the account is allowed to use the source address, but that is only based on the authentication credentials. Also, if you were to use TLS, servers might use SNI to distinguish accounts (so that you can have the same account name for distinct accounts of different mail serever names hosted on the same physical server and the same IP addresses).

• Send "AUTH LOGIN\r\n", and wait for a response.  The response should be 334.
• Send "username\r\n" (where username is Base64-encoded form of the username), and wait for a response.  The response should be 334.
• Send "password\r\n" (where password is Base64-encoded form of the password), and wait for a response.  The response should be 235.

For one, technically, you would first have to check in the EHLO response which authentication mechanisms are supported. Of course, if you only support one anyway, I guess you might as well just try that one.

But also,  LOGIN is obsoleted, and the non-obsolete plain-text mechanism, PLAIN, makes things quite a bit simpler, because you send username and password in one auth string, which then also can be supplied as the initial-response in the AUTH command itself, so there is no need to handle the command continuation.

• Send message body.  If the body is not empty, it must end with "\r\n".
• Send ".\r\n", and wait for a response.  The response should be 250.

I guess it should be noted that if a line of the message body starts with a '.', then you have to double that to '..', or else your mail might get corrupted (and, depending on the circumstances, you might have a security problem).

[Jester]

Code: [Select]

  espClient.println(F("DATA"));
  espClient.println(F("To: <info@*****.com>"));
  espClient.println(F("From: <info@*****.com>"));
  espClient.println(F("Content-Type: text/plain; charset=UTF-8"));
  espClient.println(F("Subject: 8266 sender message 2024"));
  espClient.println();
  espClient.println(F("8266 sender message 2024"));

Obviously, you can't include a "Date" header if you have no clock or calendar facility on the Arduino, so that can be omitted. And of course, as others have mentioned, you need a blank line between headers and body of the message. (BTW, if memory serves, I think println() can be used without arguments to emit just line break characters - no need to pass an empty string.) I won't go into the mess that is other content types and MIME message formatting.

Note that the Content-Type header in the syntax used here is a MIME-Header, and as such, if you use it, you also have to add a MIME-Version header:

https://datatracker.ietf.org/doc/html/rfc2045#section-4

Also, you have to include a Date header, it is mandatory in RFC5322:

https://datatracker.ietf.org/doc/html/rfc5322#section-3.6

It's probably just a matter of time before this will stop working (again) as they keep "uppping" the requirements, however for now it's working.
Funny how you guys can diagnose and solve a problem from half way around the world, but the actual email provider can't.

Made my day guys/gals

[zilp]

It's probably just a matter of time before this will stop working (again) as they keep "uppping" the requirements, however for now it's working.

If you follow the standards, things should essentially never break.

Funny how you guys can diagnose and solve a problem from half way around the world, but the actual email provider can't.

Well, the fact that they accept connections without TLS at all kinda already tells you that you maybe should be looking for a different provider, I guess. Though then this client wouldn't work any more, of course ...

[Nominal Animal]

In practice, for submission connections, servers will accept anything that is syntactically valid [in EHLO domain], because in practice, most SMTP submission clients nowadays can't really send anything particularly meaningful anyway (end user devices usually don't have DNS names, and often are behind NAT and thus at best know their RFC1918 address on the random LAN that the happen to be connected to).

Yes, the RFC mandates that SMTP servers don't reject messages just because they don't like the EHLO command, but practice trumps RFC.

But, for a local IoT server implementing a mail transfer agent, forwarding the messages to the appropriate mail servers, it really should verify the client IP address matches the local address, the EHLO domain is either that same IP address or a matching domain name that matches the filter set in e.g. Exim auth_advertise_hosts, or otherwise your IoT server may be exploitable to forward spam by the ton because the IoT appliances source code revealed the username and password.

I've mostly been on the other side of this, configuring mail transfer agents on servers, so I definitely "err" on the safe side here.  You do not want your IoT device firmware to reveal how to use the related SMTP servers as spam relays; trust me on this.

Hmm.. for a better EHLO advice, I think using either the configured (host and) domain name, or the IP address if unknown or not configured, would work best.  This way, if the mail provider requires something here, the user can configure their IoT box to comply.

For one, technically, you would first have to check in the EHLO response which authentication mechanisms are supported. Of course, if you only support one anyway, I guess you might as well just try that one.

But also,  LOGIN is obsoleted, and the non-obsolete plain-text mechanism, PLAIN, makes things quite a bit simpler, because you send username and password in one auth string, which then also can be supplied as the initial-response in the AUTH command itself, so there is no need to handle the command continuation.

Fully agreed.

I guess it should be noted that if a line of the message body starts with a '.', then you have to double that to '..', or else your mail might get corrupted (and, depending on the circumstances, you might have a security problem).

Yep, true.


I guess a better suggestion for sending email from an IoT device would be:

• Connect to the SMTP server using the appropriate port, and wait for a response.  The response should be 220.
• Send "EHLO client\r\n", and wait for a response.  The response should be 220.
  The response text contains "AUTH" followed by whitespace-separated authentication methods, which should include "PLAIN".
• Send "AUTH PLAIN creds\r\n", and wait for a response.  The response should be 235.
  Try not to hardcode creds in your firmware, and instead treat it as a (privileged) configuration string.
• Send "MAIL FROM:<sender>\r\n", and wait for a response.  The response should be 250.
• Send "RCPT TO:<recipient>\r\n", and wait for a response.  The response should be 250.
• Send "DATA\r\n", and wait for a response.  The response should be 354.
• Send each header line, for example "Subject: This is the message subject\r\n".  (Every header line must end with "\r\n".)
• Send empty line between headers and message body, "\r\n".
• Send message body.  If a line begins with a ., it should be doubled (to ..).  If the body is not empty, it must end with "\r\n".
• Send ".\r\n", and wait for a response.  The response should be 250.
• Send "QUIT\r\n", and wait for a response.  The response should be 221.

where host is the fully-qualified host name if configured, or the IP address if not configured; creds is Base64 encoded username followed by an ASCII null character followed by the password; sender is the sender mail address matching creds (and host if the SMTP server does strict EHLO checking); and recipient is the recipient mail address.

RFC 5322 describes message headers you can use, and how those headers and body can be formatted.

[quadtech]

If your environment allows, it would be preferable to use a local mail relay host to which the IOT device talks,
and the mail relay host  (for ex - linux running postfix/sendmail etc) is the one which actually talks to the ISP mail gateway. That would allow setting up TLS etc on the relay host to the external mail gateway.

[zilp]

Yes, the RFC mandates that SMTP servers don't reject messages just because they don't like the EHLO command, but practice trumps RFC.

No, it doesn't. Submission servers generally do not check, because there is nothing that they could check. Your average MUA behind NAT has no real hostname to send, and has an RFC1918 address that the server can not see, and doesn't know what IP address the server sees. So, at best, they could, following the RFC, send an IP literal with that RFC1918 address. And what is the server supposed to do with that? There is no way for the server to know whether that is actually the IP address of the device. It obviously won't match the remote address on the TCP socket. Checking against the remote address on the TCP socket makes no sense either, because the MUA doesn't know that address so that it could send it in the HELO/EHLO.

I think one could say that it has developed a conensus of sorts that the best practice is to send a localhost IP literal ('[127.0.01]') in submission EHLO. That seems to work even with rare weird servers that do have some restrictions on permissible EHLO parameters, is perfectly RFC compliant, and also doesn't leak any information (like local IP addresses, say).

Things are different with server-to-server connections, but that is not what this thread is about.

But, for a local IoT server implementing a mail transfer agent, forwarding the messages to the appropriate mail servers, it really should verify the client IP address matches the local address, the EHLO domain is either that same IP address or a matching domain name that matches the filter set in e.g. Exim auth_advertise_hosts, or otherwise your IoT server may be exploitable to forward spam by the ton because the IoT appliances source code revealed the username and password.

Are you talking about a outbound relay accepting mails from IoT devices on the local network?! That certainly should ignore the EHLO parameter. How you should do client authentication in that case depends on the circumstances, but EHLO is irrelevant for that. Using auth_advertise_hosts for that doesn't make a whole lot of sense, though. Either you have actual (per-device provisioned) credentials, then, there is no point limiting authentication based on IP addresses, or you don't, then you can just skip the SMTP AUTH part completely and just use ACLs to limit relaying to specific IP prefixes.

Hmm.. for a better EHLO advice, I think using either the configured (host and) domain name, or the IP address if unknown or not configured, would work best.  This way, if the mail provider requires something here, the user can configure their IoT box to comply.

Really, an email provider that didn't accept [127.0.0.1] wouldn't work with a lot of MUAs out there already, so there really is no need to make things configurable.

[gmb42]

If you're looking for a test SMTP server on Windows (and other OSs) I would recommend smtp4dev

[peter-h]

How different would the foregoing be with a fairly modern smtp auth service such as authsmtp.com? These also do nowadays pretty much essential stuff like DKIM. Google is now dumping all incoming mail unless it has either SPF (which is not easy to keep valid over time) or DKIM.

The connection can be plain text username and pwd; no need for TLS.

[zilp]

How different would the foregoing be with a fairly modern smtp auth service such as authsmtp.com? These also do nowadays pretty much essential stuff like DKIM. Google is now dumping all incoming mail unless it has either SPF (which is not easy to keep valid over time) or DKIM.

I have no idea what you mean by "smtp auth service". authsmtp.com seem to be an email outbound provider!?

As for DKIM and SPF, that's something that almost all email providers support nowadays, and also, it is not difficult to "keep SPF valid". Actually, there is nothing to keep valid about it, as long as your outbound servers don't change, SPF doesn't need to change either, and if your email provider changes the servers, they should take care of changing SPF, too.

The connection can be plain text username and pwd; no need for TLS.

With any reasonable email provider, that really shouldn't be supported anymore. Though no idea what you are trying to say here, really.

[peter-h]

That's a fairly usual reply tone from you zilp.

Yes of course authsmtp.com are an outbound email provider.