User agent (and other headers) for email on embedded systems?

[David_AVD]

What headers are recommended for emails sent with an embedded system?

For example I can use an Arduino to log into a host I control and send an email with minimal headers.

One thing I've noticed though is that some recipients treat it as spam if there are no user-agent or content-type headers.

Is there a specific user-agent for embedded systems, or is any name you make up just as likely to be accepted?

[krho]

You can make it up. I wouldn't be sending an email from embedded system. It's going to end in spam in most cases.

[David_AVD]

So far it seems to be ok with a user-agent string of "embedded".

When there was no user-agent and content-type headers it did indeed end up in the spam folder.

Sending email from embedded systems is quite common place these days; alerts from NAS units, routers, etc.

[alm]

If you send e-mail triggered by events, please implement some sort of rate limiting. I have heard of people being disconnected by their ISP because their NAS/IP camera/router/whatever exceeded their daily limit for number of e-mails. And getting a dozen e-mails telling you to do something is not really any less effective than getting a hundred.

[David_AVD]

The proposed system isn't for events, just a daily or weekly report.  I just mentioned events as one usage case already out there.

I'm not aware of any email per period limit for the hosted service I have.   There is a "Monthly Bandwidth Transfer" limit of course.

Either way, taking precautions to limit retries, etc is definitely on the implementation list. 

[Monkeh]

Rather than sending an email directly from the device, have a proper host send emails for it. Gather data for such via, say, MQTT.

[SeanB]

I would emulate a more common Outlook or such header, simply because a lot of mail systems tend to only think that there is only a single mail system sender available.  As well you will need to actually have proper response to greylisting that just says server busy, so it will need to remember the last message and try 15 minutes later if it has not been accepted, and to time out gracefully on errors as well. Rate limit your mail output, a lot of ISP's will disconnect you as a spammer if you exceed a limit for emails per minute as well.

[HwAoRrDk]

So, to clarify, your embedded system is connecting via SMTP to your own mail server, which is then forwarding on to the recipient's MX host? And not sending direct via SMTP to the recipient's MX host?

If so, I'd say that the chances of your messages being classified as spam or rejected by the recipient have more to do with how your own mail host is set up. Things like IP address reputation, whether you have SPF, DKIM, etc.

The only way I'd imagine X-Mailer headers to make any difference to spam classification is where it looks 'odd' in conjunction to the rest of the message. Mail filtering algorithms are far more likely to look at the message content when making classification decisions.

If it were me, I'd not put any X-Mailer header at all, and instead look at the things I previously mentioned with regard to the mail host.

Oh, and one more thing: if the body of the message contains URLs (or content that looks like one), I find it helps to avoid the message content type being HTML, and make it plain text. Many mail filtering systems examine anything that looks like a URL to see if it's 'bad', or simply mark messages down for having navigable links at all.

[alm]

I would be careful faking e-mail headers. Faking headers is something pretty much limited to spammers. Something that claims to come from Outlook but was clearly not sent using Outlook could be a red flag.

[hans]

One thing I've noticed though is that some recipients treat it as spam if there are no user-agent or content-type headers.

Google and other service providers also treat mail as spam if it's not sent over a SSL connection.
It also checks SSL certificates if you're running your own server. Also one of the reasons why I don't run my own mail server (yet).. too tedious.

[David_AVD]

So, to clarify, your embedded system is connecting via SMTP to your own mail server, which is then forwarding on to the recipient's MX host? And not sending direct via SMTP to the recipient's MX host?

Correct.  The device is connecting via port 25 on an email server that I have credentials for, and it in turn takes care of the actual delivery.  That way the effort required by the device is fairly trivial, and multiple devices can use the server as a common email gateway.

[David_AVD]

Oh, and one more thing: if the body of the message contains URLs (or content that looks like one), I find it helps to avoid the message content type being HTML, and make it plain text. Many mail filtering systems examine anything that looks like a URL to see if it's 'bad', or simply mark messages down for having navigable links at all.

Good to know.  At this stage the content should only be a list of statistics.  I was looking at using a HTML table, but maybe a plain text one would be safer.

[hli]

What you at least need are From:, To: and Subject:. It might be useful to add Date: and Message-Id. If you need to, set a Content-Type, but best is to send plain text. Nothing more is needed, and adding them won't make your email less likely to look like spam.
Note that all the other headers you are typically seeing in a mail your received are added by one of the mail servers inbetween (e.g. the outgoing of the sender or your own receiving mail server).

[C]

So, to clarify, your embedded system is connecting via SMTP to your own mail server, which is then forwarding on to the recipient's MX host? And not sending direct via SMTP to the recipient's MX host?

Correct.  The device is connecting via port 25 on an email server that I have credentials for, and it in turn takes care of the actual delivery.  That way the effort required by the device is fairly trivial, and multiple devices can use the server as a common email gateway.

If Port 25 is accessible from Internet, not a good idea.
You have a email server doing forwarding if the email goes to a different system, A possible spam relay. Should Lock down the email server if this is case. Could be a good idea to filter the received email before passing it on.
In the process of filtering, you could do many automated things on the server.  The list of statistics for each device could be added to a database with a web page for easy reading.
 

[alm]

Consumer ISPs will often block outbound connections to port 25 unless it is to their smart host. So using port 587 or 465 (with SSL). And the mail server should require authentication on port 587 and 465. And since authentication usually involves passwords, traffic to port 587 should be encrypted with STARTTLS. This can be a pain for the smaller embedded systems.

[C]

Consumer ISPs will often block outbound connections to port 25 unless it is to their smart host. So using port 587 or 465 (with SSL). And the mail server should require authentication on port 587 and 465. And since authentication usually involves passwords, traffic to port 587 should be encrypted with STARTTLS. This can be a pain for the smaller embedded systems.

Very true, but if David has a server receiving via port 25, little prevents that server from checking it is one of his devices sending email on vis ports 465 or 587.

[David_AVD]

For this initial testing I'm using port 25 with authorisation (strings as base 64), but not SSL.  What sort of processing power does SSL involve for the micro?

The other option I suggested to the client is a central (web) server that collects the data and then either emails it to the end user, or allows authenticated access to view it.

There will be multiple end users in the long term, so that's why they are asking for the simplest system possible to set up.

Each end user could have just one or many remote devices that they want to get reports from.  Email seemed the easiest to do that.

Either way a central collection / relay point is involved.  I'm open to other ideas on how to get the data to the end users.

[HwAoRrDk]

What you at least need are From:, To: and Subject:. It might be useful to add Date: and Message-Id. If you need to, set a Content-Type, but best is to send plain text. Nothing more is needed, and adding them won't make your email less likely to look like spam.

If the embedded system has no time- or date-keeping, then setting a Date header will of course be impossible; no big deal, as OP's mail host should add one itself when absent. Similarly, there's little point setting a Message-Id header, as (I presume) there will be no archiving/record-keeping of sent messages on the embedded system, so any generated ID will be meaningless. The only reason I can think of for setting one is if the ID is generated by a deterministic function (i.e. has no random element), so you can use it to later track the circumstances in which the message was composed and sent. Otherwise, again, the mail host will generate and add one if absent. I would call Content-Type a must, as this will allow you to declare the character set of the text within the message. Otherwise, if your message is anything other than pure ASCII (which is what it will be assumed to be without this header), mail clients may not render the text properly.

One extra suggestion for another header is Return-Path. If the mail account/address you're sending from is one that you may not wish to receive (or is not capable of receiving) any undeliverable bounces, you can set this header with an alternate address. For example, you may want to set this to your own address, so you know if the e-mails the embedded system is sending out aren't being delivered.

[cdev]

In addition to SPF and DKIM, there is also dmarc which tells (large) receiving servers how you want them to handle non-conforming email. You can get a daily XML-formatted report back as to how its been handled and why.

Its really an extremely different Internet than it was a few years ago when anybody could send email to anybody without any authentication that they were actually who they said they were happening at all.

A lot of email is silently deleted now. If you want your email to all get to its intended recipients, there are a lot of hoops they make you jump through.

------

I need to look into better ways to check incoming email for signs of spoofing.

 Recently Ive gotten some emails that I'm fairly sure did not come from people they claimed to be from, even though the headers appeared genuine. (outlook.com, gmail.com, aol.com senders) 

[David_AVD]

This is what I'm doing now.  Good point about the return path.

Code: [Select]

User-Agent: Embedded;
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
To: xxxx xxxx<xxx@xxxxxx.com>
From: xxx xxx <xxx@xxx.com>
Subject: Test Message


[David_AVD]

I tried adding in a Return-Path header, but it seems to be getting overridden by the mail server.

[Monkeh]

I tried adding in a Return-Path header, but it seems to be getting overridden by the mail server.

Overridden to what? It may just be SRS at work.

[David_AVD]

I tried adding in a Return-Path header, but it seems to be getting overridden by the mail server.

Overridden to what? It may just be SRS at work.

I put the Reply-Path just after the Content-Encoding, but in the received email it's at the top and set to the email address that was in the From field.

[Monkeh]

Ah, okay, misunderstanding.

Return-Path is set by the server. It is not sent to the contents of the 'From' field, but typically starts out as the argument of the 'MAIL FROM' command. They are not equivalent!

Return-Path is for handling things going wrong, for example bounces. If there are relays involved, it gets changed by the relay. From, however, is the email address of the person sending the mail - and Reply-To is where a reply should be sent, if not the sender.

If you want bounces and other errors to go to a different mailbox, use it as the argument to the 'MAIL FROM' command.

[C]

I have no idea how much security you need. When someone wants to break the system they try to find the easy spot to hack.

For E-mail, you have plain text at each e-mail server it goes through.
 
Also remember that with wireless I could receive the signals from a long distance for a long time.

Think of security this way, If you can tell someone how everything works and it's still next to imposable for wrong person to get in, It might be good security.

In long run you will probably want or need each device to be unique.

If you are going to use email then an email account for each device.

might want to look into a public key/private key setup

Might think of email to user as a notice to look at secure web site for info.

[David_AVD]

I was confused as well.     I was actually thinking of Reply-To which I've now added and that works.

I've been thinking of the email account side of things.   What could be done is have a separate email address and password for each end user (or installation). eg. 123@xyz.com, 456@xyz.com, etc.

Each remote device would have those credentials entered by the installer, along with the recipient details.  That could give some additional control.

Moving to SSL would seem like a significant effort on a micro.  How likely is unauthorised usage (spammers) when using port 25 with authorisation?

The proposed internet connection is a 4G modem.

[Monkeh]

I was confused as well.     I was actually thinking of Reply-To which I've now added and that works.

And is only of any utility if you want to send a message in response. It will not get any failure messages sent back to an email you can read.

I've been thinking of the email account side of things.   What could be done is have a separate email address and password for each end user (or installation). eg. 123@xyz.com, 456@xyz.com, etc.

Each remote device would have those credentials entered by the installer, along with the recipient details.  That could give some additional control.

Moving to SSL would seem like a significant effort on a micro.  How likely is unauthorised usage (spammers) when using port 25 with authorisation?

The proposed internet connection is a 4G modem.

Welcome to the joys of the internet of insecure things.

If anyone can sniff out the connection at any stage, they can use your login credentials. However, you could lock down the mail server to only allow these credentials to relay to specific addresses. This greatly limits the damage, unless they can fake valid emails from your device and cause hassle that way.

[David_AVD]

One thing that does bother me is someone having to maintain an email server for the life of the products.

The same would apply if I had the remote devices accessing a web server with POSTs.

I'll have to discuss the subscription service model with my client.

The only way I can see around that is to have the end users set up their own gmail accounts, but I haven't looked to see if they support non SSL logins.

[BradC]

Just fyi. This is an E-mail generated by one of my UPS network cards.
I've removed the stuff inserted by the mail servers it traverses to get to me to leave the basics the card sends.

Code: [Select]

MIME-Version: 1.0
Date: Tue, 27 Jun 2017 14:25:46 GMT
To: root@srv.home
From: apc-gfups@ups-gf.home
Cc:
Subject: System: Console user 'apc' logged out from serial port.
X-Mailer: Email Server
Content-Type: text/plain; charset=iso-8859-1
Content-Transfer-Encoding: 7bit
Message-Id: <E1dPrR0-00084J-3J@srv.home>

Name     : APC XXXXX

APC devices are used worldwide by IT departments and complaints about functionailty or lack thereof are taken pretty seriously.
I will say though, they are capable of SSL/TLS, although currently there are known issues with older APC units and Gmail. It doesn't affect me as all my stuff is internal.

[krho]

The only way I can see around that is to have the end users set up their own gmail accounts, but I haven't looked to see if they support non SSL logins.

Gmail doesn't support non ssl logins. And along with that you need at least TLS 1.0 support, with pretty strong ciphers. So forget you can do that on non beefy mcu.

The only way you can do this okayish is for your client to require setting up their own server. I'd think of providing a http script that can receive encrypted message over plain http link (there are safe encryption algorithms) that could be run on mcu. And that script can then relay the message over the smtp or just use smtp sending services like sendpost, mailgun etc.

[HwAoRrDk]

I tried adding in a Return-Path header, but it seems to be getting overridden by the mail server.

Ah, my mistake. I forgot the outbound mail host would want to override that. Really only applicable for direct-send in that case.

[David_AVD]

Thanks for all the help guys.  I seem to have a working solution now.