User agent (and other headers) for email on embedded systems?

[David_AVD]

What headers are recommended for emails sent with an embedded system?

For example I can use an Arduino to log into a host I control and send an email with minimal headers.

One thing I've noticed though is that some recipients treat it as spam if there are no user-agent or content-type headers.

Is there a specific user-agent for embedded systems, or is any name you make up just as likely to be accepted?

[krho]

You can make it up. I wouldn't be sending an email from embedded system. It's going to end in spam in most cases.

[David_AVD]

So far it seems to be ok with a user-agent string of "embedded".

When there was no user-agent and content-type headers it did indeed end up in the spam folder.

Sending email from embedded systems is quite common place these days; alerts from NAS units, routers, etc.

[alm]

If you send e-mail triggered by events, please implement some sort of rate limiting. I have heard of people being disconnected by their ISP because their NAS/IP camera/router/whatever exceeded their daily limit for number of e-mails. And getting a dozen e-mails telling you to do something is not really any less effective than getting a hundred.

[David_AVD]

The proposed system isn't for events, just a daily or weekly report.  I just mentioned events as one usage case already out there.

I'm not aware of any email per period limit for the hosted service I have.   There is a "Monthly Bandwidth Transfer" limit of course.

Either way, taking precautions to limit retries, etc is definitely on the implementation list. 

[Monkeh]

Rather than sending an email directly from the device, have a proper host send emails for it. Gather data for such via, say, MQTT.

[SeanB]

I would emulate a more common Outlook or such header, simply because a lot of mail systems tend to only think that there is only a single mail system sender available.  As well you will need to actually have proper response to greylisting that just says server busy, so it will need to remember the last message and try 15 minutes later if it has not been accepted, and to time out gracefully on errors as well. Rate limit your mail output, a lot of ISP's will disconnect you as a spammer if you exceed a limit for emails per minute as well.

[HwAoRrDk]

So, to clarify, your embedded system is connecting via SMTP to your own mail server, which is then forwarding on to the recipient's MX host? And not sending direct via SMTP to the recipient's MX host?

If so, I'd say that the chances of your messages being classified as spam or rejected by the recipient have more to do with how your own mail host is set up. Things like IP address reputation, whether you have SPF, DKIM, etc.

The only way I'd imagine X-Mailer headers to make any difference to spam classification is where it looks 'odd' in conjunction to the rest of the message. Mail filtering algorithms are far more likely to look at the message content when making classification decisions.

If it were me, I'd not put any X-Mailer header at all, and instead look at the things I previously mentioned with regard to the mail host.

Oh, and one more thing: if the body of the message contains URLs (or content that looks like one), I find it helps to avoid the message content type being HTML, and make it plain text. Many mail filtering systems examine anything that looks like a URL to see if it's 'bad', or simply mark messages down for having navigable links at all.

[alm]

I would be careful faking e-mail headers. Faking headers is something pretty much limited to spammers. Something that claims to come from Outlook but was clearly not sent using Outlook could be a red flag.

[hans]

One thing I've noticed though is that some recipients treat it as spam if there are no user-agent or content-type headers.

Google and other service providers also treat mail as spam if it's not sent over a SSL connection.
It also checks SSL certificates if you're running your own server. Also one of the reasons why I don't run my own mail server (yet).. too tedious.

[David_AVD]

So, to clarify, your embedded system is connecting via SMTP to your own mail server, which is then forwarding on to the recipient's MX host? And not sending direct via SMTP to the recipient's MX host?

Correct.  The device is connecting via port 25 on an email server that I have credentials for, and it in turn takes care of the actual delivery.  That way the effort required by the device is fairly trivial, and multiple devices can use the server as a common email gateway.

[David_AVD]

Oh, and one more thing: if the body of the message contains URLs (or content that looks like one), I find it helps to avoid the message content type being HTML, and make it plain text. Many mail filtering systems examine anything that looks like a URL to see if it's 'bad', or simply mark messages down for having navigable links at all.

Good to know.  At this stage the content should only be a list of statistics.  I was looking at using a HTML table, but maybe a plain text one would be safer.

[hli]

What you at least need are From:, To: and Subject:. It might be useful to add Date: and Message-Id. If you need to, set a Content-Type, but best is to send plain text. Nothing more is needed, and adding them won't make your email less likely to look like spam.
Note that all the other headers you are typically seeing in a mail your received are added by one of the mail servers inbetween (e.g. the outgoing of the sender or your own receiving mail server).

[C]

So, to clarify, your embedded system is connecting via SMTP to your own mail server, which is then forwarding on to the recipient's MX host? And not sending direct via SMTP to the recipient's MX host?

Correct.  The device is connecting via port 25 on an email server that I have credentials for, and it in turn takes care of the actual delivery.  That way the effort required by the device is fairly trivial, and multiple devices can use the server as a common email gateway.

If Port 25 is accessible from Internet, not a good idea.
You have a email server doing forwarding if the email goes to a different system, A possible spam relay. Should Lock down the email server if this is case. Could be a good idea to filter the received email before passing it on.
In the process of filtering, you could do many automated things on the server.  The list of statistics for each device could be added to a database with a web page for easy reading.
 

[alm]

Consumer ISPs will often block outbound connections to port 25 unless it is to their smart host. So using port 587 or 465 (with SSL). And the mail server should require authentication on port 587 and 465. And since authentication usually involves passwords, traffic to port 587 should be encrypted with STARTTLS. This can be a pain for the smaller embedded systems.

[C]

Consumer ISPs will often block outbound connections to port 25 unless it is to their smart host. So using port 587 or 465 (with SSL). And the mail server should require authentication on port 587 and 465. And since authentication usually involves passwords, traffic to port 587 should be encrypted with STARTTLS. This can be a pain for the smaller embedded systems.

Very true, but if David has a server receiving via port 25, little prevents that server from checking it is one of his devices sending email on vis ports 465 or 587.

[David_AVD]

For this initial testing I'm using port 25 with authorisation (strings as base 64), but not SSL.  What sort of processing power does SSL involve for the micro?

The other option I suggested to the client is a central (web) server that collects the data and then either emails it to the end user, or allows authenticated access to view it.

There will be multiple end users in the long term, so that's why they are asking for the simplest system possible to set up.

Each end user could have just one or many remote devices that they want to get reports from.  Email seemed the easiest to do that.

Either way a central collection / relay point is involved.  I'm open to other ideas on how to get the data to the end users.

[HwAoRrDk]

What you at least need are From:, To: and Subject:. It might be useful to add Date: and Message-Id. If you need to, set a Content-Type, but best is to send plain text. Nothing more is needed, and adding them won't make your email less likely to look like spam.

If the embedded system has no time- or date-keeping, then setting a Date header will of course be impossible; no big deal, as OP's mail host should add one itself when absent. Similarly, there's little point setting a Message-Id header, as (I presume) there will be no archiving/record-keeping of sent messages on the embedded system, so any generated ID will be meaningless. The only reason I can think of for setting one is if the ID is generated by a deterministic function (i.e. has no random element), so you can use it to later track the circumstances in which the message was composed and sent. Otherwise, again, the mail host will generate and add one if absent. I would call Content-Type a must, as this will allow you to declare the character set of the text within the message. Otherwise, if your message is anything other than pure ASCII (which is what it will be assumed to be without this header), mail clients may not render the text properly.

One extra suggestion for another header is Return-Path. If the mail account/address you're sending from is one that you may not wish to receive (or is not capable of receiving) any undeliverable bounces, you can set this header with an alternate address. For example, you may want to set this to your own address, so you know if the e-mails the embedded system is sending out aren't being delivered.

[cdev]

In addition to SPF and DKIM, there is also dmarc which tells (large) receiving servers how you want them to handle non-conforming email. You can get a daily XML-formatted report back as to how its been handled and why.

Its really an extremely different Internet than it was a few years ago when anybody could send email to anybody without any authentication that they were actually who they said they were happening at all.

A lot of email is silently deleted now. If you want your email to all get to its intended recipients, there are a lot of hoops they make you jump through.

------

I need to look into better ways to check incoming email for signs of spoofing.

 Recently Ive gotten some emails that I'm fairly sure did not come from people they claimed to be from, even though the headers appeared genuine. (outlook.com, gmail.com, aol.com senders) 

[David_AVD]

This is what I'm doing now.  Good point about the return path.

Code: [Select]

User-Agent: Embedded;
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 7bit
To: xxxx xxxx<xxx@xxxxxx.com>
From: xxx xxx <xxx@xxx.com>
Subject: Test Message


[David_AVD]

I tried adding in a Return-Path header, but it seems to be getting overridden by the mail server.

[Monkeh]

I tried adding in a Return-Path header, but it seems to be getting overridden by the mail server.

Overridden to what? It may just be SRS at work.

[David_AVD]

I tried adding in a Return-Path header, but it seems to be getting overridden by the mail server.

Overridden to what? It may just be SRS at work.

I put the Reply-Path just after the Content-Encoding, but in the received email it's at the top and set to the email address that was in the From field.

[Monkeh]

Ah, okay, misunderstanding.

Return-Path is set by the server. It is not sent to the contents of the 'From' field, but typically starts out as the argument of the 'MAIL FROM' command. They are not equivalent!

Return-Path is for handling things going wrong, for example bounces. If there are relays involved, it gets changed by the relay. From, however, is the email address of the person sending the mail - and Reply-To is where a reply should be sent, if not the sender.

If you want bounces and other errors to go to a different mailbox, use it as the argument to the 'MAIL FROM' command.

[C]

I have no idea how much security you need. When someone wants to break the system they try to find the easy spot to hack.

For E-mail, you have plain text at each e-mail server it goes through.
 
Also remember that with wireless I could receive the signals from a long distance for a long time.

Think of security this way, If you can tell someone how everything works and it's still next to imposable for wrong person to get in, It might be good security.

In long run you will probably want or need each device to be unique.

If you are going to use email then an email account for each device.

might want to look into a public key/private key setup

Might think of email to user as a notice to look at secure web site for info.