Author Topic: What actual data is used to return a favicon to the browser?  (Read 4461 times)

0 Members and 1 Guest are viewing this topic.

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Currently I have a simple HTTP server. All in C, no php etc. This works, serving a simple "system status" page. The page is generated with

Code: [Select]
static const unsigned char PAGE_START[] =
"HTTP/1.1 200 OK\r\n"
"Server: xxxxxx\r\n"
"Content-type: text/html\r\n\r\n"
"<!DOCTYPE html PUBLIC \"-//W3C//DTD HTML 4.01//EN\" \"http://www.w3.org/TR/html4/strict.dtd\">"
"<html>"
"<head>"
"  <title>xxxxxx Status</title>"
"  <meta http-equiv=Content-Type content=\"text/html; charset=utf-8\">"
"  <meta http-equiv=refresh content=1>"
"</head>"
"<body>"
"<pre><font size=\"+2\"><b>xxxxxx Status</font></b><br><br>";

and this is followed by the textual data of the page, and then this

"</p></body></html>\n"

How I am trying to return a favicon. This is because if I don't do this, the browser (Chrome) keeps requesting it all the time, so every time the page is hit, it gets two hits. If I returned the favicon, that should get cached.

The code I am using now is

<link rel=\"shortcut icon\" type=\"image/ico\" href=\"/favicon.ico\"/>

followed by the actual binary data of the favicon 16x16 file (generated with https://lvgl.io/tools/imageconverter)

Code: [Select]
static const unsigned char FAVICON[] = {
  0xfb, 0xfb, 0xfb, 0xff, /*Color of index 0*/
  0x6f, 0x6f, 0x6f, 0xff, /*Color of index 1*/

  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x2a, 0x80, 0xaa, 0xa0, 0x00,
  0x00, 0x7f, 0xc0, 0xff, 0xf0, 0x00,
  0x00, 0xff, 0xe1, 0xff, 0xf8, 0x00,
  0x00, 0x7f, 0xc0, 0xff, 0xf0, 0x00,
  0x00, 0x3f, 0x80, 0x7f, 0xa0, 0x00,
  0x00, 0x1f, 0x00, 0x3e, 0x00, 0x00,
  0x00, 0x1f, 0x00, 0xbc, 0x00, 0x00,
  0x00, 0x0f, 0x00, 0xf4, 0x00, 0x00,
  0x00, 0x1e, 0x01, 0xf0, 0x00, 0x00,
  0x00, 0x0f, 0x02, 0xe0, 0x00, 0x00,
  0x00, 0x1e, 0x07, 0xc0, 0x00, 0x00,
  0x00, 0x0f, 0x0f, 0x00, 0x00, 0x00,
  0x00, 0x1f, 0x2f, 0x80, 0x00, 0x00,
  0x00, 0x0f, 0x3c, 0x00, 0x00, 0x00,
  0x00, 0x1f, 0x7a, 0x00, 0x00, 0x00,
  0x00, 0x0f, 0xfd, 0x00, 0x00, 0x00,
  0x00, 0x1f, 0xff, 0x00, 0x00, 0x00,
  0x00, 0x0f, 0xcf, 0x80, 0x00, 0x00,
  0x00, 0x1f, 0xe7, 0x40, 0x00, 0x00,
  0x00, 0x0f, 0x83, 0xe0, 0x00, 0x00,
  0x00, 0x1f, 0x00, 0xf0, 0x00, 0x00,
  0x00, 0x0f, 0x01, 0xf4, 0x00, 0x00,
  0x00, 0x1e, 0x00, 0x3c, 0x00, 0x00,
  0x00, 0x0f, 0x00, 0x3e, 0x00, 0x00,
  0x00, 0x1e, 0x00, 0x1d, 0x00, 0x00,
  0x00, 0x0f, 0x00, 0x0f, 0x80, 0x00,
  0x00, 0x1f, 0x00, 0x07, 0xc0, 0x00,
  0x00, 0x3f, 0x80, 0x07, 0xe0, 0x00,
  0x00, 0x7f, 0xc0, 0x07, 0xf0, 0x00,
  0x00, 0xff, 0xe0, 0x03, 0xf8, 0x00,
  0x00, 0x7f, 0xc0, 0x03, 0xf0, 0x00,
  0x00, 0x2a, 0x80, 0x01, 0x50, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
};

But this isn't working. Chrome (debug console) says



I wondered if

- the image is supposed to be in base64
- a more full header is required (like the one for the working page)
- some data at the end is required to close the page

Any help much appreciated.

I did find that (stackexchange tip) adding

<link rel="icon" href="data:,">

to the top header suppresses the favicon request (works with chrome, edge and firefox) but Ideally I would still like to get the favicon to work.
« Last Edit: July 04, 2022, 08:33:16 pm by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: What actual data is used to return a favicon to the browser?
« Reply #1 on: July 04, 2022, 08:55:47 pm »
The most recent recommended way to specify the icon is this
Code: [Select]
<link rel="icon" type="image/x-icon" href="/favicon.ico"> But yours should be good enough, browsers are really tolerant here.

The file should be just raw data with no encoding or additional headers.

You can test it by entering "http://192.168.3.73:81/favicon.ico" in the address bar, your icon should be displayed on the page. If that does not happen, then favicon won't work no matter how you specify it,
Alex
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #2 on: July 04, 2022, 09:25:07 pm »
Is the data the above HTML, with the last character being > and that followed simply by the binary of the favicon, and then nothing more?

The client browser must in that case be counting bytes as they arrive so it knows the end of the data.

Also that means a CRLF instead of LF or nothing, will make a difference, because it will be taken as the start of the file.
« Last Edit: July 04, 2022, 09:51:11 pm by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: What actual data is used to return a favicon to the browser?
« Reply #3 on: July 04, 2022, 10:12:40 pm »
There is no HTML, you just send raw data of the icon.

To clarify, there is still HTTP response, which has content type and content length fields.
« Last Edit: July 04, 2022, 10:21:19 pm by ataradov »
Alex
 

Offline Whales

  • Super Contributor
  • ***
  • Posts: 1899
  • Country: au
    • Halestrom
Re: What actual data is used to return a favicon to the browser?
« Reply #4 on: July 04, 2022, 10:25:41 pm »
Are you remembering to send the HTTP header before the favicon?

Request and reply 1:
Quote
------ request from browser ------
GET /

------- reply from HTTP server -------
HTTP/1.1 200 OKmydude
Content-type: text/html

<html><marquee>Burgers</marquee></html>



Request and reply 2:
Quote
------ request from browser ------
GET /favicon.ico

------- reply from HTTP server -------
HTTP/1.1 200 OKmydude
Content-type: image/x-icon


*binary data of icon goes here*

Offline Whales

  • Super Contributor
  • ***
  • Posts: 1899
  • Country: au
    • Halestrom
Re: What actual data is used to return a favicon to the browser?
« Reply #5 on: July 04, 2022, 10:29:12 pm »
Also worth mentioning:

1. Use Wireshark to log the packets, that way you can see exactly what's happening.

2. Test in two different browser engines.  Firefox (in my experience) is more lenient about bad HTTP headers than Chromium.  Eg I think I recall Firefox accepting some forms of redirect without a proper HTTP header?  It's been a year or so since I last looked.  This caused a cgi app (cgi-fpm?) to fail only on Chrome and only when performing a certain action.

Offline T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 21686
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: What actual data is used to return a favicon to the browser?
« Reply #6 on: July 04, 2022, 11:05:09 pm »
It's any other file download; respect HTTP headers in whatever way that would be.

File format must be a compatible image.  .ico is Windows Bitmap format (among others, I think?!).  Like, mine is... let me see... oh heh this doesn't even open in my usual tools, what did I use, GIMP? Ah, there we go... oh weird, it was 16x19? Never noticed any weird rendering with that, but... wonder how I screwed that up so many years ago?!...

Oh right, it's, well, not precisely a BMP, anyway.  Here's the hex:

Code: [Select]
0000000 0000 0001 0001 1010 0000 0001 0008 0568
0000010 0000 0016 0000 0028 0000 0010 0000 0020
0000020 0000 0001 0008 0000 0000 0000 0000 0000
0000030 0000 0000 0000 0000 0000 0000 0000 0000
0000040 0000 0808 0008 0909 0009 0a0a 000a 0d0d
0000050 000d 1313 0013 1616 0016 1919 0019 1d1d
0000060 001d 2727 0027 2a2a 002a 3e3e 003e 4040
0000070 0040 4646 0046 6505 0004 4b4b 004b 5050
0000080 0050 5c5c 005c 6464 0064 9a02 0002 7272
0000090 0072 af00 0000 7f7f 007f 8d95 0085 b81e
00000a0 001e af3f 003f b82f 002a bb2a 002a bd32
00000b0 0032 c037 0037 ac73 006e ae78 006b 9d9e
00000c0 009d a5a5 00a4 a7a7 00a7 aeae 00ae cd66
00000d0 0066 b4b4 00b4 c898 0084 b7b7 00b7 b8b8
00000e0 00b8 b9b9 00b8 b9b9 00b9 caa8 0091 ccb3
00000f0 009a cdb4 009b c1c1 00c1 c2c2 00c2 d7a5
0000100 009a cbcb 00cb d3d1 00b6 cdcd 00cd dea5
0000110 00a2 e0a5 00a5 e1a3 00a3 e0ac 00a8 d3d3
0000120 00d3 d5e6 00c6 d4d4 00d4 d6e7 00c7 d6d6
0000130 00d6 d8e8 00ca d8e8 00cb e7b4 00b4 dce4
0000140 00d5 dcdc 00dc deec 00d3 dfec 00d4 e0ed
0000150 00d5 e0e0 00e0 e6e6 00e6 e7f1 00df eed2
0000160 00d2 f0d3 00d3 eaf3 00e3 ecf4 00e5 f1f7
0000170 00ec f3f7 00ef f4f8 00f0 f6fa 00f4 f8fb
0000180 00f6 f9fb 00f7 fcfd 00fb fdfb 00fb fdfd
0000190 00fc fdfe 00fd fefe 00fe ffff 00ff 0000
00001a0 0000 0000 0000 0000 0000 0000 0000 0000
*
0000430 0000 0000 0000 0000 0000 0000 0000 3b39
0000440 423d 4c47 0f51 0309 0001 3d17 393b 3e3b
0000450 4a44 514e 2e52 293c 0101 4440 3b3e 443d
0000460 4f4b 5652 5756 0533 0425 4b4f 3d44 4a42
0000470 214f 1010 3122 2f05 0a3a 4f52 424a 4e47
0000480 1652 0000 0312 5728 0d27 5256 474e 514c
0000490 1656 0000 2a02 5757 1949 5649 4c51 5250
00004a0 1656 0000 5711 4957 1524 5624 5052 0000
00004b0 0000 0000 5716 4857 1524 5724 5255 5552
00004c0 1657 0000 5716 4957 1524 5724 5255 5250
00004d0 1656 0000 5714 5757 1549 5318 5052 514c
00004e0 1656 0000 450b 5757 1d57 3615 4c51 4e47
00004f0 1652 0000 0841 5746 3f57 1b15 474d 4a42
0000500 204f 0c0c 2541 4107 5657 151c 4230 443d
0000510 4f4b 5552 5756 0623 5238 1537 321a 3e3b
0000520 4a43 514e 5652 1515 1513 1515 2d15 3b39
0000530 423d 4c47 5451 3435 0e1e 261f 2c2b 0000
0000540 0000 0000 0000 0000 0000 0000 0000 0000
*
0000570 0000 0000 0000 0000 0000 0000 0000
000057e

courtesy hexdump, which seems to be, the '*' are folding all-zeros between the nearest addresses.  So, lots of, I think unused palette entries, and some padding at the bottom for some reason.  Lemme see... there's 88 colors used, and that'll be a 256 color format, so yeah.  Heh, which doesn't save any space at all because it has to enumerate as many palette entries as there are pixels in the poor thing, and 24 or 32 bit bitmap would be better.

Anyway, it's lacking the "BM" magic code at the top, and I think the offsets are wrong too (changing the two bytes to "BM" doesn't fix it).  In any case, look up file formats, or just save it as whatever and dump that -- the whole file -- to C header.  Put that in the packet, with HTTP headers, and it should be fine.

Notably, your first attempt doesn't even contain dimensions or number of colors: just two color entries (but how could anyone know that?) then a bit array (I assume).  So there's obviously something wrong there, and it's probably just the proper .ico headers you need.

Tim
« Last Edit: July 04, 2022, 11:07:43 pm by T3sl4co1l »
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 

Online Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: What actual data is used to return a favicon to the browser?
« Reply #7 on: July 05, 2022, 12:14:39 am »
The simple solution is to respond to GET /favicon.ico with the icon data, supplying the length of the data in Content-Length header and the (synthetic or real) modification date in a Date header, and optionally a Cache-control: max-age=seconds header; and to a HEAD /favicon.ico with the same headers but no data.  That way, browsers will cache it, and only do a GET or HEAD request at seconds or longer intervals.

Another solution is to embed the favicon data in Base64 format in
    <link id="favicon" rel="shortcut icon" type="image/vnd.microsoft.icon" href="data:image/vnd.microsoft.icon;base64,data==">
element in the HTML head section.  However, since Base64 encodes three binary bytes in four characters, a 1406-byte favicon will increase each page load by 1963 bytes.

(Some, however, suggest using "image/x-icon" instead of "image/vnd.microsoft.icon".  Both seem to work in browsers equally well.)
« Last Edit: July 05, 2022, 12:19:27 am by Nominal Animal »
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #8 on: July 05, 2022, 07:03:46 am »
The reason I asked here rather than just googling it is because all the hits are ambiguous. They show stuff like

Content-type: text/html

without specifying whether the string is supposed to be followed by a CR, LF, or CRLF. If you get this wrong then obviously the "last char" will be taken as the 1st byte of the binary favicon image.

I now have the favicon being returned without errors


but it doesn't show anywhere in the browser, so it is probably in the wrong format, despite having been generated by a "favicon generator"

I am using

Code: [Select]

static const unsigned char FAVICON_START[] =
"HTTP/1.1 200 OK\r\n"
"Content-type: image/x-icon\r\n";


static const unsigned char FAVICON[] = {
  0xfb, 0xfb, 0xfb, 0xff, // Color of index 0
  0x6f, 0x6f, 0x6f, 0xff, // Color of index 1

  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x2a, 0x80, 0xaa, 0xa0, 0x00,
  0x00, 0x7f, 0xc0, 0xff, 0xf0, 0x00,
  0x00, 0xff, 0xe1, 0xff, 0xf8, 0x00,
  0x00, 0x7f, 0xc0, 0xff, 0xf0, 0x00,
  0x00, 0x3f, 0x80, 0x7f, 0xa0, 0x00,
  0x00, 0x1f, 0x00, 0x3e, 0x00, 0x00,
  0x00, 0x1f, 0x00, 0xbc, 0x00, 0x00,
  0x00, 0x0f, 0x00, 0xf4, 0x00, 0x00,
  0x00, 0x1e, 0x01, 0xf0, 0x00, 0x00,
  0x00, 0x0f, 0x02, 0xe0, 0x00, 0x00,
  0x00, 0x1e, 0x07, 0xc0, 0x00, 0x00,
  0x00, 0x0f, 0x0f, 0x00, 0x00, 0x00,
  0x00, 0x1f, 0x2f, 0x80, 0x00, 0x00,
  0x00, 0x0f, 0x3c, 0x00, 0x00, 0x00,
  0x00, 0x1f, 0x7a, 0x00, 0x00, 0x00,
  0x00, 0x0f, 0xfd, 0x00, 0x00, 0x00,
  0x00, 0x1f, 0xff, 0x00, 0x00, 0x00,
  0x00, 0x0f, 0xcf, 0x80, 0x00, 0x00,
  0x00, 0x1f, 0xe7, 0x40, 0x00, 0x00,
  0x00, 0x0f, 0x83, 0xe0, 0x00, 0x00,
  0x00, 0x1f, 0x00, 0xf0, 0x00, 0x00,
  0x00, 0x0f, 0x01, 0xf4, 0x00, 0x00,
  0x00, 0x1e, 0x00, 0x3c, 0x00, 0x00,
  0x00, 0x0f, 0x00, 0x3e, 0x00, 0x00,
  0x00, 0x1e, 0x00, 0x1d, 0x00, 0x00,
  0x00, 0x0f, 0x00, 0x0f, 0x80, 0x00,
  0x00, 0x1f, 0x00, 0x07, 0xc0, 0x00,
  0x00, 0x3f, 0x80, 0x07, 0xe0, 0x00,
  0x00, 0x7f, 0xc0, 0x07, 0xf0, 0x00,
  0x00, 0xff, 0xe0, 0x03, 0xf8, 0x00,
  0x00, 0x7f, 0xc0, 0x03, 0xf0, 0x00,
  0x00, 0x2a, 0x80, 0x01, 0x50, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
  0x00, 0x00, 0x00, 0x00, 0x00, 0x00,
};

and I have tried it with all 3 options for the end of the line before the file.

The file itself is indexed colour, 2 colours, for the smallest possible filesize. But maybe that doesn't work. What is the smallest favicon option?

Also it appears that Chrome does not cache the favicon as default behaviour so I went looking for caching directives and there are about 1000 of them online :) Firefox reportedly does cache it, and I can confirm that. But it also doesn't show it.
« Last Edit: July 05, 2022, 07:07:05 am by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: What actual data is used to return a favicon to the browser?
« Reply #9 on: July 05, 2022, 07:11:55 am »
without specifying whether the string is supposed to be followed by a CR, LF, or CRLF.

This is specified in the standard - https://www.rfc-editor.org/rfc/rfc2616#section-2.2

HTTP always uses CRLF as a line ending.

but it doesn't show anywhere in the browser, so it is probably in the wrong format, despite having been generated by a "favicon generator"
Can the browser open your link directly from the address bar?

The file itself is indexed colour, 2 colours, for the smallest possible filesize. But maybe that doesn't work. What is the smallest favicon option?
Minimal PNG file is about 70 bytes. Single pixel GIF is 42 bytes.

And it looks like people actually came up with 35 byte GIFs. This is something you can just inline in the HTML as a Base64 string like Nominal Animal suggested. It an overhead that is smaller in size than HTTP headers in the response for the favicon.
« Last Edit: July 05, 2022, 07:15:26 am by ataradov »
Alex
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #10 on: July 05, 2022, 07:54:40 am »
The link from the address bar doesn't work either.

Chrome shows a blank page.

FF says Image ... can't be displayed because it contains errors.

So I need to revisit the encoder(s) again.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: What actual data is used to return a favicon to the browser?
« Reply #11 on: July 05, 2022, 08:04:26 am »
Displaying it from the address bar is the best test. Once that works, regular request would work too.

Your debug screenshot shows 52 bytes.  This is clearly wrong. So you are either returning the wrong Content-Length or incomplete data.
« Last Edit: July 05, 2022, 08:23:21 am by ataradov »
Alex
 

Offline AndyBeez

  • Frequent Contributor
  • **
  • Posts: 856
  • Country: nu
Re: What actual data is used to return a favicon to the browser?
« Reply #12 on: July 05, 2022, 08:15:53 am »
A favicon can be any image file. You can use SVG, PNG, GIF and ICO formats on most browsers. Use the binary for a PNG as this is the most universally supported format.

RTFM : https://en.m.wikipedia.org/wiki/Favicon
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #13 on: July 05, 2022, 08:25:22 am »
I am not returning a content length. Most of the examples don't suggest this is required.

Googling suggests 16x16 and .ico format  in 8 bit colour is correct. However I have tried dozens of things now and nothing works. I am just after a simple letter "K"



(that one is 16x16)

Is there nothing after the binary block?

This is my last try - a PNG

Code: [Select]
static const unsigned char FAVICON_START[] =
"HTTP/1.1 200 OK\r\n"
"Content-type: image/x-icon\r\n";


static const unsigned char FAVICON[] = {
  0xff, 0xff, 0xff, 0xff, /*Color of index 0*/
  0x02, 0x02, 0x10, 0xff, /*Color of index 1*/

  0x00, 0x00,
  0x00, 0x00,
  0x00, 0x00,
  0x1c, 0xf0,
  0x08, 0x40,
  0x08, 0x80,
  0x09, 0x00,
  0x0a, 0x00,
  0x0d, 0x00,
  0x08, 0x80,
  0x08, 0x40,
  0x08, 0x20,
  0x1c, 0x30,
  0x00, 0x00,
  0x00, 0x00,
  0x00, 0x00,
};
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: What actual data is used to return a favicon to the browser?
« Reply #14 on: July 05, 2022, 08:30:42 am »
Content-Length is required for all responses containing Entity-Body. This is a TCP connection there is no way to determine the length of arbitrary binary data.

Also "Content-Type" must be spelled like this. You lave a lower case "t".  EDIT: Although it looks like HTTP is not case sensitive, so that's not an issue.
« Last Edit: July 05, 2022, 08:32:44 am by ataradov »
Alex
 

Offline AndyBeez

  • Frequent Contributor
  • **
  • Posts: 856
  • Country: nu
Re: What actual data is used to return a favicon to the browser?
« Reply #15 on: July 05, 2022, 08:36:40 am »
Your web server should list .ico types in the hosting config, as this is not strickly a default file type. Also the favicon needs to be in the correct directory for the '/' path.

 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: What actual data is used to return a favicon to the browser?
« Reply #16 on: July 05, 2022, 08:41:31 am »
Based on the question and constraints, this is served by a custom server, possibly on an embedded system.
Alex
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #17 on: July 05, 2022, 08:43:44 am »
This is an embedded system, not Apache or similar server from where the client can fetch files from a dir path. I am looking for

Code: [Select]
if(strncmp(buf, "GET /favicon.ico", 16) == 0)
and then returning the favicon. Same code for returning the main page (name.html) and that works. Debugging confirms this stuff is working right; at least I am returning the right sizes.

Is the content length just the binary data?

Just realised that my header is probably wrong for a .png file format. Previously I was using a .ico file format.
« Last Edit: July 05, 2022, 08:46:48 am by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: What actual data is used to return a favicon to the browser?
« Reply #18 on: July 05, 2022, 08:52:13 am »
https://datatracker.ietf.org/doc/html/rfc2616#section-14.13

It is mandatory for most responses. The only time it is not necessary for responses containing the data is if a chunked encoding is used. This is useful for streaming data, but then you have to specify chunk lengths.

Conent-Length covers all octets after the response header (ending in CRLF).

Your template does not have the final CRLF. Do you actually send it? Response header ends in an empty line.
« Last Edit: July 05, 2022, 08:55:37 am by ataradov »
Alex
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #19 on: July 05, 2022, 09:04:10 am »
This really simple thing is amazingly complicated.

Code: [Select]
static const unsigned char FAVICON_HEADER[] =
"HTTP/1.1 200 OK\r\n"
"Content-Type: image/x-png\r\n"
"Content-length: 40\r\n";


static const unsigned char FAVICON_BODY[] = {
  0xff, 0xff, 0xff, 0xff, /*Color of index 0*/
  0x02, 0x02, 0x10, 0xff, /*Color of index 1*/

  0x00, 0x00,
  0x00, 0x00,
  0x00, 0x00,
  0x1c, 0xf0,
  0x08, 0x40,
  0x08, 0x80,
  0x09, 0x00,
  0x0a, 0x00,
  0x0d, 0x00,
  0x08, 0x80,
  0x08, 0x40,
  0x08, 0x20,
  0x1c, 0x30,
  0x00, 0x00,
  0x00, 0x00,
  0x00, 0x00,
};

Still no luck. Chrome has changed the frequency at which it fetches the favicon. It now looks like it has seen so many errors it has largely given up fetching it, so I am testing with FF also.

A direct file fetch produces this



where the length value doesn't make much sense. I am using the same code as for the main page

Code: [Select]
  netconn_write(conn, FAVICON_HEADER, strlen((char*)FAVICON_HEADER), NETCONN_COPY);
  netconn_write(conn, FAVICON_BODY, sizeof(FAVICON_BODY), NETCONN_COPY);  // strlen doesn't work with binary data

The sizeof is 40 which is right.

However, using

Code: [Select]
  netconn_write(conn, FAVICON_HEADER, strlen((char*)FAVICON_HEADER), NETCONN_COPY);
  netconn_write(conn, FAVICON_BODY, sizeof((char*)FAVICON_BODY), NETCONN_COPY);

returns 68 bytes. I thought the sizeof() operator would work simply on the array, which is 40 long, so (char *) should not affect the value.

I now wonder if favicon.ico is right when the file is actually in png format?

This looks like another 1-2 day rabbit hole :)
« Last Edit: July 05, 2022, 09:20:50 am by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline AndyBeez

  • Frequent Contributor
  • **
  • Posts: 856
  • Country: nu
Re: What actual data is used to return a favicon to the browser?
« Reply #20 on: July 05, 2022, 09:19:42 am »
Does your Chrome or Firefox have a debug mode? Can you view the raw http responses?
 

Offline sokoloff

  • Super Contributor
  • ***
  • Posts: 1799
  • Country: us
Re: What actual data is used to return a favicon to the browser?
« Reply #21 on: July 05, 2022, 09:30:55 am »
I thought the sizeof() operator would work simply on the array, which is 40 long, so (char *) should not affect the value.
sizeof of a char * will be the size of a pointer on your platform, not the size of the thing that pointer is pointing to (which is not known after the cast to a char* )
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #22 on: July 05, 2022, 10:07:31 am »
Quote
Does your Chrome or Firefox have a debug mode? Can you view the raw http responses?

Chrome seems to but it displays a blank. Same with Edge.

There is something very basic going wrong here.

Nothing useful online because all google hits are based around normal web servers where you just create the favicon file and drop it somewhere.
« Last Edit: July 05, 2022, 10:11:54 am by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline T3sl4co1l

  • Super Contributor
  • ***
  • Posts: 21686
  • Country: us
  • Expert, Analog Electronics, PCB Layout, EMC
    • Seven Transistor Labs
Re: What actual data is used to return a favicon to the browser?
« Reply #23 on: July 05, 2022, 10:36:26 am »
What tools are you using, for image editing, conversion and "favicon generator"?

You linked a 46x44 image which is a capital 'K' on white background, 16x16, on black background 46x44.

FYI, the PNG format, cropped to 16x16, saved as 2-color format in PSP7 (and again as hexdump shows it) is:

Code: [Select]
0000000 5089 474e 0a0d 0a1a 0000 0d00 4849 5244
0000010 0000 1000 0000 1000 0301 0000 2500 6d3d
0000020 0022 0000 742b 5845 4374 6572 7461 6f69
0000030 206e 6954 656d 5400 6575 3520 4a20 6c75
0000040 3220 3230 2032 3530 323a 3a32 3835 2d20
0000050 3630 3030 c8d8 b19c 0000 0700 4974 454d
0000060 e607 0507 180a 8b1f 6eb3 0067 0000 7009
0000070 5948 0073 0b00 0012 0b00 0112 ddd2 fc7e
0000080 0000 0400 4167 414d 0000 8fb1 fc0b 0561
0000090 0000 0600 4c50 4554 0000 ff00 ffff d9a5
00000a0 dd9f 0000 2b00 4449 5441 da78 f863 9fff
00000b0 8201 f31e 7c33 cfdf bdf0 e19e 7fdb af86
00000c0 19ff fd3e b107 2241 19f7 9f1e 8067 0329
00000d0 a500 1c5e 53a1 edb2 0055 0000 4900 4e45
00000e0 ae44 6042 0082
00000e5

There's no weird formatting of colors or arrays of bits, there's, well there is a "PNG" near the top, and a few other bits of junk in there actually by the look of it, you can shave those out I'm sure with a compactor tool -- let me see here.  Ah, https://tinypng.com/ seems to do a good enough job of it.  And it still opens in PSP7 which is good enough for me:

Code: [Select]
0000000 5089 474e 0a0d 0a1a 0000 0d00 4849 5244
0000010 0000 1000 0000 1000 0301 0000 2500 6d3d
0000020 0022 0000 5006 544c ff45 ffff 0000 5500
0000030 d3c2 007e 0000 4925 4144 0854 63d7 0380
0000040 0f99 1c0c 0c0e 0d1c 9c0c 0c0c 0c5c bc0c
0000050 400c 5836 8144 c641 ae00 000a 2058 4103
0000060 b764 72b3 0000 0000 4549 444e 42ae 8260
0000070

Should be, paste that in there with the right length (0x70 = 112 bytes) and go.  Well, edit it for commas and 0x's obviously...

Yeah I don't have a clue where you're getting your raw image data from but it sure as hell ain't any supported image format

Tim
Seven Transistor Labs, LLC
Electronic design, from concept to prototype.
Bringing a project to life?  Send me a message!
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #24 on: July 05, 2022, 10:43:23 am »
I've tried various online tools, with .bmp .png and last .ico files, converting to "C array".

The source files are ex photoshop, 16x16 pixel. Yes the last one I posted was bigger but that was just so you could see it in the post. The actual image is 16x16 pixels.

There is a lot of conflicting info on acceptable formats.

I am now using this 256-byte one, converted with https://lvgl.io/tools/imageconverter

Code: [Select]
static const unsigned char FAVICON_HEADER[] =
"HTTP/1.1 200 OK\r\n"
"Content-Type: image/x-icon\r\n"
"Content-Length: 256\r\n";


static const unsigned char FAVICON_BODY[] = {
  0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0x20, 0x20, 0x20, 0xff, 0xff, 0x20, 0x20, 0x20, 0x20, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0xff, 0x20, 0xff, 0xff, 0xff, 0xff, 0x20, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0xff, 0x20, 0xff, 0xff, 0xff, 0x20, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0xff, 0x20, 0xff, 0xff, 0x20, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0xff, 0x20, 0xff, 0x20, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0xff, 0x20, 0x20, 0xff, 0x20, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0xff, 0x20, 0xff, 0xff, 0xff, 0x20, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0xff, 0x20, 0xff, 0xff, 0xff, 0xff, 0x20, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0xff, 0x20, 0xff, 0xff, 0xff, 0xff, 0xff, 0x20, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0x20, 0x20, 0x20, 0xff, 0xff, 0xff, 0xff, 0x20, 0x20, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
  0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff, 0xff,
};

which in Edge produces this in the debug console



and the header and the file are 66 and 256 bytes which comes to 322 so that's ok. I've checked the length value passed here

Code: [Select]
  netconn_write(conn, FAVICON_HEADER, strlen((char*)FAVICON_HEADER), NETCONN_COPY);
  netconn_write(conn, FAVICON_BODY, sizeof(FAVICON_BODY), NETCONN_COPY);

Chrome and Edge show nothing (if going to the direct URL) while FF says the image contains errors. I can't see any way to display the actual binary data coming back to the browser, and I don't have an ethernet protocol analyser.

I did try your file, with

Code: [Select]
0x5089, 0x474e, 0x0a0d, 0x0a1a, 0x0000, 0x0d00, 0x4849, 0x5244,
0x0000, 0x1000, 0x0000, 0x1000, 0x0301, 0x0000, 0x2500, 0x6d3d,
0x0022, 0x0000, 0x5006, 0x544c, 0xff45, 0xffff, 0x0000, 0x5500,
0xd3c2, 0x007e, 0x0000, 0x4925, 0x4144, 0x0854, 0x63d7, 0x0380,
0x0f99, 0x1c0c, 0x0c0e, 0x0d1c, 0x9c0c, 0x0c0c, 0x0c5c, 0xbc0c,
0x400c, 0x5836, 0x8144, 0xc641, 0xae00, 0x000a, 0x2058, 0x4103,
0xb764, 0x72b3, 0x0000, 0x0000, 0x4549, 0x444e, 0x42ae, 0x8260
};

but it completely breaks it. I declared the array as uint16_t because I can see e.g. 0x0a0d which is LFCR i.e. backwards but the 32F4 is little-endian so it should work.

If someone would be kind enough and generate an icon file C array which just displays the letter K (in Courier, fwiw) I will give it a go but otherwise I will have to abandon this, after 2 days. It is very easy on a normal web server but clearly there is some special trick involved when one is dealing with "actual bytes".

At the same time I am returning a printable-data web page without any trouble, so maybe base64 encoding would be better. OTOH I don't know if my icon file is even right...

I've been looking for somebody, as a paid consultant, who can help with this kind of thing but haven't found anyone. There is a Monday-only guy working on this project (he did the original Cube setup and later worked on the ethernet end of it) and he knows all this, but today is Tuesday :) :) So I need to solve these things on my own. I can do most things in embedded but by choice never went up the learning curve on ETH, USB, TCP etc, PHP, server-side coding and config. I do have a good guy for the server stuff and he is doing some embedded C now (setting up an embedded HTTP server on this product) but I think this favicon issue is something really simple.
« Last Edit: July 05, 2022, 08:36:06 pm by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline sokoloff

  • Super Contributor
  • ***
  • Posts: 1799
  • Country: us
Re: What actual data is used to return a favicon to the browser?
« Reply #25 on: July 05, 2022, 11:28:05 am »
Grab the favicon as returned by the STM with curl (or wget).

Here's how to grab google's; you can pattern match to adjust it for yours.

Code: [Select]
curl --output google-favicon.ico [url=http://www.google.com/favicon.ico]www.google.com/favicon.ico[/url]
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #26 on: July 05, 2022, 02:00:49 pm »
Is there a straightforward win7-64 Curl install package? I found the .se one which is some weird thing you have to compile yourself.

I also have cygwin (use it for rsync etc) but surprisingly it isn't in there. Then I found a cygwin curl and it "could not be downloaded securely"...

I found a straight wget.exe here
https://eternallybored.org/misc/wget/

It doesn't seem to play - after a ~2 sec timeout I get this



and a zero length file saved. OTOH doing it for the main web page works perfectly; the file contains the right html and displays in a browser.

To me it looks like wget is looking for more data, which never comes.

More tests with different icon files all show the right length reported by wget but the resulting file is zero size.

What looks like binary data is coming back



but I can't capture it with a > redirection to a file; presumably wget is using stderr to output to the console.

If I use -o filename option to capture the messages, I see

--2022-07-05 15:27:54--  http://192.168.1.61:8080/favicon.ico
Connecting to 192.168.1.61:8080... connected.
HTTP request sent, awaiting response... Read error (Unknown error) in headers.
Retrying.


« Last Edit: July 05, 2022, 02:31:07 pm by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: What actual data is used to return a favicon to the browser?
« Reply #27 on: July 05, 2022, 02:38:12 pm »
Do your headers end with "\r\n\r\n", i.e. empty line following the last header?  It doesn't look like they do, unless you insert an extra "\r\n" somehow.  If so, the client cannot parse the HTTP response correctly.  The download size also indicates the headers are considered part of the data payload.

You need to provide a Date: Dayname, dayNum Month Year HH:MM:SS GMT header for clients to be able to cache the data.  They use the date and Cache-Control: max-age=seconds to determine if their copy is still valid.  Outside that time range, they usually send a HEAD request to obtain the headers only, to see if the file has changed (assuming they still have a copy).

If you use a fixed date and max-age, make sure it does not exceed the 32-bit Unix time, to avoid 32-bit time bugs in browsers.  That is, choose max-age (relative to the fixed Date:) to elapse in 2037.

Edit: In other words, I'd expect you to have something like
Code: [Select]
static const unsigned char FAVICON_START[] =
    "HTTP/1.1 200 OK\r\n"
    "Content-type: image/x-icon\r\n"
    "Content-length: 256\r\n"
    "Date: Sat, 01 Jan 2022 00:00:00 GMT\r\n"
    "Cache-Control: max-age=473384600\r\n"
    "\r\n";
instead.  (But, if you ever change the icon, expect browsers to show the old version until you clear the browser cache.)
« Last Edit: July 05, 2022, 02:45:09 pm by Nominal Animal »
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: What actual data is used to return a favicon to the browser?
« Reply #28 on: July 05, 2022, 02:50:23 pm »
Do your headers end with "\r\n\r\n", i.e. empty line following the last header?  It doesn't look like they do, unless you insert an extra "\r\n" somehow.  If so, the client cannot parse the HTTP response correctly.  The download size also indicates the headers are considered part of the data payload.
This. I already pointed it out and was ignored.

There is a trivial way to debug this. Take any site and its favicon link. Write a simple client in C/Pyhon/whatever that requests that URL and received the response. Save the response byte to byte. Replay that exact response from your code. Just send the whole byte array. After that works, feel free to edit it and check what fields are important and what fields are not.
Alex
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #29 on: July 05, 2022, 02:52:49 pm »
I am making progress. I see wget receive two 0x20 bytes



which do not feature in the binary data



and sure enough wget now looks a lot better

--2022-07-05 15:44:35--  http://192.168.1.61:8080/favicon.ico
Connecting to 192.168.1.61:8080... connected.
HTTP request sent, awaiting response...
  HTTP/1.1 200 OK
  Content-Type: image/x-icon
  Content-Length: 256
Length: 256 [image/x-icon]
Saving to: 'favicon.ico.14'

     0K                                                       100% 2.51K=0.1s

2022-07-05 15:44:35 (2.51 KB/s) - 'favicon.ico.14' saved [256/256]


Nominal Animal was spot on!

And now putting http://192.168.1.61:8080/favicon.ico in a browser produces a little square in the browser :)

There is no "K" in the square though...

The filed saved by wget looks same as the one I am sending out. But that one doesn't display correctly in a graphics viewer - I chose the smallest of a series of options. Clearly that online image converter produces garbage.

But now I am a lot nearer.
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline AndyBeez

  • Frequent Contributor
  • **
  • Posts: 856
  • Country: nu
Re: What actual data is used to return a favicon to the browser?
« Reply #30 on: July 05, 2022, 02:53:39 pm »
Guys I was thinking the same but assumed that was already done to spec. The random binary character needs to be on the next line down! See the content-type string on the html version.

@peter-h When developing in any http protocol, a header request-response analyser is a must have in your toolkit. Wireshark is the goto for heavy lift analysis, but even browsers have a developer mode console that reveals network traffic and http headers. Find out, it might just save you a day of hair pulling.
« Last Edit: July 05, 2022, 03:03:15 pm by AndyBeez »
 

Online Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: What actual data is used to return a favicon to the browser?
« Reply #31 on: July 05, 2022, 02:57:23 pm »
Do your headers end with "\r\n\r\n", i.e. empty line following the last header?  It doesn't look like they do, unless you insert an extra "\r\n" somehow.  If so, the client cannot parse the HTTP response correctly.  The download size also indicates the headers are considered part of the data payload.
This. I already pointed it out and was ignored.

Yes, you did:
Your template does not have the final CRLF. Do you actually send it? Response header ends in an empty line.
And even I missed it, somehow  :-[.  I apologize, ataradov!  It was definitely not intentional.

The filed saved by wget looks same as the one I am sending out. But that one doesn't display correctly in a graphics viewer
Open it in your browser.  If it doesn't show up, it is b0rked.
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #32 on: July 05, 2022, 03:03:00 pm »
It does show up but just a white square. Obviously it is a garbage graphic. But also it is 256 bytes which is way too much. I need to find a way to produce a minimum size file, but right now I can't even get any file to work (even though they do now download correctly to the client).

I am trying various options here https://notisrac.github.io/FileToCArray/



but none seem to work. The captured .ico file is corrupt, in any program I use to view it. Is this another junk site? Or is there another little secret?



By selecting the Treat as Binary, I have it working (!!) but the data is 1150 bytes.



So the Q is... how can one reduce that file such that it remains compatible with browsers, never mind any image viewing program?

The desired file is attached as a zip and maybe somebody may know the answer :)
« Last Edit: July 05, 2022, 03:37:53 pm by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline sokoloff

  • Super Contributor
  • ***
  • Posts: 1799
  • Country: us
Re: What actual data is used to return a favicon to the browser?
« Reply #33 on: July 05, 2022, 03:48:05 pm »
Most browsers (except Opera mini, whatever that is) support using PNG for favicon:
https://caniuse.com/?search=png%20favicon

If that's the case, I'd use something like a 32x32 PNG if every byte counts.
1530265-0
« Last Edit: July 05, 2022, 03:50:45 pm by sokoloff »
 

Offline sokoloff

  • Super Contributor
  • ***
  • Posts: 1799
  • Country: us
Re: What actual data is used to return a favicon to the browser?
« Reply #34 on: July 05, 2022, 03:49:49 pm »
Or your specific version of Courier's K shrinks even smaller than the previous:
1530274-0
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #35 on: July 05, 2022, 03:59:24 pm »
How did you generate these?

There are many options online for producing icons, and many more in say photoshop, but most of them produce files which are invalid in most "simple" image viewers. And the specs for favicons are yet different (have tried loads).

If I go to PNG I have to go back to what are the right headers.

Opera is pretty dead, but not completely dead. Not sure where it is actually used now. Last definite use was Nokia Symbian phones, which died out 5+ years ago. These are the google analytics stats from a pan-European "technology-type" (not electronics) forum which I am involved in as admin

« Last Edit: July 05, 2022, 04:11:26 pm by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: What actual data is used to return a favicon to the browser?
« Reply #36 on: July 05, 2022, 04:22:05 pm »
I claim that
Code: [Select]
#define  STRINGIFY_(x)  #x
#define  STRINGIFY(x)   STRINGIFY_(x)

#define  FAVICON_BODY_LEN  123
static const unsigned char FAVICON_HEADER[] =
    "HTTP/1.1 200 OK\r\n"
    "Content-Type: image/x-icon\r\n"
    "Content-Length: " STRINGIFY(FAVICON_BODY_LEN) "\r\n"
    "Content-Encoding: gzip\r\n"
    "Date: Sat, 01 Jan 2022 00:00:00 GMT\r\n"
    "Cache-Control: max-age=473384600\r\n"
    "\r\n";
#define  FAVICON_HEADER_LEN  ((sizeof FAVICON_HEADER) - 1)

static const unsigned char FAVICON_BODY[FAVICON_BODY_LEN] = {
    0x1f, 0x8b, 0x08, 0x08, 0xea, 0xf3, 0xc3, 0x62, 0x02, 0x03, 0x66, 0x61, 0x76, 0x69, 0x63, 0x6f,
    0x6e, 0x2e, 0x69, 0x63, 0x6f, 0x00, 0x63, 0x60, 0x60, 0x04, 0x42, 0x01, 0x01, 0x06, 0x20, 0xa9,
    0xc0, 0x90, 0xc1, 0xc2, 0xc0, 0x20, 0xc6, 0xc0, 0xc0, 0xa0, 0x01, 0xc4, 0x40, 0x21, 0xa0, 0x08,
    0x44, 0x1c, 0x0c, 0x80, 0x72, 0xde, 0x31, 0x10, 0x0c, 0x03, 0xff, 0x87, 0x11, 0x60, 0x62, 0x12,
    0x80, 0x63, 0x5c, 0x72, 0xc4, 0x98, 0x41, 0x8a, 0x38, 0x31, 0xea, 0x88, 0xd5, 0x8b, 0x4d, 0x2d,
    0x29, 0x7a, 0xd1, 0xfd, 0x49, 0xaa, 0x5e, 0x64, 0x3d, 0xe4, 0xe8, 0x45, 0xd7, 0x47, 0x89, 0xfd,
    0x94, 0xf8, 0x9f, 0x92, 0xb8, 0x43, 0x4f, 0x27, 0xc8, 0x62, 0xe4, 0x86, 0xc9, 0x60, 0x06, 0x0c,
    0x14, 0x02, 0x00, 0xc2, 0xd1, 0x9b, 0xb4, 0x7e, 0x04, 0x00, 0x00,
};

should work, because the following program in Linux also works:
Code: [Select]
#define  _POSIX_C_SOURCE  200809L
#define  _GNU_SOURCE
#include <stdlib.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <signal.h>
#include <netdb.h>
#include <string.h>
#include <stdio.h>
#include <errno.h>

#define  STRINGIFY_(x)  #x
#define  STRINGIFY(x)   STRINGIFY_(x)

#define  FAVICON_BODY_LEN  123
static const unsigned char FAVICON_HEADER[] =
    "HTTP/1.1 200 OK\r\n"
    "Content-Type: image/x-icon\r\n"
    "Content-Length: " STRINGIFY(FAVICON_BODY_LEN) "\r\n"
    "Content-Encoding: gzip\r\n"
    "Date: Sat, 01 Jan 2022 00:00:00 GMT\r\n"
    "Cache-Control: max-age=473384600\r\n"
    "\r\n";
#define  FAVICON_HEADER_LEN  ((sizeof FAVICON_HEADER) - 1)

static const unsigned char FAVICON_BODY[FAVICON_BODY_LEN] = {
    0x1f, 0x8b, 0x08, 0x08, 0xea, 0xf3, 0xc3, 0x62, 0x02, 0x03, 0x66, 0x61, 0x76, 0x69, 0x63, 0x6f,
    0x6e, 0x2e, 0x69, 0x63, 0x6f, 0x00, 0x63, 0x60, 0x60, 0x04, 0x42, 0x01, 0x01, 0x06, 0x20, 0xa9,
    0xc0, 0x90, 0xc1, 0xc2, 0xc0, 0x20, 0xc6, 0xc0, 0xc0, 0xa0, 0x01, 0xc4, 0x40, 0x21, 0xa0, 0x08,
    0x44, 0x1c, 0x0c, 0x80, 0x72, 0xde, 0x31, 0x10, 0x0c, 0x03, 0xff, 0x87, 0x11, 0x60, 0x62, 0x12,
    0x80, 0x63, 0x5c, 0x72, 0xc4, 0x98, 0x41, 0x8a, 0x38, 0x31, 0xea, 0x88, 0xd5, 0x8b, 0x4d, 0x2d,
    0x29, 0x7a, 0xd1, 0xfd, 0x49, 0xaa, 0x5e, 0x64, 0x3d, 0xe4, 0xe8, 0x45, 0xd7, 0x47, 0x89, 0xfd,
    0x94, 0xf8, 0x9f, 0x92, 0xb8, 0x43, 0x4f, 0x27, 0xc8, 0x62, 0xe4, 0x86, 0xc9, 0x60, 0x06, 0x0c,
    0x14, 0x02, 0x00, 0xc2, 0xd1, 0x9b, 0xb4, 0x7e, 0x04, 0x00, 0x00,
};

static volatile sig_atomic_t  done = 0;

static void handle_done(int signum)
{
    (void)signum;  /* Generates no code, just silences unused parameter warning. */
    done = 1;
}

static int install_done(int signum)
{
    struct sigaction act;

    memset(&act, 0, sizeof act);
    sigemptyset(&(act.sa_mask));
    act.sa_handler = handle_done;
    act.sa_flags = 0; /* We want interrupt delivery to block a waiting accept(). */
    if (sigaction(signum, &act, NULL) == -1)
        return errno;

    return 0;
}

static int write_all(int fd, const void *buf, size_t len)
{
    const char        *ptr = buf;
    const char *const  end = len + (const char *)buf;
    ssize_t            n;

    while (ptr < end) {
        n = write(fd, ptr, (size_t)(end - ptr));
        if (n > 0) {
            ptr += n;
        } else
        if (n == -1) {
            return errno;
        } else {
            return errno = EIO;
        }
    }

    return 0;
}

static int serve(int fd)
{
    size_t  in_max = 4096;
    size_t  in_len = 0;
    char   *in_buf = malloc(in_max);
    ssize_t n;
    int     err;

    if (!in_buf)
        return ENOMEM;
    else
        in_buf[0] = '\0';

    /* Read until \r\n\r\n. */
    while (in_len < 4 || !memmem(in_buf, in_len, "\r\n\r\n", 4)) {

        if (in_len + 1 >= in_max) {
            size_t  new_max = (in_max | 4095) + 4097 - 32;  /* Linear growth */
            char   *new_buf = realloc(in_buf, new_max);
            if (!new_buf) {
                free(in_buf);
                return ENOMEM;
            }
            in_max = new_max;
            in_buf = new_buf;
        }

        n = read(fd, in_buf + in_len, in_max - 1 - in_len);
        if (n > 0) {
            in_len += n;
            in_buf[in_len] = '\0';
        } else
        if (!n) {
            free(in_buf);
            return ECONNABORTED;
        } else
        if (n != -1) {
            free(in_buf);
            return EIO;
        } else {
            const int saved_errno = errno;
            free(in_buf);
            return saved_errno;
        }
    }

    if (!strncmp(in_buf, "GET ", 4)) {
        printf("Received a GET request\n");
    } else
    if (!strncmp(in_buf, "HEAD ", 5)) {
        printf("Received a HEAD request\n");
    } else {
        printf("Received an invalid request\n");
        free(in_buf);
        return ECONNREFUSED;
    }

    err = write_all(fd, FAVICON_HEADER, FAVICON_HEADER_LEN);
    if (err) {
        free(in_buf);
        return err;
    }

    if (!strncmp(in_buf, "HEAD ", 5)) {
        free(in_buf);
        return 0;
    }

    err = write_all(fd, FAVICON_BODY, FAVICON_BODY_LEN);
    if (err) {
        free(in_buf);
        return err;
    }

    free(in_buf);
    return 0;
}

int main(int argc, char *argv[])
{
    struct addrinfo hints, *result, *ai;
    const char *node, *serv;
    int sockfd, connfd, err;

    if (argc != 3 || !strcmp(argv[1], "-h") || !strcmp(argv[1], "--help")) {
        const char *argv0 = (argc > 0 && argv && argv[0] && argv[0][0]) ? argv[0] : "(this)";
        fprintf(stderr, "\n");
        fprintf(stderr, "Usage: %s [ -h | --help ]\n", argv0);
        fprintf(stderr, "       %s ADDRESS PORT-OR-SERVICE\n", argv0);
        fprintf(stderr, "\n");
        return EXIT_FAILURE;
    }
    node = argv[1];
    if (!*node || !strcmp(node, "*") || !strcmp(node, "-"))
        node = NULL;
    serv = argv[2];

    if (install_done(SIGHUP) ||
        install_done(SIGINT) ||
        install_done(SIGTERM) ||
        install_done(SIGQUIT)) {
        fprintf(stderr, "Cannot install signal handlers: %s.\n", strerror(errno));
        return EXIT_FAILURE;
    }

    memset(&hints, 0, sizeof hints);
    hints.ai_family = AF_UNSPEC;     /* IPv4 or IPv6 */
    hints.ai_socktype = SOCK_STREAM; /* TCP */
    hints.ai_flags = AI_PASSIVE;
    hints.ai_protocol = 0;
    hints.ai_canonname = NULL;
    hints.ai_addr = NULL;
    hints.ai_next = NULL;
    err = getaddrinfo(node, serv, &hints, &result);
    if (err) {
        fprintf(stderr, "%s.\n", gai_strerror(err));
        return EXIT_FAILURE;
    }

    for (sockfd = -1, ai = result; ai != NULL; ai = ai->ai_next) {
        sockfd = socket(ai->ai_family, ai->ai_socktype, ai->ai_protocol);
        if (sockfd == -1)
            continue;

        if (bind(sockfd, ai->ai_addr, ai->ai_addrlen) == -1) {
            close(sockfd);
            sockfd = -1;
            continue;
        }

        if (listen(sockfd, 1)) {
            close(sockfd);
            sockfd = -1;
            continue;
        }

        break;
    }
    if (sockfd == -1) {
        fprintf(stderr, "Cannot bind to specified address and port.\n");
        return EXIT_FAILURE;
    }

    while (!done) {
        connfd = accept(sockfd, NULL, NULL);
        if (connfd == -1) {
            if (errno == EINTR)
                continue;
            fprintf(stderr, "Connection failure: %s.\n", strerror(errno));
            continue;
        }

        err = serve(connfd);
        if (err) {
            fprintf(stderr, "Service failure: %s.\n", strerror(errno));
            close(connfd);
            continue;
        }

        fprintf(stderr, "Served successfully.\n");
        close(connfd);
    }

    close(sockfd);
    return EXIT_SUCCESS;
}

The data is gzipped favicon.ico (original 1150 bytes, sha256sum 2c01904c0df69815f18b28d2a7e8e438c353b5352b42fb3699726b332b507be9; gzipped to 123 bytes with sha256sum 1b566fe5012155da352dade788b9c4eac90f98a6facdab8c1610f7674b869db0).  Browsers do support gzip transfer encoding (but we really should check that the request contained an "Accept-Encoding:" header with "gzip" as one of the comma-separated items).
« Last Edit: July 05, 2022, 04:24:40 pm by Nominal Animal »
 

Offline mariush

  • Super Contributor
  • ***
  • Posts: 5029
  • Country: ro
  • .
Re: What actual data is used to return a favicon to the browser?
« Reply #37 on: July 05, 2022, 04:29:23 pm »
Most browsers support  ICO GIF and PNG formats for favicons. Oh.. SVG is also supported by most browsers

You can also put it inline in the html page but encoded in base64 or other encoding supported by browsers  example ;

<link rel="icon" href="data:image/png;base64,[BASE 64 ENCODED BINARY DATA]">

but some browsers will request /favicon.ico either way.

You can probably reduce the amount of requests by using Cache-Control tags response tag ex:

Cache-Control: max-age=84600, public

will tell the browser it can cache the picture and reuse it for up to 84600s which is close to 24 hours.

Yeah... pretty much anything supports gzip , deflate, brotli (good for html/js, not so much for images) these days but if you send a png image it's already compressed so no need to add complexity.

Do 32x32 or 48x48 or 64x64  but use color palette and use some dithering to reduce the colors in your icon, absolutely no need to use 24 bit RGB on a favicon. Most people won't even care or notice it.


EDIT:  .. and your favicon.png can be shrunk to 120 bytes see attachment.

You saved the original icon with Photoshop and it embedded a ICC profile which is really not needed for a 2 color (bw) picture. Could probably get your photoshop do not embed it by using some save for web / optimize for web option.

or you can do like I did and Use tools that optimize the png files like pngout for example .. I attached that as well
« Last Edit: July 05, 2022, 04:38:36 pm by mariush »
 

Offline sokoloff

  • Super Contributor
  • ***
  • Posts: 1799
  • Country: us
Re: What actual data is used to return a favicon to the browser?
« Reply #38 on: July 05, 2022, 04:30:44 pm »
How did you generate these?
For the second one, I took your file and ran it through sqoosh (online).

For the first one, I started with a randomly chosen favicon generator and picked whatever version of Courier it had available in its editor.
From my history, it was: https://favicon.io/favicon-generator/ (but again, was randomly chosen)
Opera is pretty dead, but not completely dead. Not sure where it is actually used now. Last definite use was Nokia Symbian phones, which died out 5+ years ago.
Note that PNG favicons are supported in Opera and in Opera Mobile and only unsupported in Opera Mini (released 2015).
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #39 on: July 05, 2022, 06:43:47 pm »
Gzip brings it from 1150 to 364.

Unfortunately, this is more complicated... gzipped favicons don't "just work". When you try to load it, you get Windows explorer offering to save the file. Evidently it needs special headers, and there are dozens of examples online of which none work :)

I have marked my changes below with comments:

Code: [Select]
// Headers for the favicon, designed to get the client to cache it and not request it again
static const uint8_t FAVICON_HEADER[] =
"HTTP/1.1 200 OK\r\n"
"Content-Encoding: gzip\r\n"  // **
"Content-Type: image/x-gzip\r\n"  // was x-icon
"Content-Length: 364\r\n"
"Date: Sat, 01 Jan 2022 00:00:00 GMT\r\n"
"Cache-Control: max-age=473384600\r\n"
"\r\n";


// The favicon file.
// Generated originally with https://favicon.io/favicon-generator/ or photoshop etc
// Converting file to .ico: https://convertio.co/
// GZIP compressor at https://gzip.swimburger.net/
// Converted to C array format by https://notisrac.github.io/FileToCArray
// ** insert length in content-length above **
// Debug with wget -S -o a.txt http://192.168.1.61:8080/favicon.ico

static const uint8_t FAVICON_BODY[] = {
  0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x03, 0x95, 0x94, 0xcb, 0x2f, 0x03, 0x51,
  0x14, 0xc6, 0xaf, 0xa4, 0x7f, 0x40, 0x57, 0xd6, 0x96, 0xfe, 0x21, 0x22, 0xc1, 0x8e, 0x78, 0xc5,
  0x42, 0x22, 0x12, 0x21, 0x6c, 0x48, 0x2c, 0x84, 0x58, 0x88, 0x85, 0xc4, 0x46, 0xc4, 0xa3, 0x21,
  0xa1, 0x25, 0xd4, 0x2b, 0x69, 0x55, 0x84, 0x48, 0x83, 0x8a, 0x88, 0x14, 0xed, 0xc6, 0xa2, 0x5a,
  0x22, 0x66, 0x98, 0xe9, 0xe3, 0x73, 0xa6, 0x27, 0xda, 0x8e, 0x3b, 0x33, 0x35, 0xdf, 0xe4, 0x4e,
  0x72, 0xcf, 0x77, 0x7f, 0xf7, 0x75, 0xe6, 0x8c, 0x10, 0x35, 0xf4, 0x78, 0xbd, 0x82, 0xde, 0x75,
  0xa2, 0xd3, 0x23, 0x44, 0xad, 0x10, 0xa2, 0x9e, 0x1a, 0x85, 0x28, 0xc2, 0xf1, 0xa2, 0x3c, 0x42,
  0x12, 0x16, 0xa2, 0x53, 0x68, 0x58, 0x83, 0xd4, 0xd2, 0x2a, 0x4c, 0x6a, 0x59, 0x97, 0xc7, 0xcc,
  0x5f, 0x4e, 0xe3, 0x53, 0xd3, 0x10, 0x8c, 0x03, 0xbd, 0x3b, 0x1c, 0x1b, 0x3d, 0x06, 0xc2, 0x09,
  0x20, 0x5f, 0x30, 0xf3, 0x77, 0x29, 0xa0, 0xc3, 0xcf, 0x63, 0xfa, 0xf7, 0x80, 0x13, 0x1a, 0xa3,
  0xe8, 0x5a, 0xc9, 0x9f, 0x08, 0xb3, 0xe7, 0xbb, 0x85, 0xa5, 0x96, 0xae, 0xd9, 0x6f, 0xdf, 0x02,
  0x5e, 0x15, 0xd9, 0x77, 0xe2, 0xf7, 0xe3, 0xec, 0x35, 0xd3, 0x19, 0x1e, 0xd2, 0xd6, 0xf3, 0xdb,
  0xf1, 0xd1, 0x17, 0xa0, 0xc9, 0x07, 0x34, 0x92, 0x17, 0x49, 0x5a, 0xb3, 0x76, 0xfc, 0xd3, 0x1b,
  0xd0, 0xba, 0xe1, 0x7c, 0x2e, 0x3b, 0x3e, 0xa5, 0x94, 0xef, 0x6b, 0xe6, 0x0c, 0x28, 0x38, 0xe3,
  0x25, 0xbe, 0x6f, 0x17, 0x98, 0x8c, 0x00, 0x6d, 0x9b, 0xdc, 0x1f, 0x3a, 0x04, 0xf4, 0x5c, 0x15,
  0xb8, 0x82, 0xaf, 0x6c, 0xc6, 0x7d, 0x3d, 0xbf, 0x57, 0x67, 0xff, 0xee, 0x3f, 0x9b, 0x07, 0x96,
  0x6f, 0xb8, 0xdf, 0x1d, 0x00, 0x32, 0x5f, 0xee, 0xf8, 0x5f, 0xcd, 0x5d, 0x70, 0x6c, 0x20, 0x08,
  0xa8, 0x59, 0xf7, 0x7c, 0x8e, 0xf6, 0x31, 0x1e, 0xe2, 0xf8, 0x58, 0x88, 0xfb, 0x6e, 0x78, 0x43,
  0xc6, 0xba, 0xc6, 0xfa, 0x86, 0x37, 0x7b, 0x6e, 0x9f, 0x07, 0xa7, 0xef, 0x2f, 0x43, 0xb5, 0xd4,
  0x15, 0x60, 0x7f, 0x25, 0x66, 0xf6, 0x3e, 0xbe, 0x35, 0xf8, 0xef, 0x81, 0x9e, 0x6d, 0xf6, 0x87,
  0x29, 0x5f, 0x47, 0x8f, 0xe6, 0xfa, 0x51, 0x75, 0x60, 0x35, 0x56, 0xce, 0xcb, 0xe2, 0x15, 0x90,
  0x28, 0xe6, 0x45, 0xfb, 0x57, 0xfd, 0x9e, 0x26, 0x65, 0x7f, 0xf0, 0x80, 0x26, 0xc5, 0x88, 0xfc,
  0x47, 0x70, 0xa7, 0x1f, 0x6b, 0xab, 0x44, 0x60, 0x7e, 0x04, 0x00, 0x00
};

Also most stuff online is to do with serving the icon file to the client with compression on the fly. In this case I am trying to store a compressed file and serve it as-is.
« Last Edit: July 05, 2022, 06:46:06 pm by peter-h »
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14476
  • Country: fr
Re: What actual data is used to return a favicon to the browser?
« Reply #40 on: July 05, 2022, 06:50:19 pm »
Is a favicon that big of a deal for your products?
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: What actual data is used to return a favicon to the browser?
« Reply #41 on: July 05, 2022, 06:54:22 pm »
For compressed files Content-Type should remain the same as for the uncompressed file. Content-Encoding field specifies the compression.
Alex
 

Offline mariush

  • Super Contributor
  • ***
  • Posts: 5029
  • Country: ro
  • .
Re: What actual data is used to return a favicon to the browser?
« Reply #42 on: July 05, 2022, 07:13:14 pm »
Yeah ... like the above guy says.
Don't change the content-type - content encoding tells the client what compression / encoding is used.
I'd also recommend not using such high values for cache, like 473384600  seconds ... that's like 5500 days. Some browsers may ignore big values assuming it's a configuration error. 
I'd also add , public after the max-age value

And... you could make your code simpler by giving a fake date, keep it consistent, don't change it every time ... ex say 1st of the current month current year and configure maximum cache time 3-6 months.
 
Or ... instead of cache-control max-age , you could use the Expires tag with a date in the future then the date: is irrelevant and may be omitted

Cache-Control: public
Expires: Sat 31 Dec 2022 23:59:59 GMT

Basically set expires to the last day of the current year

 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #43 on: July 05, 2022, 08:13:34 pm »
OK; thanks. This works now

Code: [Select]
// Headers for the favicon, designed to get the client to cache it and not request it again
static const uint8_t FAVICON_HEADER[] =
"HTTP/1.1 200 OK\r\n"
"Content-Encoding: gzip\r\n"
"Content-Type: image/x-icon\r\n"
"Content-Length: 364\r\n"
"Date: Sat, 01 Jan 2022 00:00:00 GMT\r\n"
"Cache-Control: max-age=473384600\r\n"
"\r\n";

Quote
"Content-Length: " STRINGIFY(FAVICON_BODY_LEN) "\r\n"

Is this a GCC compiler function? It would be useful to insert a compile-time computed sizeof() into the header. But all references I can find to STRINGIFY are to client side JSON stuff which I know nothing about.

Quote
Is a favicon that big of a deal for your products?

Not so much the favicon itself, although it helps to make the product look a bit more professional. My tab would probably be the only one without an icon...

But the bigger issues are with the client hammering the server. It was found, on the page which self refreshes at 1Hz, that there were 2x as many hits, the 2nd 200ms after the first. Chrome never stopped trying to download the favicon. There is a lot of stuff online about this, Chrome v. FF behaviour, and most of it is BS, but it is also mysteriously variable. I have two systems, with Chrome, win7-64, and one never stops requesting while the other one rarely requests. FF behaves a lot better. There is also a potential impact on usability since this box may need to be accessible over a 4G connection (which is flakey at the best of times, with huge packet latencies which readily break TCP/IP) and this gets a lot worse over a VPN which ought to be used for any such remote access. So I decided to look into this, obviously not realising it would be such a rabbit-hole. As I posted, a way was found early on to block the requests with just an oddball header, but this may fail one day.

Thanks all very much for all your help :)
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: What actual data is used to return a favicon to the browser?
« Reply #44 on: July 05, 2022, 08:18:22 pm »
But all references I can find to STRINGIFY are to client side JSON stuff which I know nothing about.

Have you tried adding the language to the search? This https://gcc.gnu.org/onlinedocs/gcc-4.8.5/cpp/Stringification.html is the first result for "c stringify"

But it won't be a computed value. There is no way to do compile-time sizeof() as far as I know. This macro would just turn the whole expression in the parameter into a string.
« Last Edit: July 05, 2022, 08:32:12 pm by ataradov »
Alex
 

Offline AndyBeez

  • Frequent Contributor
  • **
  • Posts: 856
  • Country: nu
Re: What actual data is used to return a favicon to the browser?
« Reply #45 on: July 05, 2022, 08:24:20 pm »
Chrome never stopped trying to download the favicon? The date is Sat, 01 Jan 2022 00:00:00 GMT. Make that 2032 and see what happens?

You can also respond with a '304' [HTTP 304 Not Modified], rather than treat the request as new and flush the whole .ico file into your bandwidth budget.

Quote
The HTTP 304 Not Modified client redirection response code indicates that there is no need to retransmit the requested resources. It is an implicit redirection to a cached resource. This happens when the request method is a safe method, such as GET or HEAD, or when the request is conditional and uses an If-None-Match or an If-Modified-Since header.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/304

And if that doesn't work, send a 404 [ Not found ]. Chrome might just then shut the # up.
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #46 on: July 05, 2022, 08:34:14 pm »
Reading NA's post again, he has
#define  FAVICON_BODY_LEN  123
in there so the sizeof(icon size) still needs to be manually entered. I must be missing something but I am not that clever :) Anyway it's hardly a big deal.

Quote
The date is Sat, 01 Jan 2022 00:00:00 GMT. Make that 2032 and see what happens?

I didn't have that date in there at the time. Now, with a proper valid favicon, Chrome behaves ok. In fact it's quite difficult to get it to un-cache it. It seems to retain it even across tabs (for the same URL).

Quote
flush the whole .ico file into your bandwidth budget.

It is now just 364 bytes. I am happy with that.

Not sure what happened with the early posts above talking about favicons < 100 bytes. No idea how to make those (and still actually show something). If you don't want to show anything, a gzipped white square would for sure be tiny.



Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14476
  • Country: fr
Re: What actual data is used to return a favicon to the browser?
« Reply #47 on: July 05, 2022, 08:39:25 pm »
In fact it's quite difficult to get it to un-cache it. It seems to retain it even across tabs (for the same URL).

It's not just with Chrome. Firefox does the same. Getting it to load a *modified* favicon for a given web page is a pain.
 

Offline ataradov

  • Super Contributor
  • ***
  • Posts: 11260
  • Country: us
    • Personal site
Re: What actual data is used to return a favicon to the browser?
« Reply #48 on: July 05, 2022, 08:41:07 pm »
in there so the sizeof(icon size) still needs to be manually entered. I must be missing something but I am not that clever :) Anyway it's hardly a big deal.
This is more so that you can enter the number one time and use it in both the header string and in the code that sends the data.

In fact it's quite difficult to get it to un-cache it. It seems to retain it even across tabs (for the same URL).
I think Shift-F5 would request everything from scratch.

No idea how to make those (and still actually show something).
It is not easy. If even the minimal 1 pixel GIF is 35 bytes.
Alex
 

Offline AndyBeez

  • Frequent Contributor
  • **
  • Posts: 856
  • Country: nu
Re: What actual data is used to return a favicon to the browser?
« Reply #49 on: July 05, 2022, 08:44:46 pm »
I didn't have that date in there at the time. Now, with a proper valid favicon, Chrome behaves ok. In fact it's quite difficult to get it to un-cache it. It seems to retain it even across tabs (for the same URL).
Good luck killing it! Chrome does a lot of undocumented stuff.

To view what's stored in your Chrome browser cache :
Code: [Select]
chrome://view-http-cache
To view other Chrome features :
Code: [Select]
chrome://chrome-urls
 

Online Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: What actual data is used to return a favicon to the browser?
« Reply #50 on: July 05, 2022, 09:45:36 pm »
If you use Linux, or have gzip installed, I recommend you use gzip -kn9 favicon.ico to get maximally gzip-compressed file.
It compresses your 1150-byte favicon.ico (with sha256sum 2c01904c0df69815f18b28d2a7e8e438c353b5352b42fb3699726b332b507be9) to just 111 bytes (included in example below).

Quote
"Content-Length: " STRINGIFY(FAVICON_BODY_LEN) "\r\n"
Is this a GCC compiler function?
No.  It is just a macro
    #define  STRINGIFY_(arg)  #arg
    #define  STRINGIFY(arg)   STRINGIFY_(arg)
that first expands the value of FAVICON_BODY_LEN, and then converts it to a string.

It would be useful to insert a compile-time computed sizeof() into the header.
Yep, but it ain't possible.  You can do something like
Code: [Select]
struct bucket {
    union {
        struct {
            int                         id;   /* == -1 */
            size_t                      value;
        }                               size;
        struct {
            int                         len;  /* > 0 */
            const unsigned char *const  ptr;
        }                               data;
        struct {
            int                         id; /* = 0 */
            size_t                      zero;
        }                               common;
    };
};

static const unsigned char  favicon_header_1[] =
    "HTTP/1.1 200 OK\r\n"
    "Content-Encoding: gzip\r\n"
    "Content-Type: image/x-icon\r\n"
    "Content-Length: "
;
static const unsigned char  favicon_header_2[] =
                       "\r\n"
    "Date: Sat, 01 Jan 2022 00:00:00 GMT\r\n"
    "Cache-Control: max-age=473384600\r\n"
    "\r\n"
;

static const unsigned char favicon_body[] = {
    0x1f, 0x8b, 0x08, 0x00, 0x00, 0x00, 0x00, 0x00, 0x02, 0x03, 0x63, 0x60, 0x60, 0x04, 0x42, 0x01,
    0x01, 0x06, 0x20, 0xa9, 0xc0, 0x90, 0xc1, 0xc2, 0xc0, 0x20, 0xc6, 0xc0, 0xc0, 0xa0, 0x01, 0xc4,
    0x40, 0x21, 0xa0, 0x08, 0x44, 0x1c, 0x0c, 0x80, 0x72, 0xde, 0x31, 0x10, 0x0c, 0x03, 0xff, 0x87,
    0x11, 0x60, 0x62, 0x12, 0x80, 0x63, 0x5c, 0x72, 0xc4, 0x98, 0x41, 0x8a, 0x38, 0x31, 0xea, 0x88,
    0xd5, 0x8b, 0x4d, 0x2d, 0x29, 0x7a, 0xd1, 0xfd, 0x49, 0xaa, 0x5e, 0x64, 0x3d, 0xe4, 0xe8, 0x45,
    0xd7, 0x47, 0x89, 0xfd, 0x94, 0xf8, 0x9f, 0x92, 0xb8, 0x43, 0x4f, 0x27, 0xc8, 0x62, 0xe4, 0x86,
    0xc9, 0x60, 0x06, 0x0c, 0x14, 0x02, 0x00, 0xc2, 0xd1, 0x9b, 0xb4, 0x7e, 0x04, 0x00, 0x00
};

static const struct bucket  favicon_response[] = {
    { .data = { .len = sizeof favicon_header_1 - 1, .ptr = favicon_header_1 } },
    { .size = { .id = -1, .value = sizeof favicon_body } },
    { .data = { .len = sizeof favicon_header_2 - 1, .ptr = favicon_header_2 } },
    { .data = { .len = sizeof favicon_body - 1,     .ptr = favicon_body     } },
    { .common = { .id = 0, .zero = 0 } }
};

int send_all(int fd, const struct bucket *all)
{
    int err;

    for (; all->common.id != 0; all++) {
        if (all->common.id > 0) {

            /* all->data */
            err = send_data(fd, all->data.ptr, all->data.len);
            if (err)
                return err;

        } else
        if (all->common.id == -1) {

            /* all->size */
            unsigned char  number[3 * sizeof (size_t) + 4];
            unsigned char *digit = number + sizeof number;
            size_t         value = all->size.value;
            do {
                *(--digit) = '0' + (value % 10);
                value /= 10;
            } while (value > 0);
            err = send_data(fd, digit, (size_t)(number + sizeof number - digit));
            if (err)
                return err;

        } else {

            /* Invalid bucket ID */
            return BAD_BUCKET;

        }
    }

    return 0;
}
where send_data(descriptor, pointer, length) sends data to the recipient, returning 0 if success and an error code otherwise; and BAD_BUCKET is a macro that evaluates to a suitable error code.

The idea is that each bucket can be a string or a size (unsigned integer value), or possibly something else you might wish to use at run time (date and time in HTTP format, perhaps?).

However, at least in Linux, I would just generate a favicon.h header file from a favicon.ico file at build time.

If you have Bash or sh (POSIX-compatible shell), then
Code: [Select]
#!/bin/sh
if [ -r favicon.ico ]; then
    rm -f favicon.ico.gz
    gzip -kn9 favicon.ico
fi
if [ ! -r favicon.ico ]; then
    echo "No 'favicon.ico' file."
    exit 1
fi
bytes="$(od -t x1 -A none favicon.ico.gz | sed -e 's|^ *|0x|; s| |, 0x|g; s|^|    |;s|$|,|')"
length=$(stat -c '%s' favicon.ico.gz)
httpdate="$(LANG=C LC_ALL=C date -uR | sed -e 's| +*-*00*$| GMT|')"
rm -f favicon.h
cat >favicon.h <<EOF
#ifndef   FAVICON_H
#define   FAVICON_H

const unsigned char  FAVICON_HEADER[] =
    "HTTP/1.1 200 OK\r\n"
    "Content-Encoding: gzip\r\n"
    "Content-Type: image/x-icon\r\n"
    "Content-Length: $length\r\n"
    "Date: $httpdate\r\n"
    "Cache-Control: max-age=157680000, public\r\n"
    "\r\n"
;

#define  FAVICON_HEADER_LEN  ((sizeof FAVICON_HEADER) - 1)
#define  FAVICON_BODY_LEN    $length

const unsigned char  FAVICON_BODY[FAVICON_BODY_LEN] = {
$bytes
};

#endif    FAVICON_H
EOF
if run first at build time, will generate a favicon.h file with the proper headers, with todays date and expiry time in five years.
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #51 on: July 05, 2022, 09:54:35 pm »
One could send out the header in two parts, with the size dynamically generated (sprinf) at runtime.

Or set up the header in RAM (not a "const") and modify the size.

Or,  in the good old days, use self modifying code ;)
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: What actual data is used to return a favicon to the browser?
« Reply #52 on: July 05, 2022, 10:29:37 pm »
One could send out the header in two parts, with the size dynamically generated (sprinf) at runtime.
That's basically the struct bucket implementation above, except that it covers both the header and data, and in an extensible format.

But yes, there definitely are many different ways to implement these, each with their own benefits and drawbacks.
I like these sprawling discussions when approaches are suggested, and those benefits and drawbacks weighed and discussed.
It might not be optimal for you for the current case at hand, but all this sprawl can be very useful later on, in other cases.  In my experience, they do tend to be.
 

Offline mariush

  • Super Contributor
  • ***
  • Posts: 5029
  • Country: ro
  • .
Re: What actual data is used to return a favicon to the browser?
« Reply #53 on: July 06, 2022, 09:49:41 am »
If you have to get one reading per second (1 Hz) or a reading every few seconds... you may want to consider keeping connections alive and reuse them but you'd also have to change your "web server" code a bit.

See Connection: keep-alive  : https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Keep-Alive

Set a timeout of around 5 seconds, set a max requests of whatever you're comfortable with (let's say 50)  and don't close connection as soon as you're done

Then the browser may request the page and when you're done sending the page the browser may request the favicon on the same open connection... and when you refresh page once a second, your kept alive connection may be reused saving some latency.

You could also consider using Javascript on the page to retrieve new measurements once a second or as often as you want, instead of reloading the whole page... as that could be annoying (page flickering, user not able to copy/paste stuff as the page keeps reloading etc)
For example, once the page is loaded, the javascript on the page could request json encoded data which could be something like  {id:1 , value:12.4}    and next reading you'd have {id:2, value:12.8} and so on... you'd want some unique id or the measurement time to know if it's a new measurement and not the old one, and the javascript code could add the fresh reading to the screen and replace the old one or append the value to a list (maybe you want a history of the last N readings on the page)

you could extend the json api to give you more results or based on some parameter

ex  GET /api/temperature.json?from=20220706125015

and your code replies

{

  count : 2,
  results: [
    { date:"20220706125015", value: "100" },
    { date:"20220706125016", value: "110" },
  ]
  error: false
}

The javascript code requests all values from that date yyyymmddhhmmss because that's the last time it has a reading for, or the date sent through the initial html page, or simply doesn't send that date parameter and gets the latest 1..10-whatever readings.
In the example above, it gets two results and pushes them on the screen, then waits 1s or more and requests data again, with the data parameter updated.

Note there's even more complex stuff, it's now possible to "stream" something, so your server could keep connection alive and push data and Javascript on the page could open a connection and keep it alive for hours and read chunks of data and decode them and put stuff on screen.

See https://developer.mozilla.org/en-US/docs/Web/API/Streams_API#examples
« Last Edit: July 06, 2022, 09:52:00 am by mariush »
 

Online Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6264
  • Country: fi
    • My home page and email address
Re: What actual data is used to return a favicon to the browser?
« Reply #54 on: July 06, 2022, 11:04:13 pm »
To add to what mariush wrote, the WebSockets API docs at Mozilla shows how to use WebSockets (both server and client side), if you want to use WebSockets instead of plain HTTP.

The difference to plain HTTP is that with WebSockets, instead of doing a new request once every second, the server/device will just push new data once every second or so.  Here, too, you'll definitely want to use JSON for the data.  On the web page, each such push will generate a MessageEvent on the open WebSocket, and you can set an onmessage handler to handle each new message, for example to add the new data to the display.

Simply put, instead of polling the server, the client page opens the websocket connection, and then the device pushes back the data (say, once a second or so, or however often you like).

In particular, if you design your JSON data packets so that each WebSocket message has at most 125 bytes of data, each TCP packet you send will have less than 136 bytes of data.  This means that you can do with just one HTTP request-response in progress at any point in time (including WebSocket handshake), but both an active WebSocket connection and a HTTP request-response in progress at the same time.  Depending on the available resources, you could actually have more than one active WebSocket connection at a time, allowing more than one concurrent client, while only doing one HTTP request-response at a time.

(For a long time, I've been occasionally wondering how to more efficiently process HTTP request headers.  In theory, for this kind of situations (where only a few of the headers matter, and basically the values do not need to be stored, only flags as to which value was seen for which header), it is possible to do with just two TCP packet payload buffers (so a bit over 3k of buffers), using a stream-type parser.  It would be an interesting exercise to see how compact one could make this kind of HTTP+WebSocket or HTTPS+WebSocket server for embedded devices and MCUs.)
 

Offline peter-hTopic starter

  • Super Contributor
  • ***
  • Posts: 3697
  • Country: gb
  • Doing electronics since the 1960s...
Re: What actual data is used to return a favicon to the browser?
« Reply #55 on: July 07, 2022, 06:06:43 pm »
I am on the limit of my HTML expertise here but presumably it is ok for a server to just keep pushing data out to the client, at say 1 sec intervals, without the client requesting it? Isn't this what Ajax is?

Currently, for the refreshed status page, I am using

Code: [Select]
<meta http-equiv=refresh content=1>
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline mariush

  • Super Contributor
  • ***
  • Posts: 5029
  • Country: ro
  • .
Re: What actual data is used to return a favicon to the browser?
« Reply #56 on: July 07, 2022, 06:51:56 pm »
No, that code simply tells the browser to reload the page approximately 1 second after the meta tag is parsed / page is loaded / connection is closed.

AJAX is functionality introduced in browsers...
A is short for Asynchronous , J is for Javascript , X is for XML  but nowadays XML is less used, JSON is more common/ popular.

Basically you can use Javascript to request a document to be loaded in background (while the page is still being loaded on user's computer, or some time after the page is completely loaded - that's the asynchronous bit) and when the transfer is complete, the browser resumes running your javascript code and you can do something with that content (populate the page with the data, using Javascript)

See the examples in the page : https://en.wikipedia.org/wiki/Ajax_(programming)
 

Offline sokoloff

  • Super Contributor
  • ***
  • Posts: 1799
  • Country: us
Re: What actual data is used to return a favicon to the browser?
« Reply #57 on: July 07, 2022, 09:36:38 pm »
I am on the limit of my HTML expertise here but presumably it is ok for a server to just keep pushing data out to the client, at say 1 sec intervals, without the client requesting it? Isn't this what Ajax is?
That technique is called "long-polling" (if you want a term to Google) and is valid, especially if the number of clients will be fairly low. https://ably.com/blog/websockets-vs-long-polling for some discussion on the topic.

AJAX and long-polling are not the same. AJAX can be used to implement client-driven polling, but that's about the only overlap.
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf