EEVblog Electronics Community Forum

Products => Computers => Programming => Topic started by: NW27 on November 18, 2020, 12:20:01 am

Title: Embedded Webpage design tools
Post by: NW27 on November 18, 2020, 12:20:01 am
Hi All,
I'm developing some major hardware with a STM32 as the core base.
I'm going to use a ESP32 as the WiFi Webpage front end.
I will basically be display ~ 40 pieces of information across a few pages.
Some will be text (some data entry would also be nice) and some graphs (think car dashboard. but it's not)

In the long past, I used Microsoft's Frontpage for this GUI style of design.
I don't want to spend heaps of time learning HTML etc therefore would like a GUI designer to create the principal web pages, drag labels around, create edit boxes etc.
After the principal layout is done, then I'm happy to get dirty in the code to populate the fields.

Thoughts on what web page designer to use?

Thanks,
Neil.
Title: Re: Embedded Webpage design tools
Post by: Mr.B on November 18, 2020, 12:33:12 am
I don't know of a GUI like what you describe, however I have used this...
https://github.com/Hieromon/ (https://github.com/Hieromon/)

I have used the AutoConnect lib for building the config system so that an end user connects their smartphone to the ESP32 to configure the SSID and passphrase of their local WiFi to get the ESP32 on their network.

The above has a dependancy of PageBuilder.
The PageBuilder lib allows you to extend the web pages into data reporting and settings, etc.
The possibilities are pretty well endless.

Knowledge of basic HTML is useful, but not essential.

The author has been fairly active with maintenance and new features up until about June this year.

This may suit your requirements.
Title: Re: Embedded Webpage design tools
Post by: cdev on November 24, 2020, 03:23:28 am
Please don't use any nonstandard cruft-laden HTML on your product's embedded GUI web pages.

Many of "The latest features" don't bring enough value to the table to make the requirement to use those allegedly latest browsers (which are laden with spyware you cant get rid of) worthwhile.

instead, people will just abandon or not buy your product.

That is a big enough mistake in my eyes, to make an otherwise great product bad.
Title: Re: Embedded Webpage design tools
Post by: Berni on November 24, 2020, 06:33:55 am
I used Dreamweaver (Now under Abobe) to make web pages.

It lets you do both at the same time, editing the page in the GUI or editing the code directly in a side by side window seeing changes update instantly between the two.

Also you don't need a lot of fancy magic to make a simple web page look good. You can go find some good CSS file examples on the web and just slap them onto a bare bone HTML document to make it look pretty. If you need slightly more fancy things like live updating values then sprinkle some javascript in there. If you need some graphic elements, make them in your favorite bitmap editor, or just grab some of the the mountains of freely available web page graphics assets on the web.

Designing a nice looking page also takes some design skill (that us engineers often lack) and this can be sidestepped somewhat by just finding something you like the look of and shamelessly rip off the visual style (as long as its not your competitors product visual style)

But if you want a really fancy UI with interactive graphs and all that, then you are best off using one of these newfangled whizbang HTML5 JS frameworks. Making there functionality from scratch takes a ton of work, but it also means you have to deal with ridiculously deep layers of abstraction like most of these fancy new web technologies.
Title: Re: Embedded Webpage design tools
Post by: Nominal Animal on November 25, 2020, 12:54:15 am
I do recommend doing it by hand, using HTML5+CSS for the layout and text content, and embedded SVG for the graphs.  This is because an embedded SVG graph is pure text, and is easy to generate on the fly.  Also, use https://html5.validator.nu (https://html5.validator.nu/) or some other HTML5 validator, to make sure the template page (and then the pages you generate) do not have any errors.  (That one validates embedded SVG 1.1 as well.)

Consider the following example HTML.  Copy and save it to a file ending with .html, and open it in your browser.
Code: [Select]
<!DOCTYPE html>
<html>
 <head>
  <title> Example Page </title>

  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  <style type="text/css">

   html, body {
             outline: 0px none;
              margin: 0 0 0 0;
              border: 0px none;
             padding: 0 0 0 0;
          background: #f7f7f7;
               color: #000000;
         font-family: Times New Roman, serif;
         font-weight: normal;
           font-size: 100%;
   }

   .box {
               float: left;
             outline: 0.25em transparent;
              margin: 0.25em;
              border: 0.05em solid #000000;
             padding: 0.5em 1em 0.5em 1em;
          background: #ffffff;
    }

  </style>
 </head>
 <body>

  <div class="box"> Example text box </div>

  <div class="box"> Example second text box </div>

  <div class="box">
   <svg xmlns="http://www.w3.org/2000/svg" version="1.1" viewBox="0 0 512 256" width="6em" height="3em">
    <rect x="0" y="0" width="512" height="256" fill="#ffffff" stroke="none" />
    <path d="M 0,128 L 512,128 M 256,0 L 256,256" fill="none" stroke="#000000" stroke-width="1.0" />
    <path d="M 24,128 L 31,137 38,147 46,156 53,165 60,173 68,181 75,189 82,196 89,202 96,208 104,213 111,217 118,220 126,222 133,224 140,224 147,224 154,222 162,220 169,217 176,213 184,208 191,202 198,196 205,189 212,181 220,173 227,165 234,156 242,147 249,137 256,128 263,119 270,109 278,100 285,91 292,83 300,75 307,67 314,60 321,54 328,48 336,43 343,39 350,36 358,34 365,32 372,32 379,32 386,34 394,36 401,39 408,43 416,48 423,54 430,60 437,67 444,75 452,83 459,91 466,100 474,109 481,119 488,128"
          fill="none" stroke="#990000" stroke-width="3.0" />
   </svg>
  </div>

 </body>
</html>
In the above snippet, all whitespace is equivalent, and the current indentation is just an effort to make it more readable.  (That is, you can replace every sequence of spaces and newlines with a single space, and it will render the exact same way.)

The boxes are in a floating layout, meaning the browser will pack them left-to-right, moving to a new line when the next box won't fit on the same line.  Change the size of your browser window to see the effect.  The two first boxes contain just example text, and the third box is an SVG graph containing one full sine wave centered at zero.
(The SVG 1.1 path element d="" attribute commands are listed here (https://www.w3.org/TR/SVG11/paths.html#PathDataMovetoCommands), but M x,y is MoveTo, L x,y is lineto, and C x1,y1 x2,y2 x,y is CubicBeziérCurveTo, to x,y (via control points x1,y1 and x2,y2).  You do not need to repeat the letter command for a continuous second element of the same type.  Lower case letters use relative coordinates, upper case letters absolute coordinates.
For filled shapes, z closes the polygon/shape (with a final line segment to the starting point, i.e. the last MoveTo point), so that it can be filled.  Typically, I only use rect and path elements in embedded SVG 1.1.  (The exception is logo stuff, as seen on my home page (https://www.nominal-animal.net/).  That embedded SVG was drawn in Inkscape, then finessed a bit by hand (removing unneeded attributes and so on, although Inkscapes "Save as Optimized SVG" is pretty darn good by itself).

All the HTML editors I have had to work with emit a lot of extraneous stuff.  (So do Inkscape and even Gimp, so it's to be expected, I guess.)
However, using any editor and then cleaning up the output by hand is a valid approach, as long as the initial HTML validates correctly.  Some editors don't bother, which causes all sorts of layout wonkiness down the road, as browser parsers change; validation is your only hope of future-proofing the output format.

For form-type layouts, I like to use something that web designers see as a sacrilege: table-based layout.  The reason is that automatic table layout seems to behave better across display devices and fonts used.  I am not certain if using traditional HTML table elements, or if div blocks that have the appropriate display table element roles would work better, though; I do not actively do web design anymore.

Consider the following modified example, that replaces the second box with a table-based form layout:
Code: [Select]
<!DOCTYPE html>
<html>
 <head>
  <title> Example Page </title>
  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
  <style type="text/css">

   html, body {
             outline: 0px none;
              margin: 0 0 0 0;
              border: 0px none;
             padding: 0 0 0 0;
          background: #f7f7f7;
               color: #000000;
         font-family: Times New Roman, serif;
         font-weight: normal;
           font-size: 100%;
   }

   table {
             outline: 0px none;
              margin: 0 0 0 0;
              border: 1px transparent;
             padding: 0.25em 1em 0.25em 1em;
     border-collapse: collapse;
   }

   .name {
          text-align: right;
      vertical-align: text-top;
               color: #666666;
   }

   .value {
          text-align: left;
      vertical-align: text-top;
    }

   .box {
               float: left;
             outline: 0.25em transparent;
              margin: 0.25em;
              border: 0.05em solid #000000;
             padding: 0.5em 1em 0.5em 1em;
          background: #ffffff;
    }

  </style>
 </head>
 <body>

  <div class="box"> Example text box </div>

  <div class="box">
   <table>
    <tr> <td class="name"> Foo: </td> <td class="value"> 5 </td></tr>
    <tr> <td class="name"> Bar: </td> <td class="value"> True </td></tr>
    <tr> <td class="name"> Wlargh: </td> <td class="value"> non-root </td></tr>
   </table>
  </div>

  <div class="box">
   <svg xmlns="http://www.w3.org/2000/svg" version="1.1" viewBox="0 0 512 256" width="6em" height="3em">
    <rect x="0" y="0" width="512" height="256" fill="#ffffff" stroke="none" />
    <path d="M 0,128 L 512,128 M 256,0 L 256,256" fill="none" stroke="#000000" stroke-width="1.0" />
    <path d="M 24,128 L 31,137 38,147 46,156 53,165 60,173 68,181 75,189 82,196 89,202 96,208 104,213 111,217 118,220 126,222 133,224 140,224 147,224 154,222 162,220 169,217 176,213 184,208 191,202 198,196 205,189 212,181 220,173 227,165 234,156 242,147 249,137 256,128 263,119 270,109 278,100 285,91 292,83 300,75 307,67 314,60 321,54 328,48 336,43 343,39 350,36 358,34 365,32 372,32 379,32 386,34 394,36 401,39 408,43 416,48 423,54 430,60 437,67 444,75 452,83 459,91 466,100 474,109 481,119 488,128"
          fill="none" stroke="#990000" stroke-width="3.0" />
   </svg>
  </div>

 </body>
</html>

Finally, we should discuss inline versus external files.  External files can reduce the server load, if the client knows the files can be cached (i.e. do not change).  If the external files change on every load, they cannot be cached, and putting them inline will both reduce server load/resource use and make the page load faster.

If TLS (https protocol instead of http) is used, then each connection has a relatively large overhead.  On embedded devices, this overhead is so large that inserting even unchanging resources inline instead of external files can reduce overall load times and resource use (on both the server and client ends).  Since browsers nowadays basically enforce TLS use, you'll very likely use TLS; and that is exactly why I do recommend including all resources inline to the HTML page emitted.

C/C++ code-wise, I would recommend generating the pages using an array or a singly linked list of static buckets and callbacks.  A static bucket is emitted as-is to the client, and a callback is a function that generates the HTML for the inline element, say SVG graph, or a form.  This way, if you end up deciding to switch between inline and external resources, you only move things between page chains.  (Also, I would not bother trying to find out the content length at all.  Leaving it out just means the browser shows the indeterminate-progress bar instead of the determinate-progress bar; and for the server to be useful, it should be faster than a human can get exasperated; it does not need to be instant.)

A simplified example:
Code: [Select]
struct snippet {
    void *const generator(); /* Parameters to the generator function, so it can emit to the client? */
    const char *const content; /* NUL/'\0'-terminated string in ROM/Flash */
};

const struct snippet  index_page[] =
{
    { .generator = NULL, content = html_headers },
    { .generator = NULL, content = index_title },
    { .generator = NULL, content = common_prefix },
    { .generator = index_content(), content = NULL },
    { .generator = NULL, content = common_suffix },
    { .generator = NULL, content = NULL } /* Terminator */
};
These structures are assumed to be in Flash, not RAM.  If the generator function pointer is non-NULL, it is called first; then the content (which is a string, i.e. terminated with a nul '\0' byte).  I recommend you store your strings in UTF-8, for maximal compatibility with, well, everything, as the HTML snippets above declare they do.  The array is terminated with a (NULL, NULL) pair.  Note that the generator function needs parameters, so it can emit content to the client; the parameters depend on the HTTP server library you use – basically identify the client connection somehow, allowing the function to "print" further content to the client – so I omitted those.  The same generator() function and content strings can obviously be used in multiple buckets in multiple chains; on a 32-bit MCU, each snippet structure takes 8 bytes, so any content longer than that, if reused, is a candidate for a snippet.
Of course, instead of a single index_content() that generates all of the index page contents, you would split it to whatever menubars etc. you use.  Also, each box in my earlier example in this post (if you'd like that sort of layout) would be a good candidate for a generator() function to emit.  Of course, a generator() function should be able to call other generator() functions, too, so that you can have different pages with subsets of the useful information.
(I personally dislike the "dashboard" layouts with dozens of small windows; I'd rather have them logically grouped.  I can always open multiple tabs or windows if I want to see them at the same time.)

(If you use GCC for compiling, it is possible to use a single pointer, and automatic ELF section variable addresses to determine whether the pointer points to rodata (read-only data) section, or text (code) section.  Pointers outside either section can be rejected, so only generator() functions can emit variable data.)
Title: Re: Embedded Webpage design tools
Post by: cdev on November 25, 2020, 03:01:11 pm
I do recommend doing it by hand, using HTML5+CSS for the layout and text content, and embedded SVG for the graphs.  This is because an embedded SVG graph is pure text, and is easy to generate on the fly.  Also, use https://html5.validator.nu (https://html5.validator.nu/) or some other HTML5 validator, to make sure the template page (and then the pages you generate) do not have any errors.  (That one validates embedded SVG 1.1 as well.)

Consider the following example HTML.  Copy and save it to a file ending with .html, and open it in your browser.
.....


Thank you! Tons of useful tips.

I still fall back to the tools that I know best (Zope) when I need to generate arbitrary HTML, XML, or any kind of text. Its great for that and extremely flexible. But this approach wont work for small devices as it requires a lot in the way of needed resources to run. Basically a real computer, running Linux, something that's not available in anything smaller than an SBC.

So Ive been saving resources on using a embedded web server as the main user interface to small devices.

Makes hardware interoperable, which is great!

Title: Re: Embedded Webpage design tools
Post by: Berni on November 25, 2020, 03:32:37 pm

AllWinner F1C200s, 408MHz~600MHz ARM9 (stably overclockable to 720MHz, unstably over industrial temp range to 900MHz), with copackaged 64MB RAM, QFN88 0.4mm-pitch.
SoChip X3 (AllWinner A33 SiP), 1.3GHz Quad-Cortex A7, with copackaged 128MB RAM now, 512MB RAM version later, large pitch 2L-PCB possible BGA.
Huawei HiSilicon Hi3518EV200, 1GHz Cortex A7, with copackaged 64MB RAM, QFN88 0.35mm-pitch.

The first two have GPL-violated proprietary Linux under NDA as well as community maintained GPL Linux (lacking hardware accelerated video codecs basically and less optimized). The latter has official GPL Linux and RTOS.

Any tips on getting started with one of these in Europe?

I looked at some of these Chinese SoC solutions before, but they turned out out to be impossible to buy and get documentation for unless you signed a pile of NDAs, was working with large volumes and made some weird agreements with assembly houses to force you into using them in return for letting you get your hands on the chips and docs.