Author Topic: Cheap microcontroller for JavaScript or fast interpreter.  (Read 4252 times)

0 Members and 1 Guest are viewing this topic.

Offline Georgy.MoshkinTopic starter

  • Regular Contributor
  • *
  • Posts: 228
  • Country: hk
  • R&D Engineer
Cheap microcontroller for JavaScript or fast interpreter.
« on: December 18, 2024, 03:44:43 pm »
I like to use JavaScript for fast prototyping of DSP algorithms. The drawback is that I need to translate JavaScript to C each time to make it work on the STM32. I kind of get used to it, but it's pretty time consuming. I always use C-like syntax, splice()/push() translates to memmove(), etc. Most problems occur when I replace untyped let/var with C-types uint8_t/uint16_t/q31_t. Plus, I often use integer/fixed-point optimizations which are hard to implement in JavaScript with lots of Math.floor() and unsigned right shifts to obtain results close to the microcontroller version. I also tried to write directly in C with visualisation using Sciter.JS. This approach has another problem with endless maintaining of conversion code between native C/C++ and JavaScript data arrays. Why not writing directly on the MCU? Slow flashing, clunky visualization. I partially solved this on H7 by implementing serial-to-ram bootloader that accepts the binary using a custom utility executed in post-build step. Maybe there is a way to use a limited JavaScript syntax that can be translated to C automatically? But the problem is mainly with 64-bit internal number format of JavaScript numbers, and there is a lot of work with optimizing those data arrays and preventing overflows or memory waste for each particular firmware.
« Last Edit: December 18, 2024, 03:51:09 pm by Georgy.Moshkin »
better late than never
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4853
  • Country: nz
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #1 on: December 18, 2024, 05:50:49 pm »
JavaScript is never going to be tiny, or fast on a microcontroller, but here is one that has been used for more than a decade at Samsung designed for largish ones: 64k RAM, >256k flash

https://github.com/jerryscript-project/jerryscript
 
The following users thanked this post: pardo-bsso

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 9491
  • Country: fi
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #2 on: December 18, 2024, 07:08:23 pm »
But is the "final product" on MCU? Probably it is because why else you would be porting it there?

And if yes, it seems a bit backwards to let the prototyping flow limit and dictate the final output. I would try to reverse this, basically tackling the following questions:
* Why is writing DSP algorithms in C slow? Can you e.g. come up with better libraries (existing or your own), utility functions, headers, macros to help development?
* Why is Javascript needed to prototype on PC? Surely PCs run software written in C all the time. I understand the advantage of a browser, but can you come up with an alternative way to feed data into browser, or visualize from C with something else than browser?

I deal with a somewhat similar thing, developing algorithms when a final product is a resource constrained microcontrolled. I initially did algorithm prototyping in Matlab/Octave but I have reduced that to very simple proof of concepts (not something that would be complete enough to be ported) and go into C as early as possible. I spent some effort making the firmware compile as a PC version (e.g., replacing some Ethernet chip related stuff with OS socket stuff, and writing simple "simulator" functions for hw-like stuff. I spent some time to write generic enough CSV parsers for input data to enable long simulation runs bypassing the usual networking layer, and output CSVs which can be plotted in... drumroll, Matlab/Octave from where I started. This has enabled me to just jump into algorithm development in C directly and not think about porting - and worse, maintaining two copies of the algorithm in two different languages which need to be in sync.
 

Offline bson

  • Supporter
  • ****
  • Posts: 2506
  • Country: us
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #3 on: December 18, 2024, 11:48:53 pm »
I would prototype it on a desktop computer using test data in files.  Make sure it's at least working correctly and doing what it's supposed to before even considering putting it on a microcontroller, because debugging it there will be significantly more difficult.  Optimize it on the desktop, at least the things you mention.  Then port it to a microcontroller.  JS, however, is probably not fast enough, but once the basic functionality is correct as long as you don't wrap it into obscenely complicated, highly normalized functional code, just reimplementing it as C/C++, perhaps with select inline assembly, should be straightforward.
 

Offline Georgy.MoshkinTopic starter

  • Regular Contributor
  • *
  • Posts: 228
  • Country: hk
  • R&D Engineer
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #4 on: December 19, 2024, 02:27:59 am »
* Why is writing DSP algorithms in C slow? Can you e.g. come up with better libraries (existing or your own), utility functions, headers, macros to help development?
* Why is Javascript needed to prototype on PC? Surely PCs run software written in C all the time. I understand the advantage of a browser, but can you come up with an alternative way to feed data into browser, or visualize from C with something else than browser?
The slow part is updating of visualization code. With an HTML+JavaScript tab in a browser some changes can be made faster, and then I can try few smaller changes literally spending 3 seconds each by pressing F5 and looking into graphs/console. With the microcontroller tools there is a lag coming from building, flashing, looking for the output in serial monitor (e.g., responses to different commands). With a JavaScript it feels so much faster: I have notepad++ opened and a web browser tab + console on the second monitor. Make a change - F5 - ready.
It seems that development goes faster when the visualization and the algorithm both implemented using same language (JavaScript in this case). I tried STM32CubeMonitor multiple times, but the usability kind of worse than simply putting HTML buttons/canvases and using several JavaScript functions. Dynamic typing accelerates the development, but followed by a tedious conversion, very prone to bugs (overflows).

You are right about better libraries, maybe some custom solution to ease the visualization process, to exchange the data between browser and microcontroller using a more standardized way.

JavaScript is never going to be tiny, or fast on a microcontroller, but here is one that has been used for more than a decade at Samsung designed for largish ones: 64k RAM, >256k flash
https://github.com/jerryscript-project/jerryscript
I've seen this one. Interestingly, my search started from the opposite: I tried to find a C interpreter written in JavaScript, or something that allows to use C code directly from JavaScript functions.

I would prototype it on a desktop computer using test data in files.  Make sure it's at least working correctly and doing what it's supposed to before even considering putting it on a microcontroller, because debugging it there will be significantly more difficult.  Optimize it on the desktop, at least the things you mention.  Then port it to a microcontroller.  JS, however, is probably not fast enough, but once the basic functionality is correct as long as you don't wrap it into obscenely complicated, highly normalized functional code, just reimplementing it as C/C++, perhaps with select inline assembly, should be straightforward.
You are absolutely right. That's exactly what I am doing. Here is my workflow:
1. I create a simple firmware that streams sensor data to the serial port and saved to a computer file, very similar to audio recorder.
2. I load this file into my JavaScript template. I already have several templates from previous projects that allow to load and parse binary files. For example, some sensors provide 16-bit data, 32-bit data, signed/unsigned, etc. At this point I have one or several arrays filled with the recorded data
3. I start to create an algorithm. Again, have multiple snippets of code from previous projects that allow me to output 1D/2D data on HTML Canvas, scroll through it, display values under the cursor.
4. I test the algorithm on several recordings as shown in this video. I have used recordings for up/down/left/right and rotational gestures.

After algorithm is ready and tested on several recordings, the tedious part begins. I start to transfer the code from JavaScript to C.
At this point I use web serial api to stream recorded data back to the microcontroller to emulate sensor output. Resulting variables are streamed back to the browser and visualized along with JavaScript algorithm graphs. For example, I compare FFT output, that I have the same magnitudes. Or I compare correlation coefficient graph. If something is wrong, graphs will be different.

I get a very good working algorithm fast, but then I struggle with the "transfer" part. This video shows before/after and scrolling through the data recordings using web browser.

Update: I re-read my topic and here is the summary:
- I quickly develop great algorithms on my PC in JavaScript, but then I waste a lot of time porting them to C for the microcontroller
- Know about MicroPython and open-source JavaScript interpreters, but these great tools come at a cost in terms of memory and performance requirements
- Familiar with Matlab/Octave, but it still requires rewriting the algorithm in C
- Pretty experienced with STM32CubeMonitor (Node-RED based), but I find it somewhat clunky. It would feel more natural if I could access visualization tools directly from the C source code
« Last Edit: December 19, 2024, 03:25:11 am by Georgy.Moshkin »
better late than never
 

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 9491
  • Country: fi
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #5 on: December 19, 2024, 07:02:46 am »
With the microcontroller tools there is a lag coming from building, flashing, looking for the output in serial monitor (e.g., responses to different commands).

My point was, compile the microcontroller code natively on PC. When only the algorithm module changes, building on a usual PC takes split second and there is no flashing to be done. This easily reduces iteration cost from 30-60 seconds to just a few.

When writing standard-compliant C and using stdint types, portability issues are rare, the exact same code will do exact same thing on PC and on MCU. (Of course final verification and testing is ultimately needed to prove this.)
 
The following users thanked this post: SiliconWizard

Offline sharow

  • Contributor
  • Posts: 19
  • Country: 00
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #6 on: December 19, 2024, 08:38:45 am »
I tried to find a C interpreter written in JavaScript, or something that allows to use C code directly from JavaScript functions.

You can consider emscripten perhaps.
But it requires install toolchain and glue-code tho.
 

Online peter-h

  • Super Contributor
  • ***
  • Posts: 4448
  • Country: gb
  • Doing electronics since the 1960s...
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #7 on: December 19, 2024, 02:22:02 pm »
May I ask? why use JS?

Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline coppice

  • Super Contributor
  • ***
  • Posts: 10206
  • Country: gb
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #8 on: December 19, 2024, 02:44:58 pm »
I always use C-like syntax, splice()/push() translates to memmove(), etc.
You try to follow a C approach, but don't write in C? You are just trying to make your life hard.

I think most experienced people writing DSP code for MCUs do roughly what I do. I write in C. Not because it is best, but because its common to so many platforms. I take care that all my data types properly allow for compiling on both a PC and for the target MCU. I usually try to write a little code for the MCU target to capture data from the sensors, and store it in files on my PC. Then I can happily play around on the PC getting my code to process those captured files correctly. If the application is basically a serial flow I can usually just recompile for the MCU, perhaps have a little debugging of the differences between the code reading from files, and reading from the real sensors, and the application works. If there are control loops involved, additional work with the real time dynamics of the loop may be needed, debugging on the target, as that is the only place where real time is real.
 
The following users thanked this post: mikerj, Siwastaja

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 3305
  • Country: ca
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #9 on: December 19, 2024, 04:58:23 pm »
The slow part is updating of visualization code. With an HTML+JavaScript tab in a browser some changes can be made faster, and then I can try few smaller changes literally spending 3 seconds each by pressing F5 and looking into graphs/console.

Write the code in C and compile it with MSVC++ or GCC. If you can't visualize in C and must use the JavaScript, let your C program create a file with output data and pass the file to the JavaScript. This won't take much longer and will eliminate double-language problems.
 

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 9491
  • Country: fi
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #10 on: December 19, 2024, 05:04:22 pm »
If you are extremely keen on the idea of using a browser interface, maybe even use libwebsockets or similar; the PC build of your firmware would then act as web server and you could control it through the browser and get realtime data plotting on browser.
« Last Edit: December 19, 2024, 05:06:22 pm by Siwastaja »
 
The following users thanked this post: spostma

Online peter-h

  • Super Contributor
  • ***
  • Posts: 4448
  • Country: gb
  • Doing electronics since the 1960s...
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #11 on: December 19, 2024, 05:43:48 pm »
It is fairly easy to write a primitive HTTP server which just polls the IP stack for incoming data and when it sees some, it waits for say 100ms (to make sure a whole packet is available) and then fetches the packet and does a string compare for "GET..." and picks up the bit after it, so you can build a web form which acts on keyboard key inputs and mouse clicks. In the browser you need a bit of JS to make it all hang together, because a browser cannot just send keys to a web page, so JS is needed to convert a keyboard keypress into a web page request ("GET...").

The above (the timeout etc) is a hack but works perfectly in the context of getting a simple HTTP web page which accepts keystrokes, clicks on links, etc. It tends to break if there is other TCP/IP stuff going on ;)

I've done this on a 32F417 product I developed recently. All in GCC C. I don't know much about JS so I paid some guys on freelancer.com to product that, for peanuts. If you want I can send you the HTTP server source (PM me). It uses the Netconn API of LWIP, not sockets, but it should be trivial to change it to sockets. The web page is generated with C strings. Simple enough although there are some irritating gotchas because a browser can display only a properly formed web page, nothing less.

I believe there are other ways to implement a keyboard-server interface e.g. AJAX but I never played with that.

Some reading e.g. here
https://www.eevblog.com/forum/programming/what-is-the-http-server-client-interaction-to-do-file-transfers/

Much better to do this because it "just works" with a minimal target footprint (apart from text strings, the whole HTTP server is probably 1-2k) and works with any browser.

FWIW a Python interpreter (probably a similar thing to your JS interpreter) was examined, to run on the target, and it would have used maybe 50k of binary, plus 50k of RAM, plus a load of time interfacing it to the hardware features.

Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Online tszaboo

  • Super Contributor
  • ***
  • Posts: 8301
  • Country: nl
  • Current job: ATEX product design
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #12 on: December 20, 2024, 12:24:22 am »
I also have a question, can you recommend a good hammer to fit the square block to the round hole?
 
The following users thanked this post: Doctorandus_P

Offline Georgy.MoshkinTopic starter

  • Regular Contributor
  • *
  • Posts: 228
  • Country: hk
  • R&D Engineer
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #13 on: December 20, 2024, 04:36:40 am »
I have no problems with visualizing. I have used Sciter.JS as a GUI and C++ for an algorithm for a long time. What I don't like is conversion between C data and JavaScript GUI output. There are good points provided in this topic. What I want to try is to make a minimalistic JavaScript HTML Canvas framework which is controlled from the microcontroller. Something very simplistic: controller sends commands to create a 2d canvas, and web browser creates it. The same with the pixels, graphs, numbers. But the development should be done on a more powerful MCU to eliminate flashing operation. (Running from RAM). Ultimate goal is to reduce number of used IDEs. Currently, I use notepad++  for HTML/JavaScript/Sciter.JS editing, codeblocks for C++ code, stm32cubeide for  microcontroller programming. I think it can be automated in a way that everything can be done without leaving STM32CUBEIDE. Imagine a web browser window opened on a second monitor and automatically updating layout and content of various GUI elements depending on what is being sent from STM32's serial port through web serial api.
better late than never
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4853
  • Country: nz
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #14 on: December 20, 2024, 05:35:13 am »
I have no problems with visualizing. I have used Sciter.JS as a GUI and C++ for an algorithm for a long time. What I don't like is conversion between C data and JavaScript GUI output.

But why not?  JavaScript uses IEEE double precision FP values for everything, and C perfectly handles those. It's also very easy to read and write JSON from C.

I just don't understand the problem here.

It's also very easy to run C/C++ code in the web broswer if you want to, using emscripten. And it's quite frankly, stupid to run anything except C/C++ on microcontrollers.
« Last Edit: December 20, 2024, 05:37:22 am by brucehoult »
 

Offline Georgy.MoshkinTopic starter

  • Regular Contributor
  • *
  • Posts: 228
  • Country: hk
  • R&D Engineer
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #15 on: December 20, 2024, 05:38:29 am »
I've done this on a 32F417 product I developed recently. All in GCC C. I don't know much about JS so I paid some guys on freelancer.com to product that, for peanuts. If you want I can send you the HTTP server source (PM me). It uses the Netconn API of LWIP, not sockets, but it should be trivial to change it to sockets. The web page is generated with C strings. Simple enough although there are some irritating gotchas because a browser can display only a properly formed web page, nothing less.
Thank you, peter-h, and for the info on Python. It's an interesting approach, something to consider.

I also have a question, can you recommend a good hammer to fit the square block to the round hole?
Haha, I see what you mean. But when you look it from practical perspective, I am 100% correct. Let's look at this great example:
STM32CubeMonitor, a "family of tools helps to fine-tune and diagnose STM32 applications at run-time by reading and visualizing their variables in real-time".
Ok, let's try to visualize a simple 1D array using this tool and the only way you will find is... my topic on how to achieve this:
https://community.st.com/t5/stm32cubemonitor-mcus/how-to-watch-an-array-using-stm32cubemonitor/td-p/225086/page/2
Seemingly right tool has no built-in solution to display a simple one-dimensional array. Btw, I've applied for a job at ST Microelectronics, but after "carefully reviewing my application" they decided that "other candidates better fit the requirements of the position". I need pretty basic functions, but there are none or they are poorly implemented.

Another example. Assume we need to find some bug which is related to a wrong DMA stride (we don't know yet). Is "debug" a right tool for this task? No. A obvious solution is to display memory contents as a 2d image. Highlight every byte that changes over time and you will spot the problem by noticing a dashed line of pixels in less than a second. After you fix the error, the dashed line will become a half-length whole line of pixels (changed stride from 32bit to 16bit).
better late than never
 

Offline Georgy.MoshkinTopic starter

  • Regular Contributor
  • *
  • Posts: 228
  • Country: hk
  • R&D Engineer
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #16 on: December 20, 2024, 06:04:03 am »
But why not?  JavaScript uses IEEE double precision FP values for everything, and C perfectly handles those. It's also very easy to read and write JSON from C.
Here is what's happening. Let's look into some of my old Sciter.JS developments:
Code: [Select]
function updFrame()
{
let someArr = [111,222,333];
let res = Window.this.assetInterface.readWave(someArr);
let colors = ["Black", "Red", "Green", "Blue", "Orange", "Gray", "Purple", "Lime"];

context[0].fillStyle="White";
context[0].fillRect(0,0,canvas[0].width, canvas[0].height);

for (let k = 0; k < 4; k++)
{
context[0].strokeStyle = colors[k];
context[0].beginPath();
context[0].moveTo(0,0);

for (let i = 0; i < GESTURE_FIFO_SIZE; i++)
{

let x1=canvas[0].width*i/GESTURE_FIFO_SIZE;
let y1=canvas[0].height-res[i*4+k];

if (i>0) {context[0].lineTo(x1,y1);} else {context[0].moveTo(x1,y1);}
}
context[0].stroke();
}
document.getElementById("testImage").requestPaint();
requestAnimationFrame(updFrame);
}
The above code is opened in Notepad++
It displays four curves that are coming from the following C++ code:
Code: [Select]
std::vector<int> readWave(const sciter::value vec)
{
  // I can get some values sent from HTML (let someArr = [111,222,333]);
  int32_t firstVal=vec[0].get<uint32_t> ( );
  int32_t secondVal=vec[1].get<uint32_t> ( );
  int32_t thirdVal=vec[2].get<uint32_t> ( );

 // do something here
  // read data from serial port
   sendGet(&myPort,(uint8_t*)&dummy,2,GESTURE_FIFO_SIZE*4);
   std::vector<int> tempArr (GESTURE_FIFO_SIZE*4);
   for (int i=0; i<GESTURE_FIFO_SIZE*4; i++) // loop
   {
      tempArr[i]=(uint8_t) dummy[i];
   }
  // send data to HTML GUI
   return tempArr;
}
The above code is opened in CodeBlocks.
The problems are:
1. Data changes daily. I can add array, remove some array, add some variable.
So, the first problem is that I need to maintain conversion code in C++ and in JavaScript simultaneously.
2. I don't like to press Ctrl+S/Ctrl+Shift+S in notepad++, switch to Codeblocks and press F9. It's so much slower that pressing F5 in web browser. Even if it's automated.
3. Sooner or later, I start to write some processing in JavaScript GUI code. While it's wrong, it's often faster this way. It comes very naturally. While I can force myself to keep writing everything in C, in many cases it is 5 minutes vs 1 minute or even worse. For example, making two new arrays in C++, maintaining the "transfer" to Javascript VS doing it Javascript if the data is already there. Therefore, at some point I have abandoned this CodeBlocks+SciterJS approach.  I can get the working prototype much faster using a web-browser only approach. But at a cost: need to port it to embedded C later.
I did the Sciter and later SciterJS approach for 3+ years and can trace it on a Sciter forum. I don't think that it's lack of experience thing. In my opinion, it's a great tool for the final product GUI, but not for an iterative design.

About "right" tools. In context of STM32CubeMonitor adding a new array / variable means that you need to reload ELF file. It's not only clicking though. With lots of arrays and variables CubeMonitor freezes for 1+ minute before you can press the confirmation button.
« Last Edit: December 20, 2024, 06:20:17 am by Georgy.Moshkin »
better late than never
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4853
  • Country: nz
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #17 on: December 20, 2024, 06:44:04 am »
I don't know how anything can take 1 minute or 5 minutes to compile and flash something for a microcontroller. My typical experience is 5 seconds or so. But I've never used CodeBlocks or STM32CubeMonitor. Maybe they're crap, idk.

Downloading python or javascript or other interpreted code to a microcontroller can reduce turnaround to a fraction of a second, but at a 100:1 or more cost in execution speed once it gets there. And then you're looking at a 100 MHz microcontroller vs a 5 GHz PC you're already fighting well uphill for execution speed on the microcontroller, even using C.

Furthermore, Javascript on a PC uses a JIT compiler that uses many MB -- maybe 100s of MB -- of RAM to produce fast native code for the PC. That capability is never going to fit on an STM32 which I think go up to a maximum of something like 640 KB of RAM.

At a guess, you'd be looking at a 10,000:1 performance difference between JavaScript on an STM32 vs JavaScript in your PC web browser.

Even Lua doesn't currently have a JIT running on microcontrollers.

It would be possible to make some on-device  JIT for microcontrollers with a few hundred KB of RAM and a couple of MB of flash, but I don't know of anyone who has.

If you want sub-second downloading / updating of code on a microcontroller with somewhat near native code speed then I think Forth is probably your only option, unless you yourself turn out to be the first to write an on-chip JIT for JavaScript or Python or Lua or Scheme or similar.

You could also make something that compiled individual C functions on a PC and STM32 code in flash that received them, wrote them into RAM, and ran them from there. When I say "you" I mean someone, possibly not actually you.
« Last Edit: December 20, 2024, 06:46:19 am by brucehoult »
 

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 9491
  • Country: fi
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #18 on: December 20, 2024, 07:45:58 am »
To me it sounds like modifying your firmware project so that it compiles both for microcontroller and for PC, and adding libwebsockets to the PC compilation side, streaming data into web browser (so that you connect your browser to localhost) would actually be pretty close to what you want?

Javascript part would receive data (such as your 1D array) from firmware through websocket, plotting it on screen. And possibly have buttons / text fields to give commands to the firmware part.

By compiling the firmware on PC and running web server there, you don't have to think about flashing delays, resources, or networking on real hardware.

Then you can use the exact same firmware to do verification / integration testing, running longer test vectors (e.g., input as CSV files).

And the IDE problem is "doctor, it hurts when I do this" problem. If your problem is having multiple IDEs, simple solution, stop using them? I never use any and don't understand why would you do that.
« Last Edit: December 20, 2024, 07:50:29 am by Siwastaja »
 

Online peter-h

  • Super Contributor
  • ***
  • Posts: 4448
  • Country: gb
  • Doing electronics since the 1960s...
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #19 on: December 20, 2024, 08:05:40 am »
I thought the OP was looking for a way to quickly modify the properties of a DSP algorithm running in an embedded target. Hence my suggestion of a primitive HTTP server, which accepts up, down, value entry, etc.

I even implemented a file upload (there is a FAT-FS 2MB FAT12 filespace) so one can update the whole firmware. With AES256, CRC32, etc, checks ;)

Yes there must be slick libraries for that sort of client-server functionality, but then you have to learn how to use them :)
Z80 Z180 Z280 Z8 S8 8031 8051 H8/300 H8/500 80x86 90S1200 32F417
 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 3305
  • Country: ca
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #20 on: December 20, 2024, 03:10:23 pm »
I just don't understand the problem here.

Me neither. This reminds me of

"But I was thinking of a plan
To dye one's whiskers green,
And always use so large a fan
That it could not be seen."

Engineering is about simplicity. If you pile lots of things together and then work on negating the influence of these things on your work process you will waste lots of efforts and the solution will be very convoluted.

Instead, isolate your signal processing code into a self-sufficient module, make sure you can compile the code both on PC and on MCU. On PC, insert the code into an application which does visualizations. Once your code is working, all you need to do is re-compile the MCU project and run it on MCU.  That's all there's to it.

If I was doing this, my task would compile the code on PC, process a training set of hundreds (or thousands) pre-recorded gesture files, calculate statistics to evaluate recognition accuracy, isolate bad cases and print debug info on them. With all this written in C, it would probably take a few seconds to compile and run through the whole set. Then, once the test set is working, run it through much bigger and independent database of gestures to see if it still works.

Testing on the chip is prohibitively time cosuming - producing even a 100 of gestures manually takes too much time. Using pre-recorded gestures is the only way.
 
The following users thanked this post: SiliconWizard

Offline Georgy.MoshkinTopic starter

  • Regular Contributor
  • *
  • Posts: 228
  • Country: hk
  • R&D Engineer
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #21 on: December 21, 2024, 02:44:14 am »
Engineering is about simplicity. If you pile lots of things together and then work on negating the influence of these things on your work process you will waste lots of efforts and the solution will be very convoluted.
I agree about simplicity, and the risk of spending too much effort without much improvement is high. Let's not forget that I have a great working algorithms in JavaScript. MinGW C + external Sciter.JS GUI approach was irritating enough to try JS-only. To be honest, I even don't care about the time that it takes to create JS-only prototype (within reasonable limits). Because I feel great working on the prototype, interesting, there is a sense of progress and nothing stops the momentum. But later I experience the consequences of using JS instead of C.

Instead, isolate your signal processing code into a self-sufficient module, make sure you can compile the code both on PC and on MCU. On PC, insert the code into an application which does visualizations. Once your code is working, all you need to do is re-compile the MCU project and run it on MCU.  That's all there's to it.

If I was doing this, my task would compile the code on PC, process a training set of hundreds (or thousands) pre-recorded gesture files, calculate statistics to evaluate recognition accuracy, isolate bad cases and print debug info on them. With all this written in C, it would probably take a few seconds to compile and run through the whole set. Then, once the test set is working, run it through much bigger and independent database of gestures to see if it still works.

Testing on the chip is prohibitively time cosuming - producing even a 100 of gestures manually takes too much time. Using pre-recorded gestures is the only way.
A great advice! A small remark is needed here. I am not working on gesture recognition. The product prototype that I was demonstrating at the 2023's competition used STM32H7 only to demonstrate possible applications, such as real-time audio pitch for DJ turntables, image scroll/zoom on the screen, etc. The recognition algorithm itself is fully analytical, doesn't use any neural networks or patterns, and can work on a sliding window without any memory by storing temporary data in few registers on ancient 16bit MCU. That was easy stuff! Made by me to meet the criteria of the competition to get funds for a greater project. And the selling attempt was basically to meet the criteria of "entrepreneurship". I am more like a researcher, but every entity I've reached to was redirecting me to this enterpreneurship competition. So, the video above is only to demonstrate how visualization works in the browser and the performance of final algorithm running on the device.

I'll try to rephrase. Imagine I do exactly as you say, and you can control each my step. So, I have written an "isolated signal processing code into a self-sufficient module" - or have I? This code changes all the time, and the difficulty is here:
1. I am not aware of any IDE/framework that will allow me to create a windows32 executable and create custom GUI outputs in a same convenient manner as in JavaScript+HTML. There are frequent changes, and I usually have a chain of algorithms with certain inputs/outputs that sometimes I want to visualize.
2. Dynamic typing in JavaScript helps a lot but complicates the porting on limited memory device. For example, copy-pasting of something like a=b-c to C from JavaScript will create a bug if c>b and both defined as unsigned numbers (uint8_t).
« Last Edit: December 21, 2024, 02:52:46 am by Georgy.Moshkin »
better late than never
 

Offline Siwastaja

  • Super Contributor
  • ***
  • Posts: 9491
  • Country: fi
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #22 on: December 21, 2024, 07:57:49 am »
Let's not forget that I have a great working algorithms in JavaScript.

I think that if your greater plan is to have these run on various devices, including embedded (but possibly also desktop and servers), reimplement them in C and after that, keep maintaining them in C; C is probably one of the sanest choices between portability and efficiency and it seems you already have grasp of the language. Choosing right tool for the job is important, and maintaining everything in two languages is not a long term solution (and probably not even short term) as you have realized already.

Quote
1. I am not aware of any IDE/framework that will allow me to create a windows32 executable and create custom GUI outputs in a same convenient manner as in JavaScript+HTML. There are frequent changes, and I usually have a chain of algorithms with certain inputs/outputs that sometimes I want to visualize.

You need a graphics library or data visualization library or GUI library. If you have been googling IDEs and frameworks no wonder you won't find anything. You don't need IDE or framework; "frameworks" specifically are evil's tools to lock down choices and add complexity to any project.

There are so many visualization libraries available that you will struggle to choose; I have used SFML in past but it's no good for graphs etc. I'm sure you can get some recommendations here, or do a Google search.

Quote
2. Dynamic typing in JavaScript helps a lot but complicates the porting on limited memory device. For example, copy-pasting of something like a=b-c to C from JavaScript will create a bug if c>b and both defined as unsigned numbers (uint8_t).

Yeah. This is solved by switching the algorithm into one language only. If you don't feel like using graphics libraries from C or C++, JS can still be decent option for visualization; in that case you would want to output/input data from/to C using double FP data type, but keep the algorithm itself C only.
« Last Edit: December 21, 2024, 08:00:33 am by Siwastaja »
 

Offline NorthGuy

  • Super Contributor
  • ***
  • Posts: 3305
  • Country: ca
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #23 on: December 21, 2024, 05:42:56 pm »
Imagine I do exactly as you say, and you can control each my step. So, I have written an "isolated signal processing code into a self-sufficient module" - or have I? This code changes all the time,

You're in the process of creating it (since the code is changing). You currently have 2 codes - JS and C, and you must change them both in sync. This is a nightmare, even if you find/write tools to automate this somehow. I propose you rewrite it all in C and maintain it in C. Your module will have well-defined inputs and outputs. Physically, this would be a number of .c and .h files. You can sym-link them into any number of project. The point is, whether you compile the firmware for an MCU (or for a number of MCUs) of for a test program on PC (or for lots of different test programs), you always use the same files - once a file changes, this change is visible everywhere.

1. I am not aware of any IDE/framework that will allow me to create a windows32 executable and create custom GUI outputs in a same convenient manner as in JavaScript+HTML. There are frequent changes, and I usually have a chain of algorithms with certain inputs/outputs that sometimes I want to visualize.

IDE doesn't create executables, compilers do. There are two very common on Windows - MSVC++ and GCC, both free. Create a bat file or a makefile which compiles the project for you (this is very short, may be even one line). A decent text editor will let you run the file, compile the executable and even launch it.

Windows has API (WinAPI) which is installed on every Windows. It includes GDI module, which is used for drawing. Although very primitive, it let you draw various objects. For example, there's an ellipse function, which draws a circle.

https://learn.microsoft.com/en-us/windows/win32/api/wingdi/nf-wingdi-ellipse

To draw a black circle with a red circle inside, you will have to call this function twice. Doesn't look hard to me. And will be much faster from pressing button to looking at the results than any JS. Or, if you're scared of WinAPI, or need more features, there are various drawing frameworks you can use.

2. Dynamic typing in JavaScript helps a lot but complicates the porting on limited memory device. For example, copy-pasting of something like a=b-c to C from JavaScript will create a bug if c>b and both defined as unsigned numbers (uint8_t).

Looks like JS doesn't actually make your life easier. There are periods where it may make things easier (such as when you write), but then it requires porting to C. In my view, the drawbacks outweigh the benefits. So, don't use JS then. You will have to port the existing code to C (if you haven't already), but if you stop using JS afterwards, it'll prevent future complications.
 
The following users thanked this post: Siwastaja, Georgy.Moshkin

Offline tellurium

  • Frequent Contributor
  • **
  • Posts: 288
  • Country: ua
Re: Cheap microcontroller for JavaScript or fast interpreter.
« Reply #24 on: December 29, 2024, 09:13:08 am »
Take a look at https://github.com/cesanta/elk
If you want an interpreter that works on Arduino Nano.
However, whilst tiny, it is slow and SEVERELY stripped-down.
Open source embedded network library https://github.com/cesanta/mongoose
TCP/IP stack + TLS1.3 + HTTP/WebSocket/MQTT in a single file
 
The following users thanked this post: Georgy.Moshkin


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf