Author Topic: i.MX RT1062: Getting Started, on a Raspberry Pi?  (Read 8095 times)

0 Members and 1 Guest are viewing this topic.

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
i.MX RT1062: Getting Started, on a Raspberry Pi?
« on: February 14, 2023, 09:57:56 pm »
It's actually a Teensy 4.1, but since it's essentially a breakout board for that chip (as far as I can tell, all Arduinos are, plus an easy programming interface), I thought I'd see if I can ditch the (somewhat opaque) Arduino environment and go direct to baremetal.

NXP has their own IDE, SDK, and related tools, but they're either online or for x86/64 only.  I'm on a Raspberry Pi 4 with the official RasPi OS.

The Raspberry Pi Pico has an AWESOME SDK and documentation, and makes it plumb easy both to make something work now, and work well, and to dive into the details.

8-bit PICs and AVRs are simple enough that I can start with the details and build from there.  My typical project with those is:
Code: [Select]
#include <avr.h>    //figures out from the project definitions, which chip-specific header to #include
                    //same idea for pic
void main()
{
    register = value;    //register matches a name in the header file, which in turn matches the datasheet
    register = value;
    //etc.

    while(1)
    {
        //non-blocking state machine for each logical function, to keep the loop running

        //lots of polling interrupt flags, and explicitly clearing them, because they don't actually need an *immediate* response
        //most things only need to be handled "before the next event", for which polling is plenty
    }
}

void isr()
{
    //only used if absolutely necessary, so that an immediate response is *actually* immediate
}

For both 8-bit projects and desktop applications, I've come to like Code::Blocks and GCC.  But I'm not finding much on how to set that up for an RT10xx.  Help?
 
The following users thanked this post: elecdonia

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6265
  • Country: fi
    • My home page and email address
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #1 on: February 14, 2023, 11:57:42 pm »
It's actually a Teensy 4.1, but since it's essentially a breakout board for that chip
Actually, it's not.  (I'm not nitpicking, just trying to help, as I read that as conflicting expectations of exactly what Teensy 4.1 is.)

Teensy 4.x have a proprietary NXP MKL02 bootloader chip, which is connected to the JTAG and SWD pins.  Basically, you can only program the i.MX RT1062 through the properietary PJRC bootloader, unless you desorder the MKL02 chip off the board.

Teensy LC, 3.x, 4.x, and MicroMod Teensys use Teensyduino, with the core sources at Paul Stoffregen's github repository (teensy4 tree).  The toolchain is a standard ARM GCC toolchain with newlib C library.  Teensyduino can be used on the command line, although it will still use Arduino machinery to do so.  Although the GUI version of the Teensy Loader is easiest to use, one can also use the open source teensy_loader_cli to upload non-encrypted firmware images to any Teensy.  So, even though you do need Arduino stuff installed, you can definitely write Teensy code in Code::Blocks (and you will be using GCC), using the Makefile example at the Teensyduino page.  (Name your sources with .c/.cpp/.h extension, and they won't be preprocessed using the Arduino preprocessor, like .ino files are, IIRC.)

Teensy cores are pretty darned good (as you can see from the generated code, if you look at the disassembly of the ELF object files that are linked into the final HEX firmware image), and since it uses GCC, you can use both C and C++ with it (with the standard embedded C++ limitations, i.e. no exceptions etc.).
However, it is not really suitable for bare-metal programming.  The i.MX RT1062 is pretty complicated for that anyway; even the Teensy core supports a relatively simple subset of its features.  Using Teensies/Teensyduino within your own design requires buying the preprogrammed MKL02 chips off PJRC (but Paul and Robin are good people (based on how they treated their employees and the community during the pandemic, for example), and Paul is very often on the PJRC forum discussing future plans, helping with technical issues, etc.), but they're cheap and PJRC definitely does not price gouge.

I personally have used Teensies for many years (as a hobbyist) starting with a Teensy 2.0++, and it is my favourite microcontroller by far.
But, because of the proprietary bootloader connected to the JTAG and SWD pins, it really isn't at all a "break-out board for i.MX RT1062".
However, it is an excellent platform to test and experiment with, even if you relatively quickly end up making your own board (with or without the PJRC bootloader chip).  (In particular, SparkFun MicroMod Teensy is such a derivative (with trademark agreements between PJRC and SparkFun for the "Teensy" use), with SparkFun getting the MKL02 chips from PJRC, but otherwise making their own Teensy-compatible boards.)

For true bare metal, you should look at NXP MIMXRT1060-EVKB, MIMXRT1064-EVK, MIMXRT1160-EVK, or MIMXRT1170-EVK, whichever best matches your intended processor.  They are a lot more expensive, of course, but it is a much more complete evaluation kit.

Others have started with Teensy 4, then manufactured their own boards based on what they learned and decided they needed; both with and without using Teensyduino support via the PJRC MKL02 bootloader chip.  The main issue is that the chips only come in BGA footprints, but there is some advice about that on the MKL02 bootloader chip page.  I suggest you go and read the related threads at the PJRC forum, Project Guidance and Technical Support & Questions sub-boards, looking for "custom teensy" and similar subjects.  We're not allowed to use "Teensy" when naming of our own custom boards and products, though, because that would be trademark dilution.
« Last Edit: February 15, 2023, 12:01:57 am by Nominal Animal »
 
The following users thanked this post: bingo600, oilburner, tellurium

Offline westfw

  • Super Contributor
  • ***
  • Posts: 4199
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #2 on: February 15, 2023, 02:10:33 am »
Quote
It's actually a Teensy 4.1 ... I thought I'd see if I can ditch the (somewhat opaque) Arduino environment and go direct to baremetal.

Most Arduino "cores" include a copy of the appropriate gcc, CMSIS, and newlib tools, in addition to the code that makes them "Arduino.'  You could just copy them to a more dignified-sounding directory, add them to your paths, or whatever.
 

Online luiHS

  • Frequent Contributor
  • **
  • Posts: 592
  • Country: es
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #3 on: February 15, 2023, 10:16:05 am »

You can't use a Teensy outside of the Arduino environment, because you don't have SWD access to be able to program it directly.

You are better off making your own board or purchasing an evaluation board from NXP.
 
The following users thanked this post: Nominal Animal

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6265
  • Country: fi
    • My home page and email address
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #4 on: February 15, 2023, 10:37:28 am »
You can't use a Teensy outside of the Arduino environment, because you don't have SWD access to be able to program it directly.

You are better off making your own board or purchasing an evaluation board from NXP.
luiHS, you only needed two sentences to say what I tried to convey in a page full of text.  Well said, and I fully concur.  :-+
 

Offline mac.6

  • Regular Contributor
  • *
  • Posts: 225
  • Country: fr
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #5 on: February 15, 2023, 12:49:41 pm »
Just use the header file from NXP SDK, they are monolitic and easy to integrate in your build.
Same with peripheral code generated by tools (at least clock and pinmux), you are not obliged to use them in a pre-generated build like on STM32cube.
Then you can roll your own driver code from this (or reuse NXP one).
But remember that RT family is quite a set-up from arduino or rpi pico, in fact they are much more MPU than MCU (especially the big ones).
 
The following users thanked this post: newbrain, elecdonia

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #6 on: February 16, 2023, 05:35:25 am »
It's actually a Teensy 4.1, but since it's essentially a breakout board for that chip
Actually, it's not.  (I'm not nitpicking, just trying to help, as I read that as conflicting expectations of exactly what Teensy 4.1 is.)

Teensy 4.x have a proprietary NXP MKL02 bootloader chip, which is connected to the JTAG and SWD pins.  Basically, you can only program the i.MX RT1062 through the properietary PJRC bootloader, unless you desorder the MKL02 chip off the board.

I guess they've gone much farther beyond my impression of them early on.  I got started with 8-bit PIC's in high school, using a third-party IDE and toolchain that I came to understand later was probably pirated, and the surrounding circuit design just soon enough to have gotten good at it when the first Arduino's came out.  I looked at what they were then, and concluded that they were just a UART bootloader with self-writable flash, and a USB <-> UART chip wired directly to the AVR.  Otherwise a breakout board.  I could do much better, smaller, and cheaper than that, and so I didn't need to concern myself with them.  If I did end up using one, I could set up a baremetal project for that chip that didn't know any different, and just run the HEX file through the standard bootloader.  Never got to test that theory.

I just assumed that the modern concept was still the same thing: that I could set up a baremetal project for that chip that doesn't know any different, and use the provided (now unbrickable) bootloader and corresponding host app to put the resulting HEX file into it.  No?

Teensy cores are pretty darned good (as you can see from the generated code, if you look at the disassembly of the ELF object files that are linked into the final HEX firmware image), and since it uses GCC, you can use both C and C++ with it (with the standard embedded C++ limitations, i.e. no exceptions etc.).
However, it is not really suitable for bare-metal programming.  The i.MX RT1062 is pretty complicated for that anyway; even the Teensy core supports a relatively simple subset of its features.

I personally have used Teensies for many years (as a hobbyist) starting with a Teensy 2.0++, and it is my favourite microcontroller by far.
But, because of the proprietary bootloader connected to the JTAG and SWD pins, it really isn't at all a "break-out board for i.MX RT1062".
However, it is an excellent platform to test and experiment with, even if you relatively quickly end up making your own board (with or without the PJRC bootloader chip).  (In particular, SparkFun MicroMod Teensy is such a derivative (with trademark agreements between PJRC and SparkFun for the "Teensy" use), with SparkFun getting the MKL02 chips from PJRC, but otherwise making their own Teensy-compatible boards.)

I was thinking to use the Teensy 4.1 to absorb what a 20MHz AVR is currently doing, in addition to live audio DSP.  My first attempt at audio DSP was on a RasPi Pico (RP2040), but it couldn't keep up with much more than a signal generator at 48kHz, even if I dedicated a CPU core to just that.  So I got the Teensy instead.
  • The AVR to absorb is a custom LED backlight controller for an LCD monitor with dead fluorescent tubes, that currently communicates with a Raspberry Pi 4 via UART for some automation.
  • For the audio, I want to receive stereo from the same Raspberry Pi, either I2S or USB UAC2, apply some processing (encoder-adjustable highpass, dynamic range compression, crossover filters, per-driver EQ and limiting, etc.), and send the result to some 24-bit DAC's on TDM.  Also transport some TDM ADC channels back to the Pi as a "microphone".
The DAC's do need to be 24-bit, not 16-bit like the Teensy Audio Library appears to be fixed at, because the main volume control is in the *middle* of the DSP chain, not the end, and I want it to have a lot of useful range.  So it can't just change the DACs' on-chip gain while continuing to send them full-scale data.  It really does send an all-zero MSB if you pull the main volume down enough, and it still needs to be decent quality at that low level, hence the longer words.  (full-scale is LOUD too, so an all-zero MSB might not be as quiet as you'd normally imagine, but still a ways down)

So it looks like I can't use the pre-fab and working Teensy Audio Library, because it's only 16 bits.  Which is fine, because I'm also using this and some other projects to build the tools and skills to make a bigger one, which will have a *very* tight maximum latency from analog in to analog out.  (so the converters' group delay is also important)  I'm wondering if I need to write my own just to guarantee that minimal latency, even if it's not as processor-efficient.

Or if someone has already created a library that has a 3-frame pipeline with local I/O - clock in, process, clock out - where a given sample really does go all the way through that quickly regardless of how long the processing chain is, and at least 24 bits for that local I/O (probably more internally), plus USB Audio (which is *not* included in the ultra-low-latency requirement), I'd be tempted to use that.
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14488
  • Country: fr
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #7 on: February 16, 2023, 06:27:15 am »
You can't use a Teensy outside of the Arduino environment, because you don't have SWD access to be able to program it directly.

Uh, yes you can.
I have a Teensy 4.1 and used it without Arduino whatsoever, using the NXP SDK.
You just need the "Teensy loader" to upload firmware. Comes with a GUI app and a CLI.
 
The following users thanked this post: newbrain, Nominal Animal

Offline Kalvin

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #8 on: February 16, 2023, 08:12:26 am »
It's actually a Teensy 4.1, but since it's essentially a breakout board for that chip
Actually, it's not.  (I'm not nitpicking, just trying to help, as I read that as conflicting expectations of exactly what Teensy 4.1 is.)

Teensy 4.x have a proprietary NXP MKL02 bootloader chip, which is connected to the JTAG and SWD pins.  Basically, you can only program the i.MX RT1062 through the properietary PJRC bootloader, unless you desorder the MKL02 chip off the board.

PJRC is protecting their IP, or what is the purpose of this MKL02 chip?
 

Offline westfw

  • Super Contributor
  • ***
  • Posts: 4199
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #9 on: February 16, 2023, 08:41:15 am »
Quote
PJRC is protecting their IP
PJRC is protecting their "boot IP."  It's actually really clever, giving you bootloader-like functionality (load over USB) with zero program memory footprint (or impact, like needing to change the start address) in the the target microcontroller.
 
The following users thanked this post: elecdonia

Offline josip

  • Regular Contributor
  • *
  • Posts: 152
  • Country: hr
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #10 on: February 16, 2023, 08:59:45 am »
I guess that MKL02 is not only for updating firmware. It is connected also to DCDC_PSWITCH and POR_B.

Is there any open (simple) schematic beside NXP boards and  Teensy? Adafruit Metro M7 NXP iMX RT1011 is not.

IMX RT is really great but too much pins are lost on power related stuff, and I dislike special power on / off sequence.
 

Online newbrain

  • Super Contributor
  • ***
  • Posts: 1719
  • Country: se
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #11 on: February 16, 2023, 09:12:28 am »
I guess that MKL02 is not only for updating firmware. It is connected also to DCDC_PSWITCH and POR_B.
Yes, it also handles the power on sequencing that is a bit complicated in iMX Rts.


You can't use a Teensy outside of the Arduino environment
Simply false.
I don't have any Arduino stuff on my computers, but happily use Teensy 4.1.
I don't even have MCUxpresso 🤮 installed, though I use parts of the SDK.
I use picolibc instead of newlib(nano) or redlib.

The only needed special bit of SW is the Teensy.exe programmer.

Yes, making SWD (or JTAG) inaccessible is unfortunate (unless you recur to extremely delicate surgery - and then only JTAG).
Even more unfortunate is that also the Debug Monitor mechanism is out of reach.

To help with debugging, and as a bit of a challenge, I have coded a gdb stub server.
It's very loosely inspired by https://github.com/ftrias/TeensyDebug, though I have used none of that code - since it must be integrated with the applications, the GPL license would spread to it.
It simulates breakpoints with SVC instructions, it's not 100% complete, mostly because the use case sequences that gdb uses are not 100% documented, but works quite well, allowing setting breakpoints and C/instruction level single stepping. Integration with FreeRTOS allows reading thread list and per thread stack traces and local variables -thread single stepping does not work, though.
Nandemo wa shiranai wa yo, shitteru koto dake.
 

Offline Kalvin

  • Super Contributor
  • ***
  • Posts: 2145
  • Country: fi
  • Embedded SW/HW.
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #12 on: February 16, 2023, 09:47:15 am »
Quote
PJRC is protecting their IP
PJRC is protecting their "boot IP."  It's actually really clever, giving you bootloader-like functionality (load over USB) with zero program memory footprint (or impact, like needing to change the start address) in the the target microcontroller.

Thanks. I would say that in that case their bootloader is pretty cosmetic in that regard because modern chips have quite a lot of memory and the bootloader itself is typically only few KB, changing the start address in the linker script is trivial, and ARM-processors can relocate their ISR vectors. Of course, using an external bootloader chip will make the board brick-proof, but reflashing the bootloader is not too difficult (see below).

Is there any practical way which could bypass this proprietary MKL02 bootloader-chip? For example, would it be possible to flash the i.MX RT1062 using OpenOCD, or those cheap USB<->SWD dongles that are used for programming the Bluepills?

An update came to this thread while writing this message:
I guess that MKL02 is not only for updating firmware. It is connected also to DCDC_PSWITCH and POR_B.
Yes, it also handles the power on sequencing that is a bit complicated in iMX Rts.

Ok, now I understand much better.
 

Offline mac.6

  • Regular Contributor
  • *
  • Posts: 225
  • Country: fr
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #13 on: February 16, 2023, 09:52:29 am »
Quote
PJRC is protecting their IP
PJRC is protecting their "boot IP."  It's actually really clever, giving you bootloader-like functionality (load over USB) with zero program memory footprint (or impact, like needing to change the start address) in the the target microcontroller.

Well i.MX got also a ROM bootloader and secure boot, so that's a little bit weird to keep their architecture, but from a user point of view it's understandable, you keep the same sw solution across the boards without too much hassle.
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6265
  • Country: fi
    • My home page and email address
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #14 on: February 16, 2023, 10:08:39 am »
You can't use a Teensy outside of the Arduino environment, because you don't have SWD access to be able to program it directly.
Uh, yes you can.
I have a Teensy 4.1 and used it without Arduino whatsoever, using the NXP SDK.
You just need the "Teensy loader" to upload firmware. Comes with a GUI app and a CLI.
:o

Did you need to include any code to support firmware updates, or does the MKL02 chip do it all via SWD/JTAG when you press the physical programming button?

This is funny as hell to me, because I never even considered doing that.  :-+
« Last Edit: February 16, 2023, 10:10:16 am by Nominal Animal »
 

Online newbrain

  • Super Contributor
  • ***
  • Posts: 1719
  • Country: se
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #15 on: February 16, 2023, 02:03:00 pm »
:o

Did you need to include any code to support firmware updates, or does the MKL02 chip do it all via SWD/JTAG when you press the physical programming button?

This is funny as hell to me, because I never even considered doing that.  :-+
No need for Teensy specific code, and it is not even really hard to pull out (as others and I said).

The MKL02 will do everything on its own: it loads an USB capable downloader/flasher in RAM using JTAG (not SWD, for unknown reasons) then runs it, waiting for the PC side application to send data (or a reset command).

Once the SW is on Teensy's flash, there's no difference (well, some fuses are hard set) with a naked iMX RT1062, you just can't debug it (but see my post above).

Since I despise both Arduino (it has its place, but not for my needs) and MCUxpresso, I use just clang + gdb + CMSIS + SDK + Cmake + ninja + VS Code.

I also have an RPi Pico which I use as a dual USB-serial bridge and to control the Program and NMI pins (makeshift reset pin, with suitable init code in startup and handler) so I don't need to push the button and can do everything from scripts.
Nandemo wa shiranai wa yo, shitteru koto dake.
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14488
  • Country: fr
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #16 on: February 16, 2023, 08:33:42 pm »
You can't use a Teensy outside of the Arduino environment, because you don't have SWD access to be able to program it directly.
Uh, yes you can.
I have a Teensy 4.1 and used it without Arduino whatsoever, using the NXP SDK.
You just need the "Teensy loader" to upload firmware. Comes with a GUI app and a CLI.
:o

Did you need to include any code to support firmware updates, or does the MKL02 chip do it all via SWD/JTAG when you press the physical programming button?

This is funny as hell to me, because I never even considered doing that.  :-+

Nothing at all to do.
There is a programming app for Windows, but also a CLI version that I use. Source code is provided (teensy_loader_cli.c) and can be built on pretty much any platform that has libusb.
It's not hidden - doc here: https://www.pjrc.com/teensy/loader_cli.html
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6265
  • Country: fi
    • My home page and email address
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #17 on: February 17, 2023, 09:58:53 am »
No need for Teensy specific code, and it is not even really hard to pull out (as others and I said).
Yup, I agree; it is just funny to me that I never thought about doing it that way, even though I started with a Teensy 2.0++ on bare metal with just PJRC-provided header files and avr-gcc (and avr-libc), basically.

When you get comfortable inside a box, you can easily forget you're inside that box, and forget to look; even for someone like myself who doesn't usually fall into that trap.

The MKL02 will do everything on its own: it loads an USB capable downloader/flasher in RAM using JTAG (not SWD, for unknown reasons) then runs it, waiting for the PC side application to send data (or a reset command).
Right, that's what I expected/hoped.  The protocol itself is quite simple, and uses USB control messages.  (It is called "HalfKay", named after from the original AVR Teensy bootloader that fit in 512 bytes.)

I also have an RPi Pico which I use as a dual USB-serial bridge and to control the Program and NMI pins (makeshift reset pin, with suitable init code in startup and handler) so I don't need to push the button and can do everything from scripts.
Wouldn't it have been easier to just add the reboot code to your custom firmware?

See teensy_loader_cli/rebootor/usb.c:ISR(USB_COM_vect): when you receive an USB control message with bmRequestType==0x21, bRequest==HID_SET_REPORT, wIndex=0, with a payload consisting of one or more sequences of 'reboot' (6 bytes, no separators), reboot your Teensy.

Of course, that only works if you have an USB HID endpoint, otherwise you do need to use external hardware.  For that, I'd have used an DigiSpark clone, an ATtiny85 running V-USB HID code that does that, both hanging off the same USB cable hub; they're even cheaper than RP2040.

Did you need to include any code to support firmware updates, or does the MKL02 chip do it all via SWD/JTAG when you press the physical programming button?
Nothing at all to do.
Right, as newbrain explained above, the MKL02 overwrites the RAM with the necessary loader code when one presses the button.  Nice!

There is a programming app for Windows, but also a CLI version that I use. Source code is provided (teensy_loader_cli.c) and can be built on pretty much any platform that has libusb.
Yup, I linked to it (https://www.pjrc.com/teensy/loader_cli.html) in my original reply, and that page has the link to the github repository, PaulStoffregen/teensy_loader_cli, where in other repositories you can find the Teensyduino cores and various Arduino libraries as well.

At the PJRC forums, Paul has explained that the only reason the MKL02 (and AVR Teensies' HalfKay bootloader, that uses the very same protocol) is proprietary, is to make it harder for Chinese manufacturers to clone the board (as happened to e.g. SparkFun's Pro Micro), and fund the development of the boards.  Similarly, while the schematics are shown, the board files are proprietary.  It is a very small company that treats its workers well (based on how they handled the pandemic) and interacts very well with its customers –– if you have a change to say the core that others benefit but does not break compatibility, Paul not only accepts such changes and patches, but is appreciative of it.  I know that from experience.  If you look at his GitHub repos, you'll find out that he's also the author for some of the Arduino libraries, published under very permissive licenses.   (I got a Teensy 4.1 with PSRAM, Ethernet kit, and extra Flash as a gift, just for helping others on the forums there, when Teensy 4.1 came out, but so did fifty or so others as well.  So, you could say I'm biased.)
I think for a development board in the Arduino environment, that's a very good balance between proprietary and libre/free-and-open-source.

Thus, I will amend my earlier posts about the subject, as Teensy 4.x can indeed be used for baremetal programming, with the only real drawback being JTAG/SWD disabled, and the benefit/quirk of the MKL02 handling the power-on sequencing (power rails enabled at the correct order and only when they reach the correct thresholds), which one would have to otherwise do in their own hardware.

To repeat, one can buy the preprogrammed MKL02 bootloader chips for USD $6.25 in singles (as of 2023-02-16), for ones own Teensy-compatible boards (but do not use the name "Teensy" in your product).  It also enables (but does not require) public-key based firmware verification/encryption.  The downside is that it currently only works with NXP IMXRT1062DV*6B or IMXRT1062DV*6A, and requires the Flash chip to be Winbond W25Q16JV*IM or W25Q64JV*IM; other chips may or may not work.
« Last Edit: February 17, 2023, 10:04:02 am by Nominal Animal »
 
The following users thanked this post: newbrain, elecdonia

Online newbrain

  • Super Contributor
  • ***
  • Posts: 1719
  • Country: se
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #18 on: February 17, 2023, 11:06:41 pm »
Quote from: Nominal Animal
Wouldn't it have been easier to just add the reboot code to your custom firmware?
Yes and no, as you say I'd need USB, but most of all, I'd need the FW not be locked up tight.

The use of NMI, with vector table and handler code in flash, makes sure* that bringing the pin high will invoke the handler and reset the MCU (and maybe save a stack trace in PSRAM or something like that).

And after all I wanted to be able to program the Teensy "hands free" (so e.g. from a remote location, I'm always going back and forth between SE and IT), so adding another GPIO was a no-brainer, and very little effort.

*not 100%, if code gets really crazy, it could modify the pin configuration. Quite a remote eventuality, though.

EtA, I forgot while answering:
Quote from: Nominal Animal, my edits
Paul[...] If you look at his GitHub repos, you'll find out that he's also the author for some of the Arduino libraries
Another reason I appreciate his work a lot: my eyes do not bleed when looking at his code, as opposed to 93.7% of Arduino related stuff.
« Last Edit: February 18, 2023, 01:23:23 pm by newbrain »
Nandemo wa shiranai wa yo, shitteru koto dake.
 
The following users thanked this post: elecdonia, Nominal Animal

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #19 on: February 18, 2023, 09:36:58 pm »
You can't use a Teensy outside of the Arduino environment, because you don't have SWD access to be able to program it directly.

Uh, yes you can.
I have a Teensy 4.1 and used it without Arduino whatsoever, using the NXP SDK.
You just need the "Teensy loader" to upload firmware. Comes with a GUI app and a CLI.

---

I have the GUI version.  Seems to work well enough on my RasPi.  I like that it has an "auto-download" feature, that is triggered when there's a chip available to program, and a new version of the HEX file that it hasn't downloaded yet.  So I can just leave it running in that mode and forget about it.  Debug build to check for syntax errors without much effort to optimize, Release build to actually run it.  I do it that way anyway for the 8-bit stuff, with non-blocking debug spew from the UART because while debuggers are nice, breakpoints in a real-time system are often problematic.  (and I first learned without a debugger anyway)

The CLI version could also be a post-build step.  Same effect without the GUI running.  Not sure which way I'll end up going on that.

---

I guess that MKL02 is not only for updating firmware. It is connected also to DCDC_PSWITCH and POR_B.
Yes, it also handles the power on sequencing that is a bit complicated in iMX Rts.

---

The MKL02 will do everything on its own: it loads an USB capable downloader/flasher in RAM using JTAG (not SWD, for unknown reasons) then runs it, waiting for the PC side application to send data (or a reset command).

Once the SW is on Teensy's flash, there's no difference (well, some fuses are hard set) with a naked iMX RT1062, you just can't debug it (but see my post above).

---

...Teensy 4.x can indeed be used for baremetal programming, with the only real drawback being JTAG/SWD disabled, and the benefit/quirk of the MKL02 handling the power-on sequencing (power rails enabled at the correct order and only when they reach the correct thresholds), which one would have to otherwise do in their own hardware.

To repeat, one can buy the preprogrammed MKL02 bootloader chips for USD $6.25 in singles (as of 2023-02-16), for ones own Teensy-compatible boards (but do not use the name "Teensy" in your product).  It also enables (but does not require) public-key based firmware verification/encryption.  The downside is that it currently only works with NXP IMXRT1062DV*6B or IMXRT1062DV*6A, and requires the Flash chip to be Winbond W25Q16JV*IM or W25Q64JV*IM; other chips may or may not work.

---

Ooo!  Looks like an easy way to make my own projects with a capable chip, and have it "just power on and run my code" with a single power rail (into that module) like an 8-bitter does.  AND make it "brick-proof"!  I think I like that!  Well worth an extra $6.25 per processor, and the limitation to use those chips and that starting-point schematic.  (for one-offs, at least, which is all I'm thinking about at the moment)

The required processor is BGA-only, which I haven't done before, but I *have* done a QFN by hand with hot air, and I've made my own controller for a toaster oven to reflow solder paste.  I'm a stickler for detail when prepping the board for that, and I've never had a chip failure yet.  A handful of tombstoned 0603's, but that's about it.  Based on that, a BGA *should* work?  I don't think I could replace it if it doesn't, but if I don't need to, then... :-//

If I do that for my custom boards, then both the toolchain and the code will be the same as for a real Teensy.  (just don't call it one)  So I think all that's missing is to get started on that collection of hardware.

---

To get started, I guess I need a fully functional example of a CLI build environment.  It's probably obvious once I see it, but I think I do need to see it.  Then I can copy/paste it into Code::Blocks like I did for the 8-bit and desktop projects.

I tried to find that on PJRC's site, but all of their examples are for 3.x or less.  Even their link that promises that for the 4.x, goes to the 3.x page instead.  Very different chip, different architecture, not likely to work.  Is the 4.x just too new?

What (well-documented) SDK and toolchain are you guys using for a non-Arduino IMXRT1062?  Preferably with the MKL02 "manager" involved too, but I don't think it makes a difference at that point?  (same HEX file either way)
 

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #20 on: February 21, 2023, 08:54:51 pm »
The Raspberry Pi Pico has an AWESOME SDK and documentation, and makes it plumb easy both to make something work now, and work well, and to dive into the details.
Today I plugged in my brand-new Raspberry Pi Pico board for the first time and connected it to the Arduino IDE. Wow - It has a huge number of options listed under the "board" tab. I can tell there's going to be plenty of things to explore.

So far (all I've done() is to load the generic Arduino "blink" sketch. This was painless and the sketch works as it should.

On my RPi Pico the "blink" sketch consumes 53k bytes flash, 50012 bytes RAM. This suggests the RPi Pico is on a whole different scale than AVR MCU chips.
« Last Edit: February 22, 2023, 12:51:02 am by elecdonia »
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #21 on: February 21, 2023, 10:04:57 pm »
This suggests the RPi Pico is on a whole different scale than AVR MCU chips.

Yes it is!  The major limits - Flash, RAM, and Clock - are pushed much farther out now, almost to where they feel infinite at first.  And with a native 32-bit core, big numbers and high-resolution processing are not a problem either.

You even get a second CPU in the same chip, in addition to the increased capability.  That can be both a blessing and a curse.  The blessing is that you can finally have *actual* parallel processing, instead of just single-threaded multitasking.  The curse is communication between the two physical threads, without clobbering stuff in ways that are mysterious at first.

Floating-point is still in software, so still relatively slow, even with the hand-optimized library in that chip's ROM that beats what GCC does.  If you really NEED floating-point, try to make it use that library, but also see if you can use fixed-point math instead.  (count a specific fraction instead of integers: the toolchain and the hardware still think it's integers, but you know different and write the code accordingly)



But with that all said, the i.MX RT chip that I'm looking at is about that much again more than the Pico.  Only a single CPU, but 600MHz instead of the Pico's 125MHz, and hardware floating-point!  That's enough to have a decent audio library, whereas the Pico can barely keep up with a single signal generator at 48kHz according to my experiments.

But that chip doesn't seem to have a well-designed SDK to go with it.  Arduino, yes, but the 'duino framework is too much of a "closed black box" for me.  For example, the loop() function is called from a user-inaccessible system loop.  What else does that system loop do?  How is it going to affect my loop time?  Is there some blocking I/O that's going to wreck the timing of something that I coded directly?  And how is it going to clash with my reading the datasheet and configuring X peripheral for my own use?

The Pico's SDK doesn't have that problem.  If you set up a project that way, *instead* of Arduino, then the SDK is documented and comprehensive enough that you can use it to just bare-metal configure everything.  Everything that you can do with each peripheral, has a sensible function call that covers the entire range of functionality, and an example/demo project that is known to work.  You can even follow the source code to see what it's actually doing, and do that yourself if you want, instead of calling the SDK function.  I did that to convert the SDK's blocking communication functions into my own non-blocking state machines.

NXP isn't there yet, as I've seen so far.  (if it exists, but not obvious and easy to get, then that statement is still true)  Too focused on their own in-house IDE and code generator, which used to be how everyone did it.  Not anymore, I hope, with the Pico paving the way.
 
The following users thanked this post: elecdonia

Offline westfw

  • Super Contributor
  • ***
  • Posts: 4199
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #22 on: February 21, 2023, 10:30:54 pm »
Quote
the 'duino framework is too much of a "closed black box" for me.  For example, the loop() function is called from a user-inaccessible system loop.


It's not "closed" or "inaccessible."  All open source, installed as source on your computer when you add the core.
Admittedly some combination of Windows and Arduino policies can make it a bit hard to find, but in a standard windows install (using IDE 2.x), it'll end up in "C:\Users\USERNAME\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.57.2\cores\teensy4\"


It even looks like there might be instructions somewhere on using it with a Makefile and bypassing all of the Arduino code entirely...


Code: [Select]
extern "C" int main(void)
{
#ifdef USING_MAKEFILE


// To use Teensy 4.0 without Arduino, simply put your code here.
// For example:


pinMode(13, OUTPUT);
while (1) {
digitalWriteFast(13, HIGH);
delay(500);
digitalWriteFast(13, LOW);
delay(500);
}




#else
// Arduino's main() function just calls setup() and loop()....
setup();
while (1) {
loop();
yield();
}
#endif
}

 
The following users thanked this post: AaronD

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #23 on: February 22, 2023, 10:04:29 pm »
With very little effort I succeeded at installing Micro Python on my Raspberry Pi Pico board. It is connected to the Thonny Python IDE which was also very easy to install on my Win 10 machine.

I've been wanting to get more familiar with Python and this seems to be a convenient way to combine two of my interests:
Fast/powerful/inexpensive small computing modules with I//O pins and Python
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #24 on: February 23, 2023, 02:11:21 am »
It's not "closed" or "inaccessible."  All open source, installed as source on your computer when you add the core.
Admittedly some combination of Windows and Arduino policies can make it a bit hard to find, but in a standard windows install (using IDE 2.x), it'll end up in "C:\Users\USERNAME\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.57.2\cores\teensy4\"

It even looks like there might be instructions somewhere on using it with a Makefile and bypassing all of the Arduino code entirely...
Code: [Select]
extern "C" int main(void)
{
#ifdef USING_MAKEFILE


// To use Teensy 4.0 without Arduino, simply put your code here.
// For example:


pinMode(13, OUTPUT);
while (1) {
digitalWriteFast(13, HIGH);
delay(500);
digitalWriteFast(13, LOW);
delay(500);
}




#else
// Arduino's main() function just calls setup() and loop()....
setup();
while (1) {
loop();
yield();
}
#endif
}
I'll have to look at that.  Per the title, I'm on a Raspberry Pi, not Windows, but I'd imagine the folder structure to be similar, starting from the user's home.

With very little effort I succeeded at installing Micro Python on my Raspberry Pi Pico board. It is connected to the Thonny Python IDE which was also very easy to install on my Win 10 machine.

I've been wanting to get more familiar with Python and this seems to be a convenient way to combine two of my interests:
Fast/powerful/inexpensive small computing modules with I//O pins and Python
That's great!  But not really the topic of this thread.  I gave you a pass on the first one, and tried to bring the conversation back on topic.  If you wouldn't mind to start your own thread for that, it would keep the clutter down here.  Thanks!
 

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #25 on: February 23, 2023, 03:03:59 pm »
With very little effort I succeeded at installing Micro Python on my Raspberry Pi Pico board…
That's great!  But not really the topic of this thread.  I gave you a pass on the first one, and tried to bring the conversation back on topic.  If you wouldn't mind to start your own thread for that, it would keep the clutter down here.  Thanks!
Understood.  After re-reading this topic I realized I missed this point from your initial post:
Quote
NXP has their own IDE, SDK, and related tools, but they're either online or for x86/64 only.  I'm on a Raspberry Pi 4 with the official RasPi OS.
I incorrectly thought you were comparing target boards: Teensy4.1 vs. RPi Pico. But… This topic is about discovering useful RT1062 development tools which run under RasPi OS. I’m a bit surprised that official NXP development tools aren’t supported on RasPi OS. Are those NXP tools for Windows 10 only? Nothing for any flavor of Linux?

I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #26 on: February 23, 2023, 05:32:03 pm »
I’m a bit surprised that official NXP development tools aren’t supported on RasPi OS. Are those NXP tools for Windows 10 only? Nothing for any flavor of Linux?
That's how it appears to me, but westfw has a good point that I still have to check.  I might be able to use parts out of the Arduino framework, which does run on Linux, and the Pi, to make my own toolchain.
 

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #27 on: February 23, 2023, 05:40:46 pm »
...the i.MX RT1062 chip that I'm looking at is about that much again more than the Pico.  Only a single CPU, but 600MHz instead of the Pico's 125MHz, and hardware floating-point!  That's enough to have a decent audio library, whereas the Pico can barely keep up with a single signal generator at 48kHz according to my experiments.
I agree that Teensy4.1 is a better choice for audio applications compared to the the RPi Pico.

Do you have any direct experience with previous PJRC products, especially Teensy 3.5/3.6, the PJRC audio interface board, and best of all, the Teensy Audio Library?

Several years ago when the Teensy3.5 was first introduced I bought one and its companion audio interface board, having previously been very satisfied using AVR-based Teensy boards and the Teensy3.1. However the Teensy3.5 was a whole new experience for me. I was blown away by its audio capabilities along with the features of the superb PJRC audio library.

Today PJRC strongly recommends advancing to the Teensy4.1. They warn that their older boards may never be manufactured again (due to chip-a-geddon).
     https://www.pjrc.com/store/teensy41.html
     https://www.pjrc.com/store/teensy3_audio.html
     https://www.pjrc.com/teensy/td_libs_Audio.html
After finishing this post I'm ordering a Teensy4.1 along with its matching audio board. (as of this moment) both are in stock at the PJRC store.
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #28 on: February 23, 2023, 06:30:54 pm »
Example of an impressive Teensy-based audio application: This 6-voice polyphonic synthesizer was implemented with Teensy3.5 (120 MHz Cortex-M4F) and the PJRC audio library:
     https://www.pjrc.com/six-voice-polyphonic-synth/
Direct link to builder's Youtube demo video:
     
Builder's GitHub project:
     https://github.com/albnys/TeensyPoly6

In contrast the Teensy4.1 is a 600 MHz Cortex-M7. Just imagine how much more it can do! That said, I think the best reason to use the Teensy4.1 for audio processing is the PJRC audio library.  Whether or not you enjoy using the Arduino IDE, I believe it is the most direct route to creating a functional audio project containing the Teensy4.1

Another example: The same builder recently made a drum machine with Teensy4.1.  His GitHub project shows how the PJRC audio library is used:
     https://github.com/albnys/Drum-Machine
Demo of the finished drum machine:
     
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #29 on: February 23, 2023, 07:14:38 pm »
Do you have any direct experience with previous PJRC products, especially Teensy 3.5/3.6, the PJRC audio interface board, and best of all, the Teensy Audio Library?
I don't.  This is the first for me.

As for the Teensy Audio Library, it's fixed at 16-bit 44.1kHz, which is technically CD quality if you ignore the accumulated round-off error.  Pro gear pretty much always uses 24-bit converters now, so that the digital version is always better than its analog source.  (the bottom few bits are always below the analog noise floor: it's difficult, but possible to make a dedicated analog circuit out-perform a 16-bit converter, but not a 24-bit one)  And they up-convert from there to do all the processing, so that the accumulated round-off error isn't enough to make a difference at the 24-bit output converter.

For example, a popular cheap digital mixer, either as a console in back or a "smart stage box", uses 40-bit floating-point internally, between its 24-bit converters to and from the analog world.  Larger consoles that do more, and pretty much every DAW (Digital Audio Workstation - essentially an audio console that runs in software on a PC, with all of the extra features that that can offer) use 64-bit floating-point or better, again to keep the accumulated round-off error small enough to not matter.

If you keep the signal fairly "hot", then the low bit-depth doesn't matter as much, but when you turn it down a lot, it does.  Especially when you use the digital processing as the master volume control, feeding a fixed high-gain amplifier, as the pro world does and I've come to like because of what it allows you to do.

So I need more than 16 bits all around - 24-bit converters and at least 32-bit processing - which means I can't use that library.  Probably not that board either, as I would expect it to have 16-bit converters on it to match the 16-bit library.  It's good to play with, and learn about audio processing without having to dive into the weird unintuitive world of DSP code, but for me to call it a serious replacement for a DIY analog thing would require 24-bit I/O and 32-bit processing as an absolute minimum.

Soo...it looks like I *am* going to dive into the weird unintuitive world of DSP code.  ;D  Unless I can find a different library with a permissive license that doesn't have a prohibitively large buffer.  (big buffers can make things more efficient on systems that can do single-instruction-multiple-data (SIMD), or approximate it, but they also add a ton of latency equal to the buffer size)



If you're using it to make a synthesizer - melodic, percussive, whatever - then it probably doesn't matter as much.  The signal stays fairly "hot", as the master volume control is outside the scope of that project, and whatever non-idealities it still has, become part of its "character" as an instrument.

If you're using it as I am, to make a transparent processor and system manager, it needs to be better than that.
« Last Edit: February 23, 2023, 07:23:48 pm by AaronD »
 
The following users thanked this post: elecdonia

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14488
  • Country: fr
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #30 on: February 23, 2023, 08:01:16 pm »
...the i.MX RT1062 chip that I'm looking at is about that much again more than the Pico.  Only a single CPU, but 600MHz instead of the Pico's 125MHz, and hardware floating-point!  That's enough to have a decent audio library, whereas the Pico can barely keep up with a single signal generator at 48kHz according to my experiments.
I agree that Teensy4.1 is a better choice for audio applications compared to the the RPi Pico.

No kidding! It's a world apart.

But of course, it depends on what you want to do. If you stick to fixed-point, there's still a lot of cool stuff you can do with the RP2040, especially with its 2 cores, it's overclockability AND with the hardware blending peripheral, which can be used to accelerate interpolation and a lot of other things.

 
The following users thanked this post: elecdonia

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #31 on: February 23, 2023, 08:09:45 pm »
Yes, if your project is in the pro-sound or studio-grade realm you may wish to use higher resolution than 16 bits. But I doubt there are any high-resolution audio libraries with the capability of the PJRC 16-bit package which don't cost major $. I recommend evaluating and testing the PJRC items because the cost is negligible. The total invoice for the Teensy4.1 board, companion audio adaptor board, and shipping which I ordered today from PJRC was $53.63 USD. And the PJRC 16-bit audio library itself is free.
A vital  question is this:
     "What resolution does the PJRC audio library use internally, inside its functional blocks?

I don't yet have the answer to this question. However, years ago I learned when and where to use "long" variables in the code I write for 8-bit hardware like the AVR. It is certainly possible that functional blocks inside the PJRC audio library are implemented with far more than 16-bit resolution.

Do you feel that audio streams between hardware devices must have greater than 16-bit resolution?
 (For example an audio stream which comes from an RPi Pico and then flows into the Teensy4.1)

 
« Last Edit: February 24, 2023, 12:18:55 am by elecdonia »
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14488
  • Country: fr
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #32 on: February 23, 2023, 08:24:36 pm »
You can do 24-bit stuff with the RP2040 or the RT1062 alike. I was of course not talking about using any existing library. Write your own stuff.

I wrote a small synth with a RP2040 using fixed-point, hardware blending, the PIO for generating 24-bit I2S, and USB MIDI (using TinyUSB.)
Perfectly doable.
 
The following users thanked this post: elecdonia

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #33 on: February 23, 2023, 08:39:36 pm »
If you stick to fixed-point, there's still a lot of cool stuff you can do with the RP2040, especially with its 2 cores, it's overclockability AND with the hardware blending peripheral, which can be used to accelerate interpolation and a lot of other things.
The audio CD format is 2 channels of "signed 16-bit integer" audio. Although I enjoyed listening to Sony's "SACD" disc format in the early 2000's, it wasn't a commercial success. My favorite SACD discs were mixed for 4 discrete audio channels. They sound amazing! 

Personally I don't recall writing much code which required me to use floating point variables. I frequently use "long" variables (usually signed) in my code. But float? Not so much.

A major question for the OP is to learn what resolution is used inside the "16-bit" PJRC audio library?

Today I ordered a Teensy4.1 and its companion audio interface board from PJRC. (They are currently both in-stock!). I previously used the Teensy3.5, its  audio board, and an earlier version of the PJRC audio library: The results sounded absolutely terrific to me. The ease of designing audio processes with the PJRC audio library is amazing. The next step above that in my experience is a package like Native Instruments Reaktor, combined with a full-scale professional-grade DAW:
     https://www.vintagesynth.com/misc/reaktor.php
Of course this requires a high-performance PC and Windows 10. In fact most laptops aren't good enough. Reaktor has a visible gauge which shows its real-time resource consumption: I've seen 100% utilization more than occasionally.
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #34 on: February 23, 2023, 08:49:32 pm »
You can do 24-bit stuff with the RP2040 or the RT1062 alike. I was of course not talking about using any existing library. Write your own stuff. I wrote a small synth with a RP2040 using fixed-point, hardware blending, the PIO for generating 24-bit I2S, and USB MIDI (using TinyUSB.) Perfectly doable.
Is it available anywhere? The large community of starving musicians and makers certainly do need audio libraries which can operate on small hardware.
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #35 on: February 23, 2023, 09:03:46 pm »
Quote
If you keep the signal fairly "hot", then the low bit-depth doesn't matter as much, but when you turn it down a lot, it does.  Especially when you use the digital processing as the master volume control, feeding a fixed high-gain amplifier, as the pro world does and I've come to like because of what it allows you to do.
In old-school (analog) recording studios (and live-performance sound reinforcement) the audio levels for signal-processors (eq, reverb, compression, filters, noise gates) were carefully adjusted to (usually) the highest levels which didn't clip. The "Dolby" and "DBX" analog tape noise suppression systems compressed the audio before recording, then expanded it inversely on playback.
The relative position of faders (eg volume controls) in the audio chain always ended with the "master volume control." Prior to that point all levels were kept as high as possible "below clipping."
 
Can the OP's project be arranged so that "master volume" is applied in the final piece of hardware just prior to the power amps and speakers?
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14488
  • Country: fr
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #36 on: February 23, 2023, 09:22:45 pm »
You can do 24-bit stuff with the RP2040 or the RT1062 alike. I was of course not talking about using any existing library. Write your own stuff. I wrote a small synth with a RP2040 using fixed-point, hardware blending, the PIO for generating 24-bit I2S, and USB MIDI (using TinyUSB.) Perfectly doable.
Is it available anywhere? The large community of starving musicians and makers certainly do need audio libraries which can operate on small hardware.

It's not, but others have done synths too on the RP2040:


What I can share here is the PIO program for transmitting 64-bit/frame I2S (which you can use for transmitting anything from 16-bit to 32-bit samples, stereo), will work on any modern I2S DAC that takes 64-bit/frame, which is pretty much the norm now. This PIO program should be preferably used with DMA. For anyone that could find this useful.

Code: [Select]
;**         PIO assembler code for 32-bit I2S (64 bits/frame)

; Out pins:
;   0: SDATA (data out)

; Side-set pins:
;   0: BCLK (bit clock)
;   1: LRCLK (word select)

; Blocks on TX FIFO empty - the stream will stop if it's empty.

.program i2s_32_tx
.side_set 2

set x, 30           side 0b00

.wrap_target

left_word:
out pins, 1         side 0b00
jmp x-- left_word   side 0b01

out pins, 1         side 0b10  ; LSB
set x, 30           side 0b11

right_word:
out pins, 1         side 0b10
jmp x-- right_word  side 0b11

out pins, 1         side 0b00  ; LSB
set x, 30           side 0b01

.wrap

% c-sdk {

static inline void i2s_32_tx_program_init(PIO pio, uint sm, uint offset, uint SDATA_pin, uint BCLK_pin, float clk_div)
{
pio_gpio_init(pio, SDATA_pin);
pio_gpio_init(pio, BCLK_pin);
pio_gpio_init(pio, BCLK_pin + 1);

pio_sm_set_consecutive_pindirs(pio, sm, SDATA_pin, 1, true);
pio_sm_set_consecutive_pindirs(pio, sm, BCLK_pin, 2, true);

pio_sm_config c = i2s_32_tx_program_get_default_config(offset);

sm_config_set_out_pins(&c, SDATA_pin, 1);
sm_config_set_sideset_pins(&c, BCLK_pin);

sm_config_set_fifo_join(&c, PIO_FIFO_JOIN_TX);

sm_config_set_clkdiv(&c, clk_div);
sm_config_set_out_shift(&c, false, true, 32);

pio_sm_init(pio, sm, offset, &c);
pio_sm_set_enabled(pio, sm, true);
}

%}

When calling i2s_32_tx_program_init() to initialize it, clk_div should be set to the divide factor that gets you twice the I2S bit clock frequency from the SYS clock.
 
The following users thanked this post: elecdonia

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #37 on: February 23, 2023, 09:26:27 pm »
A major question for the OP is to learn what resolution is used inside the "16-bit" PJRC audio library?
I would think it'd be a major selling point if it was different, so because it doesn't say, I'm pretty sure the entire chain is 16-bit 44.1kHz.  Again, that's okay for small stuff, or just messing around, as long as you keep the signal level up.  Standalone single-module type stuff is probably fine, with an *external* volume control.  You might use it to feed a PA, for example, where the sound guy has a pro rig.  Your module has its volume all the way up, and the sound guy controls it from there.

But as soon as you start making complex processing chains with a LOT of things in series to all contribute their round-off error, and all of that needs to be "transparent" - especially if you want to include the master volume in the digital world and have some *more* processing after that and *then* feed the DAC - you really need more.

In old-school (analog) recording studios (and live-performance sound reinforcement) the audio levels for signal-processors (eq, reverb, compression, filters, noise gates) were carefully adjusted to (usually) the highest levels which didn't clip. The "Dolby" and "DBX" analog tape noise suppression systems compressed the audio before recording, then expanded it inversely on playback.
The relative position of faders (eg volume controls) in the audio chain always ended with the "master volume control." Prior to that point all levels were kept as high as possible "below clipping."
 
Can the OP's project be arranged so that "master volume" is applied in the final piece of hardware just prior to the power amps and speakers?
No.

I'm aware of all those tricks to get better quality from a recording medium that only afforded maybe 8 to 10 bits of useful dynamic range between clipping and noise floor, and I still enjoy running a pure-analog live sound rig that probably has around 14 bits or so through all of its circuitry.  (you don't normally use bits to describe analog; I'm just making the comparison)

For another example of getting better quality out of a physical medium, the RIAA curve for vinyl records essentially has tiny bass so that the stylus doesn't fly all over the surface to pick it up, and massive sibilance so that it drowns out the surface noise.  When the playback system corrects it back, it gets the bass back to (roughly) where it's supposed to be, and effectively turns down the surface noise.

But for rearranging the master volume to be all the way at the end of the chain and then making it analog, yes there are VCA's that can be controlled from a DAC, and a lot of DAC's actually have that functionality built into them anyway.  But consider how that would work with a multi-way crossover that is followed by per-driver correction and limiting.  The corrective EQ's don't care whether the volume control is before them or after, and neither do the crossover filters.  So far, the master volume could be changed from a stereo-gang before the crossover to a many-gang after the correction, but the limiters DO care.  Technically, their thresholds can be adjusted to track a gain control that follows them, so that the *output* of that gain control is never above X, but it's a whole lot easier and more intuitive (and thus more maintainable) to have the limiter literally last.  And now, because the final gain is not last, and therefore *must* be in the digital world, and because linear processing (EQ's, filters, and gains) don't care about order, it becomes easier to put the final volume control before the crossover as a stereo-gang instead of many.

Of course the consequence of this, is that the DAC now receives maybe -30dBFS most of the time, instead of -1 or so, and the amplifier is fixed at 30dB more.  Essentially, the amp is calibrated to the DAC, so that full-scale = full-scale, and never touched again.  Because the DAC receives such a low signal, it needs enough bits that it still works well.
 
The following users thanked this post: elecdonia

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #38 on: February 24, 2023, 12:14:52 am »
I wasn't thinking of using anything analog at all. My suggestion is simply that the digital function block controlling master volume would be located towards the output end of the chain, rather than early in it.

Some questions:
   Do a majority of today's sound reinforcement power amplifiers have digital audio inputs?
   For flying modular main speaker arrays: Are the power amplifiers sometimes embedded inside the speaker cabinets?
   Does the signal chain ever need to be analog at all?

(I'm asking because most of my direct experience with large sound reinforcement systems is "out of date" to say the least. My career in that field was in the 1970's and 1980's. Among other things I designed several 3-way and 4-way electronic crossovers and a limiter/protection system which reduced driver failures, especially for drivers on high frequency horns. These items, of course, were all 100% analog)

I understand the desire for maintaining 24-bit resolution past the master volume control when a considerable amount of digital audio processing must happen after the master volume. Examples of such "downstream" digital processing:
    -Digital crossover frequency low//high pass filters for the speaker cabinets
                (subwoofer-mains, additionally low-mid-high crossover filters for the main speakers if bi-amped or tri-amped)
    -Anti-clipping limiter functions
    -Thermal protection for high frequency drivers (using a simulation of their long-term power handling limits)
    -Adjustable time-alignment delay (10 or 100usec steps), especially for the relationship between mid/high frequency drivers.

To summarize:
  OP application probably does require >16-bit resolution
  That said, for early prototyping/experimentation/testing, the PJRC 16-bit audio library might still be a useful "starting point."  It already contains ready-to-use function blocks for much of what is required to make a high grade sound reinforcement system. 

   
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #39 on: February 24, 2023, 12:26:45 am »
Quote
For example, a popular cheap digital mixer, either as a console in back or a "smart stage box", uses 40-bit floating-point internally, between its 24-bit converters to and from the analog world.  Larger consoles that do more, and pretty much every DAW (Digital Audio Workstation - essentially an audio console that runs in software on a PC, with all of the extra features that that can offer) use 64-bit floating-point or better, again to keep the accumulated round-off error small enough to not matter.
There must be some libraries around for doing this level of high-resolution audio processing if the end-products are reaching the "affordable" range.
For the past 25 years the specialized TI DSP chips reigned, but it seems to me that many general-purpose MCU chips have caught up. One problem with TI is that they wouldn't even talk to prospective customers who didn't promise to order millions of their DSP chips.
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #40 on: February 24, 2023, 12:28:50 am »
You know, I never expected to have an intense discussion of high-performance digital audio processing in a forum dedicated to microcontrollers priced under $1 USD each.
My, my, how things have changed over time...
« Last Edit: February 24, 2023, 02:34:42 am by elecdonia »
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Online langwadt

  • Super Contributor
  • ***
  • Posts: 4428
  • Country: dk
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #41 on: February 24, 2023, 01:22:55 am »
For example, a popular cheap digital mixer, either as a console in back or a "smart stage box", uses 40-bit floating-point internally, between its 24-bit converters to and from the analog world.  Larger consoles that do more, and pretty much every DAW (Digital Audio Workstation - essentially an audio console that runs in software on a PC, with all of the extra features that that can offer) use 64-bit floating-point or better, again to keep the accumulated round-off error small enough to not matter.

I'm sure that is also a matter of convenience and what is available. 32bit fixed point is 150dB, that would probably go a long way but with fixed point you need to think about scaling all the time, 32bit floats are not enough (only ~24bit), lots of HW has 64 bit doubles or more so that is convenient and available



 

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #42 on: February 24, 2023, 01:56:00 am »
It's not "closed" or "inaccessible."  All open source, installed as source on your computer when you add the core.
Admittedly some combination of Windows and Arduino policies can make it a bit hard to find, but in a standard windows install (using IDE 2.x), it'll end up in "C:\Users\USERNAME\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.57.2\cores\teensy4\"

It even looks like there might be instructions somewhere on using it with a Makefile and bypassing all of the Arduino code entirely...
Code: [Select]
extern "C" int main(void)
{
#ifdef USING_MAKEFILE


// To use Teensy 4.0 without Arduino, simply put your code here.
// For example:


pinMode(13, OUTPUT);
while (1) {
digitalWriteFast(13, HIGH);
delay(500);
digitalWriteFast(13, LOW);
delay(500);
}




#else
// Arduino's main() function just calls setup() and loop()....
setup();
while (1) {
loop();
yield();
}
#endif
}
I'll have to look at that.  Per the title, I'm on a Raspberry Pi, not Windows, but I'd imagine the folder structure to be similar, starting from the user's home.
On my Pi, it's "/home/USERNAME/.arduino15/packages/teensy/hardware/avr/1.57.2/cores/teensy4"
A little bit different after the user's home directory, but not by much.  And there appears to be a complete project in there, ready for the one "add your code here" file that the Arduino IDE shows you.

It's a bit odd that it still uses "avr" as a folder, when there's no AVR involved here, but I guess that just shows the Arduino's heritage.  Maybe that level was supposed to organize by architecture, but ended up being kept for everything?  Anyway, I think it'll work.

Of course, I had to install the board first, to get any of that:
https://www.pjrc.com/teensy/td_download.html
 

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #43 on: February 24, 2023, 02:30:47 am »
Human hearing is logarithmic rather than linear.

In the very first days of digital audio, well before ADC or DAC devices had >12 bits, there was a remarkable yet simple hardware design called “continuously variable delta slope modulation” which sampled the analog audio at a very high rate (>50kHz), producing a single bit signifying whether the input voltage was rising or falling. Delay was generated by sending this stream of single bits through a bank of early dynamic RAM chips, often 1b by 16k, 64k, 256k memory cells per chip. The analog signal was recovered by a simple opamp integrator. I remember being quite impressed with the audio quality. It wasn’t noisy or grainy. It even measured well: THD+noise <0.5%

To summarize, a lot can be accomplished in the digital audio domain by having a thorough understanding of the underlying math. Forcing more bits through the hardware is also an effective brute-force approach. But it isn’t the only method which works.
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #44 on: February 24, 2023, 05:00:42 am »
Some questions:
   Do a majority of today's sound reinforcement power amplifiers have digital audio inputs?
   For flying modular main speaker arrays: Are the power amplifiers sometimes embedded inside the speaker cabinets?
   Does the signal chain ever need to be analog at all?
Starting with the second question: Yes.  They're called "powered speakers" or "active speakers", and they're been around for a long time now.

Originally, it was just a comparable-sized amp shoved into the same cabinet with everything else the same.  Single amp channel driving a passive crossover, and it would blow tweeters just as well as a passive rig if you turned it way up and walked away.  One venue that I worked at had a set of those, and a guy that was very good at replacing the tweeters!  I was part of upgrading that rig to digital, with the requirement that "just anybody" should still be able to use it, so I put a "postage stamp" of an analog board in there for them, which fed into the main digital one that had limiters on that feed!  The controls for the digital were kept in a locked closet, to be brought out and used only by the more skilled people, who load the defaults back in when they're done.

Now though, the line-level input feeds an ADC into a DSP with only a handful of user-accessible settings, and that DSP drives a dedicated amp channel for each driver.  This allows per-driver correction and protection, and relaxes some of the acoustic design requirements, as the things that are easily corrected electronically don't have to be accounted for anymore when building the box.  So the acoustic design can focus only on what the electronics still can't do very well, and the combination produces the most accurate box possible in terms of converting a small electrical signal into far-field air pressure.  (within the constraints, of course, that a mass market puts on the overall architecture - the classic over/under woofer/tweeter box is not all that good, but it's what people are used to and therefore want)

Most powered speakers (that are sold to the general public) also have multiple inputs, with XLR/TRS combo jacks and a mic preamp for at least one of them.  If you only have a single mic, then you don't even need a mixer: just plug the mic directly into the powered speaker and use the mic/line switch to turn the preamp on.  (I have yet to see one with phantom power, so dynamic mics only)

For the first question, some amps do have digital inputs, even networking (Dante, for one example), so that the only analog in the system at all might be the mics and the amps' power stages (Class-D is still very much analog, despite the on/off output devices), but pretty much all of them still have analog inputs available.  The ones that have DSP built-in have an ADC following the attenuator; otherwise it feeds into the amp itself as usual, even for Class-D.

For the third question, analog is a very effective and easy way to resample.  Have a DAC feed an ADC, with each controlled by a different system with its own clock, and it "just works".  Digital resampling can be enough of a hassle to just hang the whole thing and do that instead.  If one system wants to change its sample rate, then it can tell its converter to do that, and the other doesn't have to know or care.  So in otherwise all-digital rigs, there may still be some analog links in a few places.  DAC's to amps, for one example.
Of course, *anything* in the analog world is subject to the analog rules, including headroom and noise pickup, so it still has to be done *right*.

I understand the desire for maintaining 24-bit resolution past the master volume control when a considerable amount of digital audio processing must happen after the master volume. Examples of such "downstream" digital processing:
    -Digital crossover frequency low//high pass filters for the speaker cabinets
                (subwoofer-mains, additionally low-mid-high crossover filters for the main speakers if bi-amped or tri-amped)
    -Anti-clipping limiter functions
    -Thermal protection for high frequency drivers (using a simulation of their long-term power handling limits)
    -Adjustable time-alignment delay (10 or 100usec steps), especially for the relationship between mid/high frequency drivers.
Yes to all of the above, except that the thermal protection could easily be measured instead of simulated.  Just put a thermal probe in there, connected to the DSP logic...
And for that matter, I've also seen accelerometers on the speaker cone...

To summarize:
  OP application probably does require >16-bit resolution
  That said, for early prototyping/experimentation/testing, the PJRC 16-bit audio library might still be a useful "starting point."  It already contains ready-to-use function blocks for much of what is required to make a high grade sound reinforcement system. 
Pretty much, yes.  Though I've done enough research with the idea of rolling my own, that I think I'm ready to just do that instead of learning a library (even if it barely takes anything to learn) and then ditching it because I need more.
https://dspguru.com/
https://webaudio.github.io/Audio-EQ-Cookbook/audio-eq-cookbook.html
https://www.musicdsp.org/en/latest/Filters/64-biquad-c-code.html
https://www.musicdsp.org/en/latest/Analysis/186-frequency-response-from-biquad-coefficients.html (to read back what it's actually doing)
https://www.earlevel.com/main/2012/11/26/biquad-c-source-code/
https://www.earlevel.com/main/2003/03/02/the-digital-state-variable-filter/
Etc.
That's just for frequency filters.  Similar for dynamics and other things.

And I've done enough live sound that I know what the "black box" modules do anyway.  No need to learn that; just how to make them in software.  Or find a suitably-licensed library that is also suitable for my end use.

Human hearing is logarithmic rather than linear.
Sorta.  Perceived amplitude is logarithmic, which is probably what you're referring to.  But the process itself is quite the exercise in Linear Systems.  It's not a single microphone per ear, but a series of acoustic bandpasses followed by peak detecting nerves.  A "biological RTA", if you will.  Before all of that is an acoustic/mechanical gain element (with a non-flat frequency response if you pay attention) that is controlled by the average of that RTA.  A "feedback compressor".  Older ears lose the ability to gain-down, which is probably why they complain more about volume.  It really IS louder for them, and painful, and at risk of losing more sensitivity, while the young kid next to them is perfectly fine.  (if your grandpa took you to a concert, he probably sacrificed more than you realized to make your day)

Also, because of that system architecture, the risk of pain and hearing loss depends on the mix and is not a single number.  A narrow, prominent range of frequencies, like a screaming guitar amp that shoots past the musician's knees and drills a hole in the back wall, leaves the RTA-average low enough to not activate your compressor, and so that range gets blasted even with young and healthy ears.  But if you turn UP the rest of the band to match (mix it well), then the average is high enough to make the compressor work, and then it's fine.

So it's important to have a FOH Engineer that knows what he's doing, on a rig that covers well, and keep the stage as quiet as possible.  No "acoustic spotlights" coming directly off the stage.  There are ways to do that without killing the "perfect" guitar sound: USE THEM!!!  A speaker-level DI, for one example, with a dummy load in place of the speaker, and the Monitor Engineer feeds a healthy amount of that back to the guitarist's floor wedge or IEM.  (In-Ear Monitor; basically glorified ear-buds)
(About that dummy load: Never run a tube/valve amp open-circuit.  It'll saturate the output transformer, which then becomes a short-circuit to the amp itself.  They tend to blow up when you do that.  Always have some kind of a load attached whenever it's powered on, that matches its rated impedance.  Likewise for a solid-state amp with a transformer output, though the relative cheapness of transistors means that it *might* (!) actually have enough additional circuitry to protect itself.  Still won't sound the same though, without enough load to keep the protection from activating.  Solid-state with no transformer is fine to run open.)

(Also, for smaller gigs, FOH and ME are probably the same guy, but a fair number of rigs are starting to have musician-controlled monitors now, either from a personal mixer that receives multitrack from the FOH board or a phone/tablet app that directly controls their output of the board.)

In the very first days of digital audio, well before ADC or DAC devices had >12 bits, there was a remarkable yet simple hardware design called “continuously variable delta slope modulation” which sampled the analog audio at a very high rate (>50kHz), producing a single bit signifying whether the input voltage was rising or falling. Delay was generated by sending this stream of single bits through a bank of early dynamic RAM chips, often 1b by 16k, 64k, 256k memory cells per chip. The analog signal was recovered by a simple opamp integrator. I remember being quite impressed with the audio quality. It wasn’t noisy or grainy. It even measured well: THD+noise <0.5%

To summarize, a lot can be accomplished in the digital audio domain by having a thorough understanding of the underlying math. Forcing more bits through the hardware is also an effective brute-force approach. But it isn’t the only method which works.
With such low bit depth, you'd need a lot higher sample rate than that to produce decent quality.  (50MHz, maybe, not kHz)  But that's still the basic principle behind a modern audio ADC: sample in the low- to mid-MHz range, run a high-order digital lowpass at that rate with a corner frequency below Nyquist for the final output rate, and then pick samples out at that final rate to send to the world.  The high initial sample rate and digital lowpass mean that the analog anti-aliasing filter doesn't have to be anything special - its Nyquist target is in the MHz range - and the effective averaging of high-frequency noise fills in the lower bits of what would otherwise be a crummy converter.

That's also the (very) basic concept of a digital resampler: stuff a bunch of zeros between input samples to make a much higher sample rate, lowpass that at the high rate with a suitable corner frequency for Nyquist at the output rate, and pick samples out at the new rate.  Of course, there are TONS of optimizations that are mathematically equivalent to that, but don't require nearly as much processing power.  For example, if you're using a Finite Impulse Response (convolution) lowpass, then each output sample is calculated independently, and so you only have to run the filter at the output rate, not the high intermediate one.  If you're using an Infinite Impulse Response (more like analog behavior), then every output sample depends on everything, so you do have to do ALL the work.  But IIR requires much less effort per sample than FIR, so without optimized hardware for either one, I'm not really sure which is "better".

Something else that I find fascinating to consider, is the idea that the universe is inherently digital because quantum.  That allows me to compare the translation between different digital systems, with the translation between digital and analog.  In that context, it's almost the same thing: convert to stupidly-high quality, then back down to what you want to end up with.  The universe is simply that high-quality digital...except for the unavoidable noise that completely drowns out that amount of detail.

You know, I never expected to have an intense discussion of high-performance audio signal processing in a forum dedicated to microcontrollers...
My, my, how things have changed over time...
:)
I find myself comparing the RT1062 to an earlier Raspberry Pi.  I would not be surprised if someone got a Linux kernel to run on a Teensy 4.x.  So what's a microcontroller?  And what's a desktop PC?  Is there much of a difference anymore?  :-//
I also think about the early days of computing: entire rooms, eventually shrinking to something that would (barely) fit on a desk, with an amount of processing power that we solidly associate with 10-year-old microcontrollers today.  Those were the pioneers of today's desktop systems, so could such a "desktop" system be built on one of our 10-year-old microcontrollers?  (my guess is yes, with some caveats)
« Last Edit: February 24, 2023, 05:12:30 am by AaronD »
 

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #45 on: February 24, 2023, 06:13:14 am »
A major question for the OP is to learn what resolution is used inside the "16-bit" PJRC audio library?
I would think it'd be a major selling point if it was different, so because it doesn't say, I'm pretty sure the entire chain is 16-bit 44.1kHz.

...in a standard windows install (using IDE 2.x), it'll end up in "C:\Users\USERNAME\AppData\Local\Arduino15\packages\teensy\hardware\avr\1.57.2\cores\teensy4\"
I'll have to look at that.  Per the title, I'm on a Raspberry Pi, not Windows, but I'd imagine the folder structure to be similar, starting from the user's home.
On my Pi, it's "/home/USERNAME/.arduino15/packages/teensy/hardware/avr/1.57.2/cores/teensy4"
...there appears to be a complete project in there...

Part of that project is a module called "AudioStream", that has both a header (.h) and a source (.cpp) file.  Towards the top of the header file is:
(broken up into separately scrollable sections, if your viewer is small enough to require that, and some parts omitted for clarity)
Code: [Select]
// AUDIO_BLOCK_SAMPLES determines how many samples the audio library processes
// per update.  It may be reduced to achieve lower latency response to events,
// at the expense of higher interrupt and DMA setup overhead.
//
// Less than 32 may not work with some input & output objects.  Multiples of 16
// should be used, since some synthesis objects generate 16 samples per loop.
//
// Some parts of the audio library may have hard-coded dependency on 128 samples.
// Please report these on the forum with reproducible test cases.  The following
// audio classes are known to have problems with smaller block sizes:
//   AudioInputUSB, AudioOutputUSB, AudioPlaySdWav, AudioAnalyzeFFT256,
//   AudioAnalyzeFFT1024
Code: [Select]
#ifndef AUDIO_BLOCK_SAMPLES
#define AUDIO_BLOCK_SAMPLES  128
#endif

#ifndef AUDIO_SAMPLE_RATE_EXACT
#define AUDIO_SAMPLE_RATE_EXACT 44100.0f
#endif

#define AUDIO_SAMPLE_RATE AUDIO_SAMPLE_RATE_EXACT
Code: [Select]
typedef struct audio_block_struct {
uint8_t  ref_count;
uint8_t  reserved1;
uint16_t memory_pool_index;
int16_t  data[AUDIO_BLOCK_SAMPLES];
} audio_block_t;

So it looks like this Audio Library is indeed 16 bits all the way through, with a default buffer of 128 samples and a strong recommendation to not change that.  I would also not be surprised if at least some of the modules were hard-coded for the default sample rate too, so changing that will result in all of their frequencies being off by that much.  (DSP functions know nothing about time, only samples)

The default buffer size at the default sample rate gives 128(samp) / 44100(samp/sec) = 0.0029(sec), or 2.9ms, of latency.
For comparison, the pro digital thing that I mentioned in an earlier post specifies 0.8ms analog to analog.  Having read the datasheet for several different audio ADC's and DAC's, I'm convinced that most if not all of that is in the converters, not the DSP.  So that DSP probably runs a single sample all the way through.  Buffer of 1, or no buffer, depending on how you think of it, and that's also what I think a later one of my projects is going to need.

That later project will effectively undo the acoustic lowpass through a wall that can't be removed (and a bunch of other things, but that's the core function), and so its analog-to-analog latency needs to line up with the speed of sound across a short distance.  At ~13.5in/ms (~34.3cm/ms), that's not very long, so I'm looking at running that system at a higher sample rate, just to allow the converters to use less-aggressive filters and thus reduce the latency by several samples *in addition to* having less time per sample.
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6265
  • Country: fi
    • My home page and email address
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #46 on: February 24, 2023, 09:49:23 am »
Just like all the other parts of Teensyduino, the source code for the Audio library can be found at Paul Stoffregen's Github, specifically the PaulStoffregen/Audio repo.  They are quite permissively licensed, so you can create your own derivative, using either single-precision floating-point (24 bits of precision, Teensy 4.x does have single- and double-precision support in hardware), or 32-bit integer/fixed point for computation, or even double-precision floating point (53 bits of precision).

Also note that that library works with the Audio System Design Tool for Teensy Audio Library, a standalone HTML+JS tool that works in your browser (no server or internet connection needed), and can generate the Arduino sketch (source code) needed to implement the graphically defined audio system.
 

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #47 on: February 24, 2023, 01:37:36 pm »
Just like all the other parts of Teensyduino, the source code for the Audio library can be found at Paul Stoffregen's Github, specifically the PaulStoffregen/Audio repo.  They are quite permissively licensed, so you can create your own derivative, using either single-precision floating-point (24 bits of precision, Teensy 4.x does have single- and double-precision support in hardware), or 32-bit integer/fixed point for computation, or even double-precision floating point (53 bits of precision).
Nice!  But I didn't see a license file in there.  As I understand, the legal answer is "no", unless specified otherwise, and I don't see the specification otherwise.  Where is that?

There's still the 128-sample buffer that I'd rather not have (that's probably to allow the audio design to co-exist with a naive sketch without dropping out), so it's not just a matter of changing the datatype.  But I can't imagine it'd be *too* hard to read Paul's source, understand what it does, and write my own from that...once I see the license for myself and like it.

The multi-way speaker system that I described earlier is fairly low-power.  16-ohm "junk drawer" speakers, probably bridged, from a 12-24V DC supply.  (yes, I know that's a big range; I'll figure it out later)  I'm currently bridging them from the 12V rail of an ATX supply without a sub, and I could use a bit more than that.  I have a powered sub waiting to go in once I figure out the processing, so I guess I'll see what that does first.  Taking the bass out of the mains is going to help a lot by itself.

Another project idea is for an actual PA, and was originally going to use a 19" rack of small DSP amps that all get the same analog signal to drive a 4-way "true full-range" system.  No separate sub needed.  I'm starting to think about using a Teensy4 / RT1062 for that too: forget the rack entirely, and just mount the custom PCB('s) directly on/in the cabinet.



Also, I've not had good experience with the Pi Pico's tinyusb library for audio.  It works at first, but there seems to be a bug somewhere that gets it "stuck" when the host stops sending samples.  When the host restarts, tinyusb doesn't.  Since one of my projects is going to need ~8 tracks of high-quality audio each direction between itself and a PC, how's the Teensy USB library?
« Last Edit: February 24, 2023, 01:43:01 pm by AaronD »
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6265
  • Country: fi
    • My home page and email address
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #48 on: February 24, 2023, 02:47:22 pm »
Just like all the other parts of Teensyduino, the source code for the Audio library can be found at Paul Stoffregen's Github, specifically the PaulStoffregen/Audio repo.  They are quite permissively licensed, so you can create your own derivative, using either single-precision floating-point (24 bits of precision, Teensy 4.x does have single- and double-precision support in hardware), or 32-bit integer/fixed point for computation, or even double-precision floating point (53 bits of precision).
Nice!  But I didn't see a license file in there.
The license is at the beginning of each file.  It makes accepting contributions from other developers clearer; the license is impossible to miss.

But I can't imagine it'd be *too* hard to read Paul's source, understand what it does, and write my own from that...once I see the license for myself and like it.
As you can see in the contributor graph, it's not just Paul, but quite a few PJRC forum regulars and contributors in there.  But the code is indeed quite clean, once you get used to the mixed freestanding C/C++ style used.

Since one of my projects is going to need ~8 tracks of high-quality audio each direction between itself and a PC, how's the Teensy USB library?
Even over USB Serial (which on the Linux end has quite a bit of overhead, as it goes through the tty layer), I can sustain well over 200 Mbit/s in one direction.
Even 32-bit 192 kHz raw audiostream is just 6.144 Mbit/s, so the USB is definitely not a problem.

In case you haven't realized, RPi Pico only supports USB 1.1, 12 Mbit/s, whereas Teensy 4.0/4.1/MicroMod supports USB 2.0, 480 Mbit/s.

Teensyduino itself supports various combinations of USB endpoints (like USB HID + USB Serial), as in selectable directly from the Arduino GUI, so multi-endpoint USB is also well supported.  Teensy 4.0/4.1/MicroMod supports 8 bidirectional endpoints on each USB controller.  (They have two OTG/EHCI HS/FS/LS USB cores, core 0 wired to the programming USB connector, with the other on pads in 4.0 and pins in 4.1.)
 
The following users thanked this post: newbrain

Online Marco

  • Super Contributor
  • ***
  • Posts: 6723
  • Country: nl
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #49 on: February 24, 2023, 03:24:59 pm »
The Tympan library is an expanded reimagining of the Teensy audio library for 32 bit floating point, created for the Tympan hearing aid project.
 
The following users thanked this post: elecdonia, Nominal Animal, AaronD

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #50 on: February 24, 2023, 04:16:34 pm »
The Tympan library is an expanded reimagining of the Teensy audio library for 32 bit floating point, created for the Tympan hearing aid project.
Thanks! I’ll look into that!

OT:   I’m fascinated by the Tympan project because at my age (73) my personal internal biological carbon-based audio system (my ears) don’t function as well as they did in my 20’s. My thought is to model the human cochlea. Some of the tiny hairs in it act as microphones. They are attached to nerve cells. Others act as tuned resonators, providing gain and equalization in selected frequency bands. Although they don’t feed directly into the auditory nerve cells, they do interact with the “sensing” hairs to amplify/equalize the incoming vibrations. Also important for me is to is to review recent scientific discoveries about the hearing of other species: Dogs and birds have fascinating capabilities in their auditory systems.
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #51 on: February 24, 2023, 04:25:44 pm »
At this point I would suggest changing the title of this topic to something like:

“Real-time audio processing with RT1062 (Teensy4.1) and RPi Pico”

I’ve been struck by the breadth of audio engineering expertise and raw coding talent possessed by the contributors to this topic. Also, every day I am finding impressive examples of fully functional audio processing systems implemented on Teensy4.1 and RPi Pico hardware. Nothing like this existed 10 years ago.
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #52 on: February 24, 2023, 04:38:37 pm »
Quote
As you can see in the contributor graph, it's not just Paul, but quite a few PJRC forum regulars and contributors in there.  But the code is indeed quite clean, once you get used to the mixed freestanding C/C++ style used.
This is a signature characteristic of the Arduino project: Segregate the C++ code into libraries. Simplify the visible code in applications (e.g. “sketches” in Arduino-speak). I feel this makes the Arduino environment more user-friendly, especially for teaching newbies how to write code.

As a person who mainly coded in assembly language from 1980-2010, my exposure to Arduino helped me gain a thorough understanding of C and C++.
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #53 on: February 24, 2023, 05:20:51 pm »
Quote
The multi-way speaker system that I described earlier is fairly low-power.  16-ohm "junk drawer" speakers, probably bridged, from a 12-24V DC supply.  (yes, I know that's a big range; I'll figure it out later)  I'm currently bridging them from the 12V rail of an ATX supply without a sub, and I could use a bit more than that.  I have a powered sub waiting to go in once I figure out the processing, so I guess I'll see what that does first.  Taking the bass out of the mains is going to help a lot by itself.
Three cheers for working directly with the audio hardware!
One of my long-term activities is repair/refurbishing of home and pro audio gear which was manufactured from 1950 to the present time.
Recent projects:

Repairing home-theater powered subwoofers (Klipsch, Infinity, Boston Acoustics). A very common failure is caused by small electrolytics on the class-D amplifier boards drying out (example: 4.7uF/25V). Also I had to learn how the “BASH” architecture works: Bridged analog output stage powered from a variable voltage DC source which tracks the audio signal level. It’s an odd circuit design. BASH amplifier PC boards look cheesy but they do function after installing decent quality small electrolytics. Note: You should be able to find plenty of powered subwoofers cheap, free,or “on-the-curb.” Faulty drivers are rare. Bad small electrolytics on the class-D amplifier boards are ubiquitous. Most of these “dead” subwoofers are repairable.

Repairing home-theater receivers (Onkyo, Yamaha). Main issue is intermittent TI DSP chips. TI had issues with the metallization layer. Failures didn’t occur until 3-5 years of age. TI redesigned their fab process.  Replacement DSP chips (improved) are available.



Quote
Another project idea is for an actual PA, and was originally going to use a 19" rack of small DSP amps that all get the same analog signal to drive a 4-way "true full-range" system.  No separate sub needed.  I'm starting to think about using a Teensy4 / RT1062 for that too: forget the rack entirely, and just mount the custom PCB('s) directly on/in the cabinet.
Somewhere I have docs for my 1980-era 3 or 4-way active electronic crossover with anti-clipping limiters on each output. It could be re-implemented digitally.
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: Real-time audio processing with RT1062 (Teensy4.x)
« Reply #54 on: February 24, 2023, 08:23:28 pm »
At this point I would suggest changing the title of this topic to something like:

“Real-time audio processing with RT1062 (Teensy4.1) and RPi Pico”
Yeah, I think you're right.  Though I don't remember much about the Pico here, except that it's barely enough for some very small projects like simple synthesizers and the like.
 

Online Marco

  • Super Contributor
  • ***
  • Posts: 6723
  • Country: nl
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #55 on: February 24, 2023, 09:03:54 pm »
There's still the 128-sample buffer that I'd rather not have (that's probably to allow the audio design to co-exist with a naive sketch without dropping out), so it's not just a matter of changing the datatype.  But I can't imagine it'd be *too* hard to read Paul's source, understand what it does, and write my own from that...once I see the license for myself and like it.

The Tympan library also makes this a little easier to change, you can do it from code instead of having to mess with library recompilation.

I don't know down to what size you can reasonably set audio_block_samples, but I'd just try to set it to two and see from there. At 600 MHz having  to push/pop the register file more often is a lot less relevant than at 48 MHz.
 

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: Real-time audio processing with RT1062 (Teensy4.x)
« Reply #56 on: February 24, 2023, 09:16:13 pm »
The Tympan library is an expanded reimagining of the Teensy audio library for 32 bit floating point, created for the Tympan hearing aid project.
I'll have to look at that too!  Thanks!

The license is at the beginning of each file.  It makes accepting contributions from other developers clearer; the license is impossible to miss.
Oh, okay.  I was looking for an overall license that I could read, that covers the entire library, without the specific sources appearing in my history to say that I've been "contaminated" before I find that one of them has some non-compatible restrictions.  Don't know if I'm going to sell something yet or not, but I don't want to get tied up in IP trouble because I've seen something that has that restriction before I made mine.

Tympan does have an overall license file, for the MIT license, which pretty much says I can do anything I want as long as I keep that notice and the license that goes with it.  My understanding is that I can't close my copy of the source and sell that - I have to keep it open - but I think that's okay.  If I do sell something, it'll be the hardware that runs that software, not the software itself.

Even over USB Serial (which on the Linux end has quite a bit of overhead, as it goes through the tty layer), I can sustain well over 200 Mbit/s in one direction.
Even 32-bit 192 kHz raw audiostream is just 6.144 Mbit/s, so the USB is definitely not a problem.
I was more concerned with the USB device library "going to sleep" or "getting stuck in standby" or whatever it does, when the host stops sending audio samples.  That was my trouble with tinyusb.

It would work perfectly well, until the player app had been stopped or even paused for a few seconds and the Linux host's audio system stopped sending silence.  Changing the output device to I2S on the GPIO header, allowed me to put a probe on it to see what the host driver was actually doing, and it does indeed stop all activity and then restart when something starts playing again.  Apparently, tinyusb didn't like that stop and restart, because it worked perfectly before the first stop, but never came back after.

Technically, I could have a background task, maybe aplay from /dev/zero to that card, just to always have something playing, but that's kinda "hacky".  So my concern was that the Teensy library still works after no-data for a while.

In case you haven't realized, RPi Pico only supports USB 1.1, 12 Mbit/s, whereas Teensy 4.0/4.1/MicroMod supports USB 2.0, 480 Mbit/s.
Yes.  I was thinking that the Pico would be enough for decent-quality stereo both ways on USB, and I could get the concepts going on that, and then switch to something with a better PHY for the 8-channel one.  For a while, I was wondering about the RT1010 as a standalone chip, and I would figure out later how to support it, but it looks like the RT1062 with Arduino-like support would be easier.

Teensyduino itself supports various combinations of USB endpoints (like USB HID + USB Serial), as in selectable directly from the Arduino GUI, so multi-endpoint USB is also well supported.  Teensy 4.0/4.1/MicroMod supports 8 bidirectional endpoints on each USB controller.  (They have two OTG/EHCI HS/FS/LS USB cores, core 0 wired to the programming USB connector, with the other on pads in 4.0 and pins in 4.1.)
I was thinking about Audio+HID, which I don't see in the list.  But Audio+Serial can work.  The HID or Serial connection goes to a custom app that I have yet to write, to manage DSP coefficients far beyond what the Audio spec is made for, so I'm still flexible on that.  Either disable the Audio's control functions - don't even report them as available - or have some dummy code that makes it look like they work but they really don't do anything, or maybe they do actually control the master volume while everything including that works in the app...  Several ways to do that.

As I understand, it's a pair of endpoints per function, like Audio or HID, etc., not per channel of Audio.  So even if I made a 32x32 Audio card, it would still use only one endpoint each direction.

There's still the 128-sample buffer that I'd rather not have (that's probably to allow the audio design to co-exist with a naive sketch without dropping out), so it's not just a matter of changing the datatype.  But I can't imagine it'd be *too* hard to read Paul's source, understand what it does, and write my own from that...once I see the license for myself and like it.

The Tympan library also makes this a little easier to change, you can do it from code instead of having to mess with library recompilation.
I would hope that for a real-time mic-to-speaker thing, with the two so close together physically, that it would have a similar constraint to mine, of absolute-minimum inherent latency plus *maybe* an explicit delay of just a handful of samples but no more.  That almost dictates single-sample processing with no buffer at all, and to run the converters as fast as can be afforded to reduce their latency as well.  (high sample rate, not because it sounds any better - it doesn't - but to allow the converters to use less aggressive filters, which in turn cuts the latency way down, more than just the time between samples)
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6265
  • Country: fi
    • My home page and email address
Re: Real-time audio processing with RT1062 (Teensy4.x)
« Reply #57 on: February 24, 2023, 11:03:47 pm »
The license is at the beginning of each file.  It makes accepting contributions from other developers clearer; the license is impossible to miss.
Oh, okay.  I was looking for an overall license that I could read, that covers the entire library
I haven't checked every file in the Audio library, but they tend to be
Quote
Copyright 20nn Paul Stoffregen

Development of this audio library was funded by PJRC.COM, LLC by sales of Teensy and Audio Adaptor boards.  Please support PJRC's efforts to develop open source software by purchasing Teensy or other PJRC products.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice, development funding notice, and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

It is sufficient your derivatives include the text from the files you used (once per exact text suffices), in the sources if you provide those, or in the documentation, explain that parts of the product were developed using Teensy Audio Library by Paul Stoffregen and others, under the following conditions: (followed by the above text, but do check each file you use as a basis for deriving your own).

So, not exactly onerous.  You don't need to provide sources of your own versions, only include the permission notice in the documentation (and/or sources, if you provide sources).  This can be considered a derivative of the MIT license.

Tympan does have an overall license file, for the MIT license, which pretty much says I can do anything I want as long as I keep that notice and the license that goes with it.  My understanding is that I can't close my copy of the source and sell that - I have to keep it open - but I think that's okay.
No, you do not need to keep derivative or combined sources open, when using the MIT license.

What you do need to do, is mention in the sources (if you provide sources) or/and in the documentation, that parts of the product were developed using <whatever-MIT-licensed-thing> using the following license: <followed-by-the-license-text>.

I was more concerned with the USB device library "going to sleep" or "getting stuck in standby" or whatever it does, when the host stops sending audio samples.  That was my trouble with tinyusb.
No; as long as the port provides power, Teensy will continue working.

For example, when using USB Serial, (serial) evaluates to True when there is an application that has the serial port open, and to False when there isn't.  (It is useful and kinda necessary when printing debug output.  If no application has the serial port open, printing to it (from Teensy) will eventually block, because unlike Windows, other OSes do not actually discard the information by default.)

It would work perfectly well, until the player app had been stopped or even paused for a few seconds and the Linux host's audio system stopped sending silence.
That just means the player app closes the character device when it's not playing.  You can detect that on Teensy easily, but there is no automagic go-to-sleep tied to it at all.

Teensyduino itself supports various combinations of USB endpoints (like USB HID + USB Serial), as in selectable directly from the Arduino GUI, so multi-endpoint USB is also well supported.  Teensy 4.0/4.1/MicroMod supports 8 bidirectional endpoints on each USB controller.  (They have two OTG/EHCI HS/FS/LS USB cores, core 0 wired to the programming USB connector, with the other on pads in 4.0 and pins in 4.1.)
I was thinking about Audio+HID, which I don't see in the list.
You can modify the core files to export Audio+Keyboard or Audio+Keyboard+Joystick or whatever you need.

In hardware/teensy/avr/boards.txt, you add the new usbtype (USB_AUDIO_HID, for example); then the corresponding USB details into hardware/teensy/avr/cores/teensy4/usb_desc.h.  If you use diff -u and patch, you can keep these changes in a diff file, and submit to Paul after testing, or just keep them as a diff/patch so you can easily upgrade Arduino and Teensyduino to newer versions without having to do the changes by hand.

USB Audio uses two endpoints and three interfaces.  Keyboard uses two endpoints and two interfaces (one for normal keys aka KEYBOARD, one for media keys aka KEYMEDIA).  Joystick and MIDI both use one endpoint and one interface.  RawHID uses one or two endpoints and one interface.  Serial uses two or three endpoints and three interfaces.  If you don't use serial, seremu takes one endpoint and one interface.  Teensy 4.x and MicroMod have 8 bidirectional endpoints, and one is reserved for configuration; so, basically, you can pick any combination that uses at most seven endpoints.
Audio+Keyboard+Serial, Audio+MIDI+RawHID+Serial, Audio+Keyboard+seremu, Audio+MIDI+RawHID+seremu are all possible.
 

Online Marco

  • Super Contributor
  • ***
  • Posts: 6723
  • Country: nl
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #58 on: February 25, 2023, 05:04:53 am »
It's been a while since I looked into this space, but just want to point out that for the actual Raspberry Pi there's a new kid on the block too. Elk OS. It's Linux, but with a real time kernel underneath running the actual audio processing (with standard API plugins, VSTx, LV2). The minimum buffer size for which it is compiled is 16, but for all the talk of pros a lot of pro stuff gets done on Macs and the latency there is going to be ten times a 32 sample latency.
 
The following users thanked this post: elecdonia

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14488
  • Country: fr
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #59 on: February 25, 2023, 10:08:16 pm »
At this point I would suggest changing the title of this topic to something like:

“Real-time audio processing with RT1062 (Teensy4.1) and RPi Pico”

Well, after reading it all, to me it looks more like: "real-time audio processing RT1062 (Teensy4.1) and RPi Pico using an existing library so you don't have to write a single line of low-level code".

While, as I pointed out, it's possible to do quite interesting stuff even on a RP2040 writing your own code and taking advantage of the embedded peripherals, most of what I've read is about existing libraries.

Nothing wrong with that, but there is a definite difference between "what is possible" and "what is possible with existing libraries".
That applies to pretty much any software development too these days it seems, so nothing really specific to this thread. Just a thought. You can ignore that and keep going. :popcorn:
 

Online Marco

  • Super Contributor
  • ***
  • Posts: 6723
  • Country: nl
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #60 on: February 26, 2023, 03:03:05 pm »
You can take the library down to single sample latency. I doubt all the filters are assembly/intrinsic optimised yet. Plenty of low level coding to be done with existing code bases if it takes your fancy.

Bootstrapping a micro and implementing an interrupt and event loop isn't all there is at low level.
 

Offline Nominal Animal

  • Super Contributor
  • ***
  • Posts: 6265
  • Country: fi
    • My home page and email address
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #61 on: February 26, 2023, 04:15:30 pm »
Most of the Teensy Audio Library features were dictated by the Teensy Audio Adaptor Board (aka Audio shield), which supports I2S (8 channels) or TDM (16 channels), using NXP SGTL5000 chip.  It's evolved quite a bit in the last seven years for different Teensy models (3.1, 3.0-3.6, 4.x), but still keeping the core SGTL5000 chip.  Things like the sample buffer size derive from this use pattern and needs.

So, it is best to not consider the Teensy Audio library as a generic audio library, but more like one dedicated for SGTL5000 but which can be used for other purposes as well.
« Last Edit: February 26, 2023, 04:17:32 pm by Nominal Animal »
 
The following users thanked this post: AaronD

Offline elecdonia

  • Frequent Contributor
  • **
  • Posts: 399
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #62 on: February 28, 2023, 09:16:38 pm »
Today PJRC strongly recommends advancing to the Teensy4.1. They warn that their older boards may never be manufactured again (due to chip-a-geddon).
     https://www.pjrc.com/store/teensy41.html
     https://www.pjrc.com/store/teensy3_audio.html
     https://www.pjrc.com/teensy/td_libs_Audio.html

After finishing this post I'm ordering a Teensy4.1 along with its matching audio board. (as of this moment) both are in stock at the PJRC store.
The Teensy 4.1 and its audio board arrived yesterday. I'm eager to get them hooked up and running!
I’m learning to be a leading-edge designer of trailing-edge technology.
 

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #63 on: February 28, 2023, 10:36:26 pm »

Most Arduino "cores" include a copy of the appropriate gcc, CMSIS, and newlib tools, in addition to the code that makes them "Arduino.'  You could just copy them to a more dignified-sounding directory, add them to your paths, or whatever.

Since I despise both Arduino (it has its place, but not for my needs) and MCUxpresso, I use just clang + gdb + CMSIS + SDK + Cmake + ninja + VS Code.

I think I'm finally starting to understand what that means.  But compared to the 8-bitters that I'm used to:

I am properly intimidated by the i.MX RT. ;)  It's a complex beast plus a new (to me) programming environment.

This is certainly a complex MCU.
...
I would really suggest starting with the NXP SDK instead. There's a ton of examples.

That's probably excellent advice, but:

I’m a bit surprised that official NXP development tools aren’t supported on RasPi OS. Are those NXP tools for Windows 10 only? Nothing for any flavor of Linux?
That's how it appears to me, but westfw has a good point that I still have to check.  I might be able to use parts out of the Arduino framework, which does run on Linux, and the Pi, to make my own toolchain.

The problem that I see so far isn't so much the complex chip, but the complex toolchain that nobody seems to have a tutorial for.  "Just install this and use it"...which is only for Windows, and I don't have Windows.  And trying to parse the Arduino boilerplate project, which appears to be a unified library for the entire chip that can do absolutely anything and everything that the single "add your code here" file might call for, is making my head spin.

Is there a step-by-step tutorial to using the RT1062 as a standalone chip?  Or close enough that I can find/replace the name and have it work?

Looks like CMSIS is going to be useful, and it even comes with a DSP add-on that does a lot more than just audio.  And the Apache 2.0 license is good too.  Though in reading the source for its filter algorithms, I still see too much reliance on buffers for my near-zero-latency project, so I might just use that part for reference instead.
This sort of middleware is not really necessary for 8-bit projects, so I don't have any experience with it yet, as those are simple enough to just bit-bang the registers as described in their (single, complete!) datasheet.

And since PJRC's MKL02 support chip ties up the debugging lines, it won't be necessary to include that part of the toolchain.  Since I learned originally without that feature anyway, I don't think I need it now either.  (nor is it particularly useful if USB dies when I stop it to inspect memory, or a power-controlled thing goes immediately to 100% and blows up, etc.)



What I'm comparing to:
  • Microchip's MPLAB runs everywhere, and is okay if you don't mind NetBeans being stupid.  (seems to be a common problem with all generic IDE's that a vendor-specific thing might be based on)
  • AVR didn't have an official IDE when it became popular, so there are lots of tutorials for how to install and use that toolchain with whatever IDE you might have, or CLI-only if you prefer that.  And Code::Blocks has it directly in its New Project Wizard.  I would guess that other standalone IDE's do too.
  • Microchip's XC8 compiler is actually a standalone command-line thing, which means that any IDE can use it.  So I've also done some PIC projects with that in Code::Blocks by copying and modifying an existing AVR project to use that toolchain instead.  Then I switched to SDCC for the PIC by following the instructions on that site.  Not as good as the fully-paid version of XC8, but a lot better than the free XC8!
My copy of Code::Blocks (20.03, even after the standard sudo apt {update|full-upgrade|autoremove} - has it been abandoned?) doesn't seem to know about the Teensy 4.x or the RT1062.  I seem to remember things appearing in the New Project Wizard after installing their toolchains, but this one doesn't:
  • It has an option for ARM, but it ends up with a short list of dev boards that doesn't include that, and requires me to pick one.
  • It also has an option for Arduino (yet another alternative to the official Arduino IDE, I guess), but it only allows the AVR GCC compiler.

The RPi Pico SDK as a CMake project in VSCode feels to me like the perfect blend of simplicity and capability.  The main() function is yours (that's a massively big deal!), and everything the chip does is available as a well-documented and discoverable SDK function with readable source code, but it doesn't have a million files in your face all at once to keep track of like the Arduino boilerplate does.  Granted you're not supposed to look at that boilerplate, but still...
The only thing I don't like about VSCode is that the text editor lacks a few features that Code::Blocks has, and I keep tripping over that.  (insert/overwrite is a big one, for modifying datasets quickly)

Is there a pre-built project somewhere, or a tutorial that makes one, for the RT1062 (or just a quick config-change away) with an empty or trivial main() function, and builds like that with no errors or warnings*, and uses the CMSIS Pack for that chip like the Pico uses its SDK?



* I've seen some that you have to add something to make it build.  Umm...add what?!  It's practically DOA for me, and a sign of shoddy work, if you can't even ship a working starting point.  If the most trivial thing possible doesn't even work, how am I to trust anything else?
« Last Edit: February 28, 2023, 10:45:53 pm by AaronD »
 
The following users thanked this post: sicco

Offline westfw

  • Super Contributor
  • ***
  • Posts: 4199
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #64 on: March 01, 2023, 08:01:30 am »
Quote
Is there a step-by-step tutorial to using the RT1062 as a standalone chip?
I don't think so.  That sort of thing didn't really last past the 80s. :-(
That's part of why Arduino was so revolutionary.
(I guess TI did some pretty in-depth tutorials for their "launchpad" boards.  Usually, such tutorials would be for a specific vendor development board, rather than chip level.)
Quote
AVR didn't have an official IDE when it became popular
Sure it did.  "AVR Studio" (though version 4, a custom Atmel IDE) and the later "Atmel Studio" (v5,6,7, based on Visual Studio) existing long before Arduino.  It didn't used to include a C Compiler, which you had to install separately ("WinAVR" if you wanted one that wasn't $$$.  Several other compilers were also "supported.")Windows only, of course.  "Crosspack AVR" existed for MacOS (just the compiler and some generic tools.)
Quote
Microchip's XC8 compiler is actually a standalone command-line thing, which means that any IDE can use it.
As is gcc.  I don't think that ANY of the modern IDEs have the compiler intimately linked into the IDE.
A generic arm-eabi-none-gcc toolchain can compile for pretty much any of the ARM Cortex M chips (at least.)  You just have to gather all the chip-specific .h file and libraries sort-of manually.  :-(
You might want to look at "platformio" ( https://docs.platformio.org/en/latest/platforms/nxpimxrt.html ); my understanding is that it lies somewhere in between Arduino and Vendor-specific IDEs.  I haven't used it myself (not yet another IDE), but I believe it's supposed to be more chip-independent than most vendor IDEs.


 

Offline mac.6

  • Regular Contributor
  • *
  • Posts: 225
  • Country: fr
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #65 on: March 01, 2023, 12:35:23 pm »
Just download the SDK for linux with all toolchain, this will gives you basic SDK with cmake support.
Unzip it, install your own arm-none-gcc (from ARM or your distro), then export variable ARMGCC_DIR (usually /usr) and go into boards/evkmimxrt1064
From there you can compile the demo you want
for example for the hello_world demo:
cd boards/evkmimxrt1064/demo_apps/hello_world/armgcc
./build_all.sh

Oh yes there is a ton of different configuration for each demo, that's why it's a big step from 8bits micro.
 

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #66 on: March 01, 2023, 06:23:19 pm »
I’m a bit surprised that official NXP development tools aren’t supported on RasPi OS. Are those NXP tools for Windows 10 only? Nothing for any flavor of Linux?
That's how it appears to me, but westfw has a good point that I still have to check.  I might be able to use parts out of the Arduino framework, which does run on Linux, and the Pi, to make my own toolchain.

Just download the SDK for linux with all toolchain, this will gives you basic SDK with cmake support.

Where?

Quote
Is there a step-by-step tutorial to using the RT1062 as a standalone chip?
I don't think so.  That sort of thing didn't really last past the 80s. :-(
That's part of why Arduino was so revolutionary.

Huh.  I was getting started with AVR in the late '00s / early '10s.  (PIC came before that, with a 3rd-party toolchain that someone gave me for free)  Google gave me a bunch then, to build a project from scratch for AVR.  I just picked one, and it worked.

I had just gotten good at that when the first Arduino came out.  The bare chip on a custom PCB was so easy for me that I never saw the point.  And I still don't, except for making it "brick-proof".  I can understand a basic, know-nothing beginner programming with busy-loop delays, but once you see that working, I think (quite strongly) that the next step should be to convert that into a clearly non-blocking state machine that polls flags instead.  Everything else is then based on that.

Not making that transition early, is a major failure as I see it, especially when applications like mine come along that need to control the timing absolutely precisely, and they just can't.  Not obviously anyway.  The Arduino framework tries to do absolutely everything while keeping the busy-loop delays, and that leads to some concerns about latency and throughput.



But I did just think of something:

The Arduino framework does handle this chip well already (of course it does, it's aimed at beginners!), and there's a remote possibility of open-sourcing a project or two in addition to selling the custom PCB that it runs on.  So if customers are going to modify it, then the Arduino framework might be the best thing after all, and I just need to see if it'll work as-is.  Possibly yes.

I'd still write my own Audio DSP code, and not use the library that already works but is limited to a specific chip that can't do what I need.  But if I can satisfy the timing with cycles to spare, then that's really all that matters.

There are only 2 lines of code in the Arduino's main loop - the loop() function and an event handler called yield() - and I was concerned about the event handler being bloated and not being able to pare it down to where the rest of the code could keep up.  (96kHz audio or faster, with no buffer)  Maybe it's okay?  I could just comment the yield() call out, but what problems does *that* cause?!  (besides needing to either ship my entire custom framework or tell customers how to modify theirs)

Trying to avoid customer headaches, and gotchas for the library functions that I do want to use, I wonder how to minimize the event handler's effect while still having all of the functionality that I need?  I'm perfectly okay in principle to have my own version of it, probably calling the same API myself, so that I KNOW what the timing is going to be...but that doesn't really work well from a customer's standpoint.



In all of this, I've been trying to avoid interrupts as much as possible, not because I'm scared of them, but as a possible holdover from the 8-bit world, where I really want to reserve them for things that *really do* need IMMEDIATE attention.  In the 8-bit world, the interrupt controller disables itself completely until the ISR returns, so that the first interrupt that fires, regardless of priority, runs all the way to completion before another one has a chance.  Thus, using interrupts for everything kills the ability to time them precisely, especially when you have an ISR that takes forever to finish!
(big motivation there, to have everything non-blocking, with polled flags for things that would otherwise be low-priority interrupts)

But maybe I do want the DSP code to be entirely interrupt-driven.  If it were the highest priority (or the only) interrupt in the system, then that would guarantee the timing, regardless of what the rest of the code is doing or how bloated it might be.  Even if I end up spending 90% of the CPU time in that one ISR, that's probably okay.  With a 600MHz clock, that leaves me with effectively 60MHz for everything else, fairly evenly spaced, which is *still* an improvement on the 8-32MHz clocks that I'm used to from the 8-bit world!

I also like to have a "CPU load" signal somewhere that I can put an oscilloscope on.  In this case, it might be the Teensy's on-board LED, which I would keep for my custom board along with the other Teensy-related stuff.  Turn it off at the start of the DSP ISR and on at the end, so if you can still see it, the audio is okay.  If it goes out completely, you're trying to do too much.  And its frequency can be checked on a 'scope, as the actual sample rate.



So I guess I just need to get the classic "Hello World" to a terminal via USB (in lieu of a "real" debugger), using the stock Arduino framework, build from there while keeping the stock framework, and see when or even *if* it breaks.
 

Online newbrain

  • Super Contributor
  • ***
  • Posts: 1719
  • Country: se
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #67 on: March 01, 2023, 11:27:16 pm »
Where?
Here:
https://mcuxpresso.nxp.com/en/builder?hw=MIMXRT1064xxxxA

You need to register an NXP account to get there, though.

Include all the extra libraries you need, (e.g. FreeRTOS or Azure RTOS, Arm DSP etc.), select your OS and the toolchain (or use All) and after a short wait you'll be able to download the package.
There should be plenty of examples.
Nandemo wa shiranai wa yo, shitteru koto dake.
 

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #68 on: March 02, 2023, 03:30:44 am »
Where?
Here:
https://mcuxpresso.nxp.com/en/builder?hw=MIMXRT1064xxxxA

You need to register an NXP account to get there, though.

Include all the extra libraries you need, (e.g. FreeRTOS or Azure RTOS, Arm DSP etc.), select your OS and the toolchain (or use All) and after a short wait you'll be able to download the package.
There should be plenty of examples.

Does it work for Linux on a Raspberry Pi?  And does it have a hobby-friendly license and pricing (free and permissive) that also allows me to sell my project if I decide to do that later?

I never saw any of that when I was looking.  Windows only, and their chip-specific CMSIS library comes with a license file that makes me wonder if NXP ends up owning my project instead of me.
https://mcuxpresso.nxp.com/cmsis_pack/repo/NXP.MIMXRT1062_DFP.16.0.0.pack
 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14488
  • Country: fr
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #69 on: March 02, 2023, 05:43:46 am »
Where?
Here:
https://mcuxpresso.nxp.com/en/builder?hw=MIMXRT1064xxxxA

You need to register an NXP account to get there, though.

Include all the extra libraries you need, (e.g. FreeRTOS or Azure RTOS, Arm DSP etc.), select your OS and the toolchain (or use All) and after a short wait you'll be able to download the package.
There should be plenty of examples.

Does it work for Linux on a Raspberry Pi?  And does it have a hobby-friendly license and pricing (free and permissive) that also allows me to sell my project if I decide to do that later?

All you need is Cmake and arm-none-eabi-gcc, both being available on just any Linux distro I can think of. The rest is just source files, you do not need anything else.

I never saw any of that when I was looking.  Windows only, and their chip-specific CMSIS library comes with a license file that makes me wonder if NXP ends up owning my project instead of me.
https://mcuxpresso.nxp.com/cmsis_pack/repo/NXP.MIMXRT1062_DFP.16.0.0.pack

Most of the SDK is under a BSD-3 license. The CMSIS has a specific license that I admit I have never really had a look at, but if anything CMSIS is ARM stuff and not NXP. And pretty much all vendors of ARM-based MCUs do distribute CMSIS, and I have never encountered, or heard of, any licensing issue about that. I think we would have by now. :horse:
 

Online newbrain

  • Super Contributor
  • ***
  • Posts: 1719
  • Country: se
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #70 on: March 02, 2023, 08:41:24 am »
Does it work for Linux on a Raspberry Pi?  And does it have a hobby-friendly license and pricing (free and permissive) that also allows me to sell my project if I decide to do that later?

I never saw any of that when I was looking.  Windows only, and their chip-specific CMSIS library comes with a license file that makes me wonder if NXP ends up owning my project instead of me.
https://mcuxpresso.nxp.com/cmsis_pack/repo/NXP.MIMXRT1062_DFP.16.0.0.pack
I don't know where you looked, but:
  • The SDK is a source file collection - there is nothing Windows specific in there as Silicon Wizard said.
  • CMSIS is not from NXP, it's Arm SW and is distributed under an Apache 2.0 license (more below)
  • There is, in the SDK root, an optional NXP license, which might have scared you. See below for more details.

About Apache 2.0:
This is probably the best permissive, not copyleft, Open Source licenses around, striking the right mix of freedom and protection. FSF advise its use in case one doesn't want to go the GPL copyleft way.
You are able to distribute your SW in source or binary form and with a license (also non free) of your choice, provided that you abide by Apache's very mild requirements.
Remember that it's not compatible with GPL2 (or earlier) licensed code, only with GPL3, as it imposes additional restrictions not covered in GPL2.

About the NXP OPT license:
Yes that's scary. ST has a similar one.
BUT! If you carefully read it, you'll find that the license can be superseded by the specific component own licenses, and by what is listed in the  SW-Content-Register.txt file.
If you go there, you'll find that all the SDK drivers are licensed by NXP under the very permissive BSD-3 license. Other parts and libraries have their own licenses, e.g. MIT for FreeRTOS, mostly BSD-3 and MIT.
If you are using one of the non-MIT, non-BSD, or non-Apache components, check its license, but they are very few.

Disclaimer: IANAL, TINLA. OTOH, I have worked a lot with licensing issues of internal and third party SW as a FOSS product owner.

EtA: I'd be very careful around Azure RTOS. In SW-Content-Register.txt it points to NXP OPT license file, which in turn points to the terms of Microsoft license. Now, Microsoft's license for Azure RTOS is quite permissive, and the iMX RT 10xx is part of the LICENSED-HARDWARE.txt file, but it's not one of the standard FOSS licenses, and might have compatibility problems with some of them. If you intend to use it, get expe(nsive)rt advice, not from some rando on a forum.

EtA: Merci à Mac.6 pour le TL;DR  ;)
« Last Edit: March 02, 2023, 09:12:19 am by newbrain »
Nandemo wa shiranai wa yo, shitteru koto dake.
 

Offline mac.6

  • Regular Contributor
  • *
  • Posts: 225
  • Country: fr
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #71 on: March 02, 2023, 08:58:42 am »
NXP OTP licence is only for specific part of the sdk (there is a comprehensive SCR file, look here). Like hab secure boot (binary only), or some middlewares included, Nothing that matters for you unless you want to use these parts.

The only downside for you is that examples are only for the EVK board, so you need to provide your own board/platform files (or adapt the one of the EVK). This is doable but may requires quite a lot of fiddling with chip configuration (pinmux, clocks).

 

Offline SiliconWizard

  • Super Contributor
  • ***
  • Posts: 14488
  • Country: fr
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #72 on: March 02, 2023, 10:37:18 pm »
The Azure stuff, I personally don't care about, so that's fine with me.
Now the secure boot thing is more concerning - they have introduced it in relatively recent versions of the SDK - at least as far as I could tell - and their examples are now using it, so it would take some extra work not to use it. At least, again as far as I've seen. I've developed a couple projects based on the NXP SDK and suddenly I had to change a few things for generating boot binaries.

Now unless you sell millions of products based on that, I sincerely doubt NXP's going to go after you for using secure boot not strictly adhering to their license terms, but YMMV.
 
The following users thanked this post: AaronD

Offline AaronDTopic starter

  • Frequent Contributor
  • **
  • Posts: 260
  • Country: us
Re: i.MX RT1062: Getting Started, on a Raspberry Pi?
« Reply #73 on: March 03, 2023, 01:52:49 pm »
Now the secure boot thing is more concerning - they have introduced it in relatively recent versions of the SDK - at least as far as I could tell - and their examples are now using it, so it would take some extra work not to use it. At least, again as far as I've seen. I've developed a couple projects based on the NXP SDK and suddenly I had to change a few things for generating boot binaries.
:o
Thanks for that warning!  I wonder if an argument for "entrapment" would hold up in court?

Now unless you sell millions of products based on that, I sincerely doubt NXP's going to go after you for using secure boot not strictly adhering to their license terms, but YMMV.
I remember when online music sharing first became popular, and record companies intentionally destroyed random individuals "to make an example".  It was a complete loss to prosecute those cases, but that wasn't the point.  (the industry eventually lost anyway in practicality, but it was still hell for the people who were made an example of)  So :-//
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf