i.MX RT1062: Getting Started, on a Raspberry Pi?

[elecdonia]

The Tympan library is an expanded reimagining of the Teensy audio library for 32 bit floating point, created for the Tympan hearing aid project.

Thanks! I’ll look into that!

OT:   I’m fascinated by the Tympan project because at my age (73) my personal internal biological carbon-based audio system (my ears) don’t function as well as they did in my 20’s. My thought is to model the human cochlea. Some of the tiny hairs in it act as microphones. They are attached to nerve cells. Others act as tuned resonators, providing gain and equalization in selected frequency bands. Although they don’t feed directly into the auditory nerve cells, they do interact with the “sensing” hairs to amplify/equalize the incoming vibrations. Also important for me is to is to review recent scientific discoveries about the hearing of other species: Dogs and birds have fascinating capabilities in their auditory systems.

[elecdonia]

At this point I would suggest changing the title of this topic to something like:

“Real-time audio processing with RT1062 (Teensy4.1) and RPi Pico”

I’ve been struck by the breadth of audio engineering expertise and raw coding talent possessed by the contributors to this topic. Also, every day I am finding impressive examples of fully functional audio processing systems implemented on Teensy4.1 and RPi Pico hardware. Nothing like this existed 10 years ago.

[elecdonia]

As you can see in the contributor graph, it's not just Paul, but quite a few PJRC forum regulars and contributors in there.  But the code is indeed quite clean, once you get used to the mixed freestanding C/C++ style used.

This is a signature characteristic of the Arduino project: Segregate the C++ code into libraries. Simplify the visible code in applications (e.g. “sketches” in Arduino-speak). I feel this makes the Arduino environment more user-friendly, especially for teaching newbies how to write code.

As a person who mainly coded in assembly language from 1980-2010, my exposure to Arduino helped me gain a thorough understanding of C and C++.

[elecdonia]

The multi-way speaker system that I described earlier is fairly low-power.  16-ohm "junk drawer" speakers, probably bridged, from a 12-24V DC supply.  (yes, I know that's a big range; I'll figure it out later)  I'm currently bridging them from the 12V rail of an ATX supply without a sub, and I could use a bit more than that.  I have a powered sub waiting to go in once I figure out the processing, so I guess I'll see what that does first.  Taking the bass out of the mains is going to help a lot by itself.

Three cheers for working directly with the audio hardware!
One of my long-term activities is repair/refurbishing of home and pro audio gear which was manufactured from 1950 to the present time.
Recent projects:

Repairing home-theater powered subwoofers (Klipsch, Infinity, Boston Acoustics). A very common failure is caused by small electrolytics on the class-D amplifier boards drying out (example: 4.7uF/25V). Also I had to learn how the “BASH” architecture works: Bridged analog output stage powered from a variable voltage DC source which tracks the audio signal level. It’s an odd circuit design. BASH amplifier PC boards look cheesy but they do function after installing decent quality small electrolytics. Note: You should be able to find plenty of powered subwoofers cheap, free,or “on-the-curb.” Faulty drivers are rare. Bad small electrolytics on the class-D amplifier boards are ubiquitous. Most of these “dead” subwoofers are repairable.

Repairing home-theater receivers (Onkyo, Yamaha). Main issue is intermittent TI DSP chips. TI had issues with the metallization layer. Failures didn’t occur until 3-5 years of age. TI redesigned their fab process.  Replacement DSP chips (improved) are available.

Another project idea is for an actual PA, and was originally going to use a 19" rack of small DSP amps that all get the same analog signal to drive a 4-way "true full-range" system.  No separate sub needed.  I'm starting to think about using a Teensy4 / RT1062 for that too: forget the rack entirely, and just mount the custom PCB('s) directly on/in the cabinet.

Somewhere I have docs for my 1980-era 3 or 4-way active electronic crossover with anti-clipping limiters on each output. It could be re-implemented digitally.

[AaronD]

At this point I would suggest changing the title of this topic to something like:

“Real-time audio processing with RT1062 (Teensy4.1) and RPi Pico”

Yeah, I think you're right.  Though I don't remember much about the Pico here, except that it's barely enough for some very small projects like simple synthesizers and the like.

[Marco]

There's still the 128-sample buffer that I'd rather not have (that's probably to allow the audio design to co-exist with a naive sketch without dropping out), so it's not just a matter of changing the datatype.  But I can't imagine it'd be *too* hard to read Paul's source, understand what it does, and write my own from that...once I see the license for myself and like it.

The Tympan library also makes this a little easier to change, you can do it from code instead of having to mess with library recompilation.

I don't know down to what size you can reasonably set audio_block_samples, but I'd just try to set it to two and see from there. At 600 MHz having  to push/pop the register file more often is a lot less relevant than at 48 MHz.

[AaronD]

The Tympan library is an expanded reimagining of the Teensy audio library for 32 bit floating point, created for the Tympan hearing aid project.

I'll have to look at that too!  Thanks!

The license is at the beginning of each file.  It makes accepting contributions from other developers clearer; the license is impossible to miss.

Oh, okay.  I was looking for an overall license that I could read, that covers the entire library, without the specific sources appearing in my history to say that I've been "contaminated" before I find that one of them has some non-compatible restrictions.  Don't know if I'm going to sell something yet or not, but I don't want to get tied up in IP trouble because I've seen something that has that restriction before I made mine.

Tympan does have an overall license file, for the MIT license, which pretty much says I can do anything I want as long as I keep that notice and the license that goes with it.  My understanding is that I can't close my copy of the source and sell that - I have to keep it open - but I think that's okay.  If I do sell something, it'll be the hardware that runs that software, not the software itself.

Even over USB Serial (which on the Linux end has quite a bit of overhead, as it goes through the tty layer), I can sustain well over 200 Mbit/s in one direction.
Even 32-bit 192 kHz raw audiostream is just 6.144 Mbit/s, so the USB is definitely not a problem.

I was more concerned with the USB device library "going to sleep" or "getting stuck in standby" or whatever it does, when the host stops sending audio samples.  That was my trouble with tinyusb.

It would work perfectly well, until the player app had been stopped or even paused for a few seconds and the Linux host's audio system stopped sending silence.  Changing the output device to I2S on the GPIO header, allowed me to put a probe on it to see what the host driver was actually doing, and it does indeed stop all activity and then restart when something starts playing again.  Apparently, tinyusb didn't like that stop and restart, because it worked perfectly before the first stop, but never came back after.

Technically, I could have a background task, maybe aplay from /dev/zero to that card, just to always have something playing, but that's kinda "hacky".  So my concern was that the Teensy library still works after no-data for a while.

In case you haven't realized, RPi Pico only supports USB 1.1, 12 Mbit/s, whereas Teensy 4.0/4.1/MicroMod supports USB 2.0, 480 Mbit/s.

Yes.  I was thinking that the Pico would be enough for decent-quality stereo both ways on USB, and I could get the concepts going on that, and then switch to something with a better PHY for the 8-channel one.  For a while, I was wondering about the RT1010 as a standalone chip, and I would figure out later how to support it, but it looks like the RT1062 with Arduino-like support would be easier.

Teensyduino itself supports various combinations of USB endpoints (like USB HID + USB Serial), as in selectable directly from the Arduino GUI, so multi-endpoint USB is also well supported.  Teensy 4.0/4.1/MicroMod supports 8 bidirectional endpoints on each USB controller.  (They have two OTG/EHCI HS/FS/LS USB cores, core 0 wired to the programming USB connector, with the other on pads in 4.0 and pins in 4.1.)

I was thinking about Audio+HID, which I don't see in the list.  But Audio+Serial can work.  The HID or Serial connection goes to a custom app that I have yet to write, to manage DSP coefficients far beyond what the Audio spec is made for, so I'm still flexible on that.  Either disable the Audio's control functions - don't even report them as available - or have some dummy code that makes it look like they work but they really don't do anything, or maybe they do actually control the master volume while everything including that works in the app...  Several ways to do that.

As I understand, it's a pair of endpoints per function, like Audio or HID, etc., not per channel of Audio.  So even if I made a 32x32 Audio card, it would still use only one endpoint each direction.

There's still the 128-sample buffer that I'd rather not have (that's probably to allow the audio design to co-exist with a naive sketch without dropping out), so it's not just a matter of changing the datatype.  But I can't imagine it'd be *too* hard to read Paul's source, understand what it does, and write my own from that...once I see the license for myself and like it.

The Tympan library also makes this a little easier to change, you can do it from code instead of having to mess with library recompilation.

I would hope that for a real-time mic-to-speaker thing, with the two so close together physically, that it would have a similar constraint to mine, of absolute-minimum inherent latency plus *maybe* an explicit delay of just a handful of samples but no more.  That almost dictates single-sample processing with no buffer at all, and to run the converters as fast as can be afforded to reduce their latency as well.  (high sample rate, not because it sounds any better - it doesn't - but to allow the converters to use less aggressive filters, which in turn cuts the latency way down, more than just the time between samples)

[Nominal Animal]

The license is at the beginning of each file.  It makes accepting contributions from other developers clearer; the license is impossible to miss.

Oh, okay.  I was looking for an overall license that I could read, that covers the entire library

I haven't checked every file in the Audio library, but they tend to be

Copyright 20nn Paul Stoffregen

Development of this audio library was funded by PJRC.COM, LLC by sales of Teensy and Audio Adaptor boards.  Please support PJRC's efforts to develop open source software by purchasing Teensy or other PJRC products.

Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights to use, copy, modify, merge, publish, distribute, sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is furnished to do so, subject to the following conditions:

The above copyright notice, development funding notice, and this permission notice shall be included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

It is sufficient your derivatives include the text from the files you used (once per exact text suffices), in the sources if you provide those, or in the documentation, explain that parts of the product were developed using Teensy Audio Library by Paul Stoffregen and others, under the following conditions: (followed by the above text, but do check each file you use as a basis for deriving your own).

So, not exactly onerous.  You don't need to provide sources of your own versions, only include the permission notice in the documentation (and/or sources, if you provide sources).  This can be considered a derivative of the MIT license.

Tympan does have an overall license file, for the MIT license, which pretty much says I can do anything I want as long as I keep that notice and the license that goes with it.  My understanding is that I can't close my copy of the source and sell that - I have to keep it open - but I think that's okay.

No, you do not need to keep derivative or combined sources open, when using the MIT license.

What you do need to do, is mention in the sources (if you provide sources) or/and in the documentation, that parts of the product were developed using <whatever-MIT-licensed-thing> using the following license: <followed-by-the-license-text>.

I was more concerned with the USB device library "going to sleep" or "getting stuck in standby" or whatever it does, when the host stops sending audio samples.  That was my trouble with tinyusb.

No; as long as the port provides power, Teensy will continue working.

For example, when using USB Serial, (serial) evaluates to True when there is an application that has the serial port open, and to False when there isn't.  (It is useful and kinda necessary when printing debug output.  If no application has the serial port open, printing to it (from Teensy) will eventually block, because unlike Windows, other OSes do not actually discard the information by default.)

It would work perfectly well, until the player app had been stopped or even paused for a few seconds and the Linux host's audio system stopped sending silence.

That just means the player app closes the character device when it's not playing.  You can detect that on Teensy easily, but there is no automagic go-to-sleep tied to it at all.

Teensyduino itself supports various combinations of USB endpoints (like USB HID + USB Serial), as in selectable directly from the Arduino GUI, so multi-endpoint USB is also well supported.  Teensy 4.0/4.1/MicroMod supports 8 bidirectional endpoints on each USB controller.  (They have two OTG/EHCI HS/FS/LS USB cores, core 0 wired to the programming USB connector, with the other on pads in 4.0 and pins in 4.1.)

I was thinking about Audio+HID, which I don't see in the list.

You can modify the core files to export Audio+Keyboard or Audio+Keyboard+Joystick or whatever you need.

In hardware/teensy/avr/boards.txt, you add the new usbtype (USB_AUDIO_HID, for example); then the corresponding USB details into hardware/teensy/avr/cores/teensy4/usb_desc.h.  If you use diff -u and patch, you can keep these changes in a diff file, and submit to Paul after testing, or just keep them as a diff/patch so you can easily upgrade Arduino and Teensyduino to newer versions without having to do the changes by hand.

USB Audio uses two endpoints and three interfaces.  Keyboard uses two endpoints and two interfaces (one for normal keys aka KEYBOARD, one for media keys aka KEYMEDIA).  Joystick and MIDI both use one endpoint and one interface.  RawHID uses one or two endpoints and one interface.  Serial uses two or three endpoints and three interfaces.  If you don't use serial, seremu takes one endpoint and one interface.  Teensy 4.x and MicroMod have 8 bidirectional endpoints, and one is reserved for configuration; so, basically, you can pick any combination that uses at most seven endpoints.
Audio+Keyboard+Serial, Audio+MIDI+RawHID+Serial, Audio+Keyboard+seremu, Audio+MIDI+RawHID+seremu are all possible.

[Marco]

It's been a while since I looked into this space, but just want to point out that for the actual Raspberry Pi there's a new kid on the block too. Elk OS. It's Linux, but with a real time kernel underneath running the actual audio processing (with standard API plugins, VSTx, LV2). The minimum buffer size for which it is compiled is 16, but for all the talk of pros a lot of pro stuff gets done on Macs and the latency there is going to be ten times a 32 sample latency.

[SiliconWizard]

At this point I would suggest changing the title of this topic to something like:

“Real-time audio processing with RT1062 (Teensy4.1) and RPi Pico”

Well, after reading it all, to me it looks more like: "real-time audio processing RT1062 (Teensy4.1) and RPi Pico using an existing library so you don't have to write a single line of low-level code".

While, as I pointed out, it's possible to do quite interesting stuff even on a RP2040 writing your own code and taking advantage of the embedded peripherals, most of what I've read is about existing libraries.

Nothing wrong with that, but there is a definite difference between "what is possible" and "what is possible with existing libraries".
That applies to pretty much any software development too these days it seems, so nothing really specific to this thread. Just a thought. You can ignore that and keep going.

[Marco]

You can take the library down to single sample latency. I doubt all the filters are assembly/intrinsic optimised yet. Plenty of low level coding to be done with existing code bases if it takes your fancy.

Bootstrapping a micro and implementing an interrupt and event loop isn't all there is at low level.

[Nominal Animal]

Most of the Teensy Audio Library features were dictated by the Teensy Audio Adaptor Board (aka Audio shield), which supports I2S (8 channels) or TDM (16 channels), using NXP SGTL5000 chip.  It's evolved quite a bit in the last seven years for different Teensy models (3.1, 3.0-3.6, 4.x), but still keeping the core SGTL5000 chip.  Things like the sample buffer size derive from this use pattern and needs.

So, it is best to not consider the Teensy Audio library as a generic audio library, but more like one dedicated for SGTL5000 but which can be used for other purposes as well.

[elecdonia]

Today PJRC strongly recommends advancing to the Teensy4.1. They warn that their older boards may never be manufactured again (due to chip-a-geddon).
     https://www.pjrc.com/store/teensy41.html
     https://www.pjrc.com/store/teensy3_audio.html
     https://www.pjrc.com/teensy/td_libs_Audio.html

After finishing this post I'm ordering a Teensy4.1 along with its matching audio board. (as of this moment) both are in stock at the PJRC store.

The Teensy 4.1 and its audio board arrived yesterday. I'm eager to get them hooked up and running!

[AaronD]

Most Arduino "cores" include a copy of the appropriate gcc, CMSIS, and newlib tools, in addition to the code that makes them "Arduino.'  You could just copy them to a more dignified-sounding directory, add them to your paths, or whatever.

Since I despise both Arduino (it has its place, but not for my needs) and MCUxpresso, I use just clang + gdb + CMSIS + SDK + Cmake + ninja + VS Code.

I think I'm finally starting to understand what that means.  But compared to the 8-bitters that I'm used to:

I am properly intimidated by the i.MX RT.   It's a complex beast plus a new (to me) programming environment.

This is certainly a complex MCU.
...
I would really suggest starting with the NXP SDK instead. There's a ton of examples.

That's probably excellent advice, but:

I’m a bit surprised that official NXP development tools aren’t supported on RasPi OS. Are those NXP tools for Windows 10 only? Nothing for any flavor of Linux?

That's how it appears to me, but westfw has a good point that I still have to check.  I might be able to use parts out of the Arduino framework, which does run on Linux, and the Pi, to make my own toolchain.

The problem that I see so far isn't so much the complex chip, but the complex toolchain that nobody seems to have a tutorial for.  "Just install this and use it"...which is only for Windows, and I don't have Windows.  And trying to parse the Arduino boilerplate project, which appears to be a unified library for the entire chip that can do absolutely anything and everything that the single "add your code here" file might call for, is making my head spin.

Is there a step-by-step tutorial to using the RT1062 as a standalone chip?  Or close enough that I can find/replace the name and have it work?

Looks like CMSIS is going to be useful, and it even comes with a DSP add-on that does a lot more than just audio.  And the Apache 2.0 license is good too.  Though in reading the source for its filter algorithms, I still see too much reliance on buffers for my near-zero-latency project, so I might just use that part for reference instead.
This sort of middleware is not really necessary for 8-bit projects, so I don't have any experience with it yet, as those are simple enough to just bit-bang the registers as described in their (single, complete!) datasheet.

And since PJRC's MKL02 support chip ties up the debugging lines, it won't be necessary to include that part of the toolchain.  Since I learned originally without that feature anyway, I don't think I need it now either.  (nor is it particularly useful if USB dies when I stop it to inspect memory, or a power-controlled thing goes immediately to 100% and blows up, etc.)


What I'm comparing to:

• Microchip's MPLAB runs everywhere, and is okay if you don't mind NetBeans being stupid.  (seems to be a common problem with all generic IDE's that a vendor-specific thing might be based on)
• AVR didn't have an official IDE when it became popular, so there are lots of tutorials for how to install and use that toolchain with whatever IDE you might have, or CLI-only if you prefer that.  And Code::Blocks has it directly in its New Project Wizard.  I would guess that other standalone IDE's do too.
• Microchip's XC8 compiler is actually a standalone command-line thing, which means that any IDE can use it.  So I've also done some PIC projects with that in Code::Blocks by copying and modifying an existing AVR project to use that toolchain instead.  Then I switched to SDCC for the PIC by following the instructions on that site.  Not as good as the fully-paid version of XC8, but a lot better than the free XC8!

My copy of Code::Blocks (20.03, even after the standard sudo apt {update|full-upgrade|autoremove} - has it been abandoned?) doesn't seem to know about the Teensy 4.x or the RT1062.  I seem to remember things appearing in the New Project Wizard after installing their toolchains, but this one doesn't:

• It has an option for ARM, but it ends up with a short list of dev boards that doesn't include that, and requires me to pick one.
• It also has an option for Arduino (yet another alternative to the official Arduino IDE, I guess), but it only allows the AVR GCC compiler.

The RPi Pico SDK as a CMake project in VSCode feels to me like the perfect blend of simplicity and capability.  The main() function is yours (that's a massively big deal!), and everything the chip does is available as a well-documented and discoverable SDK function with readable source code, but it doesn't have a million files in your face all at once to keep track of like the Arduino boilerplate does.  Granted you're not supposed to look at that boilerplate, but still...
The only thing I don't like about VSCode is that the text editor lacks a few features that Code::Blocks has, and I keep tripping over that.  (insert/overwrite is a big one, for modifying datasets quickly)

Is there a pre-built project somewhere, or a tutorial that makes one, for the RT1062 (or just a quick config-change away) with an empty or trivial main() function, and builds like that with no errors or warnings*, and uses the CMSIS Pack for that chip like the Pico uses its SDK?


_{* I've seen some that you have to add something to make it build.  Umm...add what?!  It's practically DOA for me, and a sign of shoddy work, if you can't even ship a working starting point.  If the most trivial thing possible doesn't even work, how am I to trust anything else?}

[westfw]

Is there a step-by-step tutorial to using the RT1062 as a standalone chip?

I don't think so.  That sort of thing didn't really last past the 80s. :-(
That's part of why Arduino was so revolutionary.
(I guess TI did some pretty in-depth tutorials for their "launchpad" boards.  Usually, such tutorials would be for a specific vendor development board, rather than chip level.)

AVR didn't have an official IDE when it became popular

Sure it did.  "AVR Studio" (though version 4, a custom Atmel IDE) and the later "Atmel Studio" (v5,6,7, based on Visual Studio) existing long before Arduino.  It didn't used to include a C Compiler, which you had to install separately ("WinAVR" if you wanted one that wasn't $$$.  Several other compilers were also "supported.")Windows only, of course.  "Crosspack AVR" existed for MacOS (just the compiler and some generic tools.)

Microchip's XC8 compiler is actually a standalone command-line thing, which means that any IDE can use it.

As is gcc.  I don't think that ANY of the modern IDEs have the compiler intimately linked into the IDE.
A generic arm-eabi-none-gcc toolchain can compile for pretty much any of the ARM Cortex M chips (at least.)  You just have to gather all the chip-specific .h file and libraries sort-of manually.  :-(
You might want to look at "platformio" ( https://docs.platformio.org/en/latest/platforms/nxpimxrt.html ); my understanding is that it lies somewhere in between Arduino and Vendor-specific IDEs.  I haven't used it myself (not yet another IDE), but I believe it's supposed to be more chip-independent than most vendor IDEs.

[mac.6]

Just download the SDK for linux with all toolchain, this will gives you basic SDK with cmake support.
Unzip it, install your own arm-none-gcc (from ARM or your distro), then export variable ARMGCC_DIR (usually /usr) and go into boards/evkmimxrt1064
From there you can compile the demo you want
for example for the hello_world demo:
cd boards/evkmimxrt1064/demo_apps/hello_world/armgcc
./build_all.sh

Oh yes there is a ton of different configuration for each demo, that's why it's a big step from 8bits micro.

[AaronD]

I’m a bit surprised that official NXP development tools aren’t supported on RasPi OS. Are those NXP tools for Windows 10 only? Nothing for any flavor of Linux?

That's how it appears to me, but westfw has a good point that I still have to check.  I might be able to use parts out of the Arduino framework, which does run on Linux, and the Pi, to make my own toolchain.

Just download the SDK for linux with all toolchain, this will gives you basic SDK with cmake support.

Where?

Is there a step-by-step tutorial to using the RT1062 as a standalone chip?

I don't think so.  That sort of thing didn't really last past the 80s. :-(
That's part of why Arduino was so revolutionary.

Huh.  I was getting started with AVR in the late '00s / early '10s.  (PIC came before that, with a 3rd-party toolchain that someone gave me for free)  Google gave me a bunch then, to build a project from scratch for AVR.  I just picked one, and it worked.

I had just gotten good at that when the first Arduino came out.  The bare chip on a custom PCB was so easy for me that I never saw the point.  And I still don't, except for making it "brick-proof".  I can understand a basic, know-nothing beginner programming with busy-loop delays, but once you see that working, I think (quite strongly) that the next step should be to convert that into a clearly non-blocking state machine that polls flags instead.  Everything else is then based on that.

Not making that transition early, is a major failure as I see it, especially when applications like mine come along that need to control the timing absolutely precisely, and they just can't.  Not obviously anyway.  The Arduino framework tries to do absolutely everything while keeping the busy-loop delays, and that leads to some concerns about latency and throughput.


But I did just think of something:

The Arduino framework does handle this chip well already (of course it does, it's aimed at beginners!), and there's a remote possibility of open-sourcing a project or two in addition to selling the custom PCB that it runs on.  So if customers are going to modify it, then the Arduino framework might be the best thing after all, and I just need to see if it'll work as-is.  Possibly yes.

I'd still write my own Audio DSP code, and not use the library that already works but is limited to a specific chip that can't do what I need.  But if I can satisfy the timing with cycles to spare, then that's really all that matters.

There are only 2 lines of code in the Arduino's main loop - the loop() function and an event handler called yield() - and I was concerned about the event handler being bloated and not being able to pare it down to where the rest of the code could keep up.  (96kHz audio or faster, with no buffer)  Maybe it's okay?  I could just comment the yield() call out, but what problems does *that* cause?!  (besides needing to either ship my entire custom framework or tell customers how to modify theirs)

Trying to avoid customer headaches, and gotchas for the library functions that I do want to use, I wonder how to minimize the event handler's effect while still having all of the functionality that I need?  I'm perfectly okay in principle to have my own version of it, probably calling the same API myself, so that I KNOW what the timing is going to be...but that doesn't really work well from a customer's standpoint.


In all of this, I've been trying to avoid interrupts as much as possible, not because I'm scared of them, but as a possible holdover from the 8-bit world, where I really want to reserve them for things that *really do* need IMMEDIATE attention.  In the 8-bit world, the interrupt controller disables itself completely until the ISR returns, so that the first interrupt that fires, regardless of priority, runs all the way to completion before another one has a chance.  Thus, using interrupts for everything kills the ability to time them precisely, especially when you have an ISR that takes forever to finish!
(big motivation there, to have everything non-blocking, with polled flags for things that would otherwise be low-priority interrupts)

But maybe I do want the DSP code to be entirely interrupt-driven.  If it were the highest priority (or the only) interrupt in the system, then that would guarantee the timing, regardless of what the rest of the code is doing or how bloated it might be.  Even if I end up spending 90% of the CPU time in that one ISR, that's probably okay.  With a 600MHz clock, that leaves me with effectively 60MHz for everything else, fairly evenly spaced, which is *still* an improvement on the 8-32MHz clocks that I'm used to from the 8-bit world!

I also like to have a "CPU load" signal somewhere that I can put an oscilloscope on.  In this case, it might be the Teensy's on-board LED, which I would keep for my custom board along with the other Teensy-related stuff.  Turn it off at the start of the DSP ISR and on at the end, so if you can still see it, the audio is okay.  If it goes out completely, you're trying to do too much.  And its frequency can be checked on a 'scope, as the actual sample rate.


So I guess I just need to get the classic "Hello World" to a terminal via USB (in lieu of a "real" debugger), using the stock Arduino framework, build from there while keeping the stock framework, and see when or even *if* it breaks.

[newbrain]

Where?

Here:
https://mcuxpresso.nxp.com/en/builder?hw=MIMXRT1064xxxxA

You need to register an NXP account to get there, though.

Include all the extra libraries you need, (e.g. FreeRTOS or Azure RTOS, Arm DSP etc.), select your OS and the toolchain (or use All) and after a short wait you'll be able to download the package.
There should be plenty of examples.

[AaronD]

Where?

Here:
https://mcuxpresso.nxp.com/en/builder?hw=MIMXRT1064xxxxA

You need to register an NXP account to get there, though.

Include all the extra libraries you need, (e.g. FreeRTOS or Azure RTOS, Arm DSP etc.), select your OS and the toolchain (or use All) and after a short wait you'll be able to download the package.
There should be plenty of examples.

Does it work for Linux on a Raspberry Pi?  And does it have a hobby-friendly license and pricing (free and permissive) that also allows me to sell my project if I decide to do that later?

I never saw any of that when I was looking.  Windows only, and their chip-specific CMSIS library comes with a license file that makes me wonder if NXP ends up owning my project instead of me.
https://mcuxpresso.nxp.com/cmsis_pack/repo/NXP.MIMXRT1062_DFP.16.0.0.pack

[SiliconWizard]

Where?

Here:
https://mcuxpresso.nxp.com/en/builder?hw=MIMXRT1064xxxxA

You need to register an NXP account to get there, though.

Include all the extra libraries you need, (e.g. FreeRTOS or Azure RTOS, Arm DSP etc.), select your OS and the toolchain (or use All) and after a short wait you'll be able to download the package.
There should be plenty of examples.

Does it work for Linux on a Raspberry Pi?  And does it have a hobby-friendly license and pricing (free and permissive) that also allows me to sell my project if I decide to do that later?

All you need is Cmake and arm-none-eabi-gcc, both being available on just any Linux distro I can think of. The rest is just source files, you do not need anything else.

I never saw any of that when I was looking.  Windows only, and their chip-specific CMSIS library comes with a license file that makes me wonder if NXP ends up owning my project instead of me.
https://mcuxpresso.nxp.com/cmsis_pack/repo/NXP.MIMXRT1062_DFP.16.0.0.pack

Most of the SDK is under a BSD-3 license. The CMSIS has a specific license that I admit I have never really had a look at, but if anything CMSIS is ARM stuff and not NXP. And pretty much all vendors of ARM-based MCUs do distribute CMSIS, and I have never encountered, or heard of, any licensing issue about that. I think we would have by now.

[newbrain]

Does it work for Linux on a Raspberry Pi?  And does it have a hobby-friendly license and pricing (free and permissive) that also allows me to sell my project if I decide to do that later?

I never saw any of that when I was looking.  Windows only, and their chip-specific CMSIS library comes with a license file that makes me wonder if NXP ends up owning my project instead of me.
https://mcuxpresso.nxp.com/cmsis_pack/repo/NXP.MIMXRT1062_DFP.16.0.0.pack

I don't know where you looked, but:

• The SDK is a source file collection - there is nothing Windows specific in there as Silicon Wizard said.
• CMSIS is not from NXP, it's Arm SW and is distributed under an Apache 2.0 license (more below)
• There is, in the SDK root, an optional NXP license, which might have scared you. See below for more details.

About Apache 2.0:
This is probably the best permissive, not copyleft, Open Source licenses around, striking the right mix of freedom and protection. FSF advise its use in case one doesn't want to go the GPL copyleft way.
You are able to distribute your SW in source or binary form and with a license (also non free) of your choice, provided that you abide by Apache's very mild requirements.
Remember that it's not compatible with GPL2 (or earlier) licensed code, only with GPL3, as it imposes additional restrictions not covered in GPL2.

About the NXP OPT license:
Yes that's scary. ST has a similar one.
BUT! If you carefully read it, you'll find that the license can be superseded by the specific component own licenses, and by what is listed in the  SW-Content-Register.txt file.
If you go there, you'll find that all the SDK drivers are licensed by NXP under the very permissive BSD-3 license. Other parts and libraries have their own licenses, e.g. MIT for FreeRTOS, mostly BSD-3 and MIT.
If you are using one of the non-MIT, non-BSD, or non-Apache components, check its license, but they are very few.

Disclaimer: IANAL, TINLA. OTOH, I have worked a lot with licensing issues of internal and third party SW as a FOSS product owner.

EtA: I'd be very careful around Azure RTOS. In SW-Content-Register.txt it points to NXP OPT license file, which in turn points to the terms of Microsoft license. Now, Microsoft's license for Azure RTOS is quite permissive, and the iMX RT 10xx is part of the LICENSED-HARDWARE.txt file, but it's not one of the standard FOSS licenses, and might have compatibility problems with some of them. If you intend to use it, get expe(nsive)rt advice, not from some rando on a forum.

EtA: Merci à Mac.6 pour le TL;DR 

[mac.6]

NXP OTP licence is only for specific part of the sdk (there is a comprehensive SCR file, look here). Like hab secure boot (binary only), or some middlewares included, Nothing that matters for you unless you want to use these parts.

The only downside for you is that examples are only for the EVK board, so you need to provide your own board/platform files (or adapt the one of the EVK). This is doable but may requires quite a lot of fiddling with chip configuration (pinmux, clocks).

[SiliconWizard]

The Azure stuff, I personally don't care about, so that's fine with me.
Now the secure boot thing is more concerning - they have introduced it in relatively recent versions of the SDK - at least as far as I could tell - and their examples are now using it, so it would take some extra work not to use it. At least, again as far as I've seen. I've developed a couple projects based on the NXP SDK and suddenly I had to change a few things for generating boot binaries.

Now unless you sell millions of products based on that, I sincerely doubt NXP's going to go after you for using secure boot not strictly adhering to their license terms, but YMMV.

[AaronD]

Now the secure boot thing is more concerning - they have introduced it in relatively recent versions of the SDK - at least as far as I could tell - and their examples are now using it, so it would take some extra work not to use it. At least, again as far as I've seen. I've developed a couple projects based on the NXP SDK and suddenly I had to change a few things for generating boot binaries.

Thanks for that warning!  I wonder if an argument for "entrapment" would hold up in court?

Now unless you sell millions of products based on that, I sincerely doubt NXP's going to go after you for using secure boot not strictly adhering to their license terms, but YMMV.

I remember when online music sharing first became popular, and record companies intentionally destroyed random individuals "to make an example".  It was a complete loss to prosecute those cases, but that wasn't the point.  (the industry eventually lost anyway in practicality, but it was still hell for the people who were made an example of)  So