EEVblog Electronics Community Forum
Products => Computers => Programming => Topic started by: DiTBho on April 01, 2023, 04:32:16 pm
-
The Google coral M.2 Accelerator is a small ASIC designed by Google that accelerates Tensor Flow Lite models in a power efficient manner.
It's available in an M.2 module/E-key that includes two Edge TPU ML accelerators, each with their own PCIe Gen2 x1 interface.
- - -
anyone?
any experience?
- - -
tempted to buy one for my new MIPS/EL SBC :o :o :o
-
Some Python examples, here (https://coral.ai/docs/m2/get-started/#requirements) (get started)
-
Required software components? here (https://coral.ai/docs/notes/build-coral/#required-components) more information on building libraries, misc and kernel drivers, to run inference on the Edge TPU
Hardware, any constraints? Sure, being a strong interrupt driven co-processor, whichever SBC you choose, MSI-X is required by the Edge TPU Silicon Engine over PCIe, otherwise without it there's no way to get interrupts and therefore no way to know when operations have completed.
I see, 64 bit is also required as a hard constraint - won't this work on a MIPS32? Do you need MIPS64? :o :o :o
(I think it has to do with large buffers allocated on the stack)
-
What do you want to do with it?
-
world domination .... something ;D
-
world domination .... something ;D
Good luck, you have a lot of competition out there already. :-DD
-
world domination .... something ;D
Good luck, you have a lot of competition out there already. :-DD
this (https://www.youtube.com/watch?v=7edeOEuXdMU)
yeah, already too many people laugh, you have to exercise to laugh louder :o :o :o
-
first crazy app: Imagine having your super secret router inside a full metal case in your camping tent, at a hacker camp meeting - do you really believe that your router disk is securely protected by an internal contact button as simple "tamper proof sensor?
Do you think it is better secured with a bicycle chain lock?
Do you feel it is better secured with 5 Kg of locks?
last time in the nearby tent they showed how to open all kinds of locks
TPUs rely on computer vision and visual deep learning at the edge, so your router can actually "see" who (a human being, but not you!!!) is looking around its metal case and run defensive algorithms(1) :o :o :o :o
(1) send me a message on my Smartphone "Mayday Mayday, this is your router talking, stop hitting on the cos-player girl dressed as TombRider or Trinity or Captain Marvel (input from your taste profile) and get right back to the tent before the hackers crack my rootfs with tons of Trojan horses and all kind of root*hit !!!"
Planned for: next summer!
-
world domination .... something ;D
Good luck, you have a lot of competition out there already. :-DD
this (https://www.youtube.com/watch?v=7edeOEuXdMU)
yeah, already too many people laugh, you have to exercise to laugh louder :o :o :o
I would have posted a video about a real event rather than fictional to illustrate the point, but that probably wouldn't please moderation.
::)
-
first crazy app: Imagine having your super secret router inside a full metal case in your camping tent, at a hacker camp meeting - do you really believe that your router disk is securely protected by an internal contact button as simple "tamper proof sensor?
Do you think it is better secured with a bicycle chain lock?
Do you feel it is better secured with 5 Kg of locks?
last time in the nearby tent they showed how to open all kinds of locks
TPUs rely on computer vision and visual deep learning at the edge, so your router can actually "see" who (a human being, but not you!!!) is looking around its metal case and run defensive algorithms(1) :o :o :o :o
(1) send me a message on my Smartphone "Mayday Mayday, this is your router talking, stop hitting on the cos-player girl dressed as TombRider or Trinity or Captain Marvel (input from your taste profile) and get right back to the tent before the hackers crack my rootfs with tons of Trojan horses and all kind of root*hit !!!"
Planned for: next summer!
OK. :-DD
-
I would have posted a video about a real event
I don't have a video handy yet, but I'll probably record something.
Anyway, aside from the *playful aspect*, these things are made for catching bugs.
For example, two years ago, no one had noticed a terrifying bug on the PCI of a Linux/MIPS router until I had the sick idea of trying to get some weird PCI card to work.
-
MSI-X is required by the Edge TPU Silicon Engine over PCIe, otherwise without it there's no way to get interrupts and therefore no way to know when operations have completed.
While it's usually easy with x86-SBC, with MIPS and PowerPC SBC it's not easy to tell immediately if it's supported or not.
For MikroTik's hw things, Yesterday I opened a Ticket to speak directly with someone :-//
-
Note the "not supported" and "doesn't work" are very different things.
Quite a few Arm and RISC-V SBCs have M.2 but it's usually M-key for SSDs. Some have an E-key for Wifi/BT, but the ones I've looked at only have 1 PCIe lane to them and the Coral needs 2.
-
Good point.
Which restricts the usable set.
-
Note the "not supported" and "doesn't work" are very different things.
With bug-fixing(2) I meant - doesn't work *even* if it's supposed(1) supported(3).
(1) because so declared by its documentation
(2) the most shocking discovery happened with the RSP, since the first bug was the documentation: it said nothing except "PCI 2.1 compliant", so I assumed "full PCI compliant" whereas it was rather "only partially implemented", and that's the annoying part with a lot of SBCs around and modern technology.
(3) with the latest router the documentation didn't say you have to "clear a bit in the controller" before issuing a PCI request, so the SBC was randomly pushing junk into memory with a probability of one in every million IOp of disaster - > took weeks to figure out what was wrong
-
There are three version of Google Coral accelerators for SBC
- Mini PCIe Accelerator, datasheet (https://coral.ai/static/files/Coral-Mini-PCIe-datasheet.pdf)
- M.2 Accelerator A+E key, datasheet (https://coral.ai/static/files/Coral-M2-datasheet.pdf)
- M.2 Accelerator B+M key, datasheet (https://coral.ai/static/files/Coral-M2-datasheet.pdf)
B+M is confusing, since
Key B = sATA
Key M = PCIe x2
Key B+M = sATA or PCIe
so why key B?!?
also, the datasheet says
Features
Google Edge TPU ML accelerator
...
PCIe Gen2 x1 interface
so, is it 1 lane?!? :o :o :o
-
Key A = PCIe x2 and/or USB
Key E = PCIe x2 and/or USB
Key A + E = PCIe x2 and/or USB, but it depends on the internal wiring of the motherboard
=> more confusing => to be avoided if it's possible!
-
https://coral.ai/static/files/Coral-M2-datasheet.pdf
This on-device ML processing reduces latency, increases data privacy, and removes the need for a constant
internet connection :-+
Excellent.
Looks to me like a nice little toy to play with.
-
so, after a lot of personal research, most non-x86 SBCs don't have MSI support over PCIe
it's really frustrating, datasheets don't even mention any detail about the PCIe.
-
Can it implement GPT-4 though? ;D
-
The Google coral M.2 Accelerator is a small ASIC designed by Google that accelerates Tensor Flow Lite models in a power efficient manner.
It's available in an M.2 module/E-key that includes two Edge TPU ML accelerators, each with their own PCIe Gen2 x1 interface.
anyone?
any experience?
This works just fine on Arm64 platforms.
-
This works just fine on Arm64 platforms.
64bit arm :o :o :o :o
which SBC? with m.2? with miniPCIe?
-
so, after a lot of personal research, most non-x86 SBCs don't have MSI support over PCIe
Really? How does that even work?
-
Really? How does that even work?
works but without MSI support.
-
Really? How does that even work?
works but without MSI support.
My point was: MSI(-X) is the only way to do interrupts on PCIe. There are PCI interrupt emulation messages, but it's all MSI based. So either that means it has no interrupts at all, or it means something different than what I understand. It's hard for me to see how a PCIe bus that doesn't support interrupts either meaningfully implements PCIe or supports much of anything.
-
My point was: MSI(-X) is the only way to do interrupts on PCIe. There are PCI interrupt emulation messages, but it's all MSI based. So either that means it has no interrupts at all, or it means something different than what I understand. It's hard for me to see how a PCIe bus that doesn't support interrupts either meaningfully implements PCIe or supports much of anything.
so, according to your reasoning then every single bunch that offers miniPCIe PCIe should also offer MSI :o :o :o
Message Signalled Interrupts are an alternative in-band method of signalling an interrupt, using special in-band messages to replace traditional out-of-band assertion of dedicated interrupt lines.
Supported
- in PCI bus since its v2.2
- later available PCI Express bus, but not always present in every SoC PCIe
- in some non-PCI architectures
Just to re-check again, I contacted the MediaTek' technical support
>The Google Corale M.2 card needs MSI-X support.
> Your datasheet doesn't mention it: is MSI-X supported?
Not supported, MT7621 SoC is PCIe 1.1 host.
-
This works just fine on Arm64 platforms.
64bit arm :o :o :o :o
which SBC? with m.2? with miniPCIe?
With my self designed SBCs using iMX8 SoC or Jetson TX2 module. Both have M.2 slot and PCIe.
-
my self designed SBCs using iMX8 SoC or Jetson TX2 module. Both have M.2 slot and PCIe.
impressed :o :o :o
what about the software side in the userland?
standard SDK from Google (so, python-based) and standard kernel drivers? as described by them?
or something hacked/customized?
and, may I ask for what is the TPU used for?
-
My point was: MSI(-X) is the only way to do interrupts on PCIe. There are PCI interrupt emulation messages, but it's all MSI based. So either that means it has no interrupts at all, or it means something different than what I understand. It's hard for me to see how a PCIe bus that doesn't support interrupts either meaningfully implements PCIe or supports much of anything.
so, according to your reasoning then every single bunch that offers miniPCIe PCIe should also offer MSI :o :o :o
Message Signalled Interrupts are an alternative in-band method of signalling an interrupt, using special in-band messages to replace traditional out-of-band assertion of dedicated interrupt lines.
No, this is wrong. MSI is optional in parallel PCI but mandatory in PCIe. PCIe has no out-of-band interrupt signalling mechanism at all. Go ahead and check: the connector pinout has no interrupt pins. The conventional PCI out of band interrupts (INTA, INTB, INTC, and INTD) are implemented as dedicated messages in PCIe. This is mostly necessary to allow PCI to PCIe bridges, native PCIe devices would not normally use them.
Note that MSI and MSI-X are different. MSI-X allowed more interrupts and multiple interrupt destinations to allow more flexible interrupt routing in multi-processor systems. I don't know when MSI-X became mandatory, it's possible that early PCIe versions did not have mandatory support for MSI-X.
It's also possible that there is some confusion between the interrupt signalling mechanism (which is always MSI in PCie) and how the host interrupt controller lets you configure things. I'm not familiar with the host and OS level configuration.
-
MSI is optional in parallel PCI but mandatory in PCIe. PCIe has no out-of-band interrupt signalling mechanism at all. Go ahead and check: the connector pinout has no interrupt pins. The conventional PCI out of band interrupts (INTA, INTB, INTC, and INTD) are implemented as dedicated messages in PCIe. This is mostly necessary to allow PCI to PCIe bridges, native PCIe devices would not normally use them.
Note that MSI and MSI-X are different. MSI-X allowed more interrupts and multiple interrupt destinations to allow more flexible interrupt routing in multi-processor systems. I don't know when MSI-X became mandatory, it's possible that early PCIe versions did not have mandatory support for MSI-X.
It's also possible that there is some confusion between the interrupt signalling mechanism (which is always MSI in PCie) and how the host interrupt controller lets you configure things. I'm not familiar with the host and OS level configuration.
ah, ok, now I got it, thank you for your clarification ;D
That's probably what they meant when they replied:
MSI yes by default
MSI-X not supported in ePCI v1.1
Now I have to understand why and if the Google Coral strictly really needs MSI-X instead of MSI.
Then I will have to re-check the Linux kernel support for Mediatek MIPS SoCs
-
My understanding of the way this works:
MSI: each endpoint (bus/device/function) gets a single unique memory address. It writes to that address to signal an interrupt. The interrupt number (up to 32) is part of the message payload.
MSI-X: each endpoint is given multiple unique memory addresses, one per requested interrupt. Also, more interrupts are supported (up to 2048)
Message routing is only based on the address not the data. So a given device using MSI can only send interrupt messages to a single group of cores. MSI-X uses distinct addresses for each interrupt source so they can be more easily routed to a particular processor/core.
This is all at the PCIe transaction level. You also need to consider the capabilities of your platform interrupt controller. I guess (I'm not sure) it's also possible that a given platform interrupt controller doesn't support that many interrupts and just combines them into one per device. If a platform does this but your device driver isn't written to gracefully handle this situation then it won't work even if the interrupt signalling is MSI-X. Again, I could be wrong about this, I don't know much about how platform interrupt controllers or the device driver architecture works, I'm more familiar with the PCIe transaction layer.