Tiny MCU architectures: AVR, PIC, 8051, STM8, MSP430, custom RISC...

[josuah]

A lot of interesting approaches, and promising low-power, low-price, high-reliability (in various amount) in the 8-bit world.
I did not find a place that was comparing the architecture of these MCU, so I am collecting wisdom and trying to dive down this littlebig world and report what I found.

Note: I am not curious about which is better, just trying to understand what makes an 8-bit instruction set modern, and what is the way followed by 8-bit MCUs.

AVR is said to be a newer architecture, has a maintained GCC port (it was about to be dropped but survived!).
It has an interrupt system requiring the push all registers used by the ISR, which I think GCC can minimize as an optimization.

8051 have multiple register banks which dodge that problem by switching to a new set of registers while entering an interrupt and going back to the usual register set when leaving.
12T instructions (12 clocks ticks per CPU instruction) for everything... only for very old 8051, and 1T for most instructions for the modern 8051.

PIC uses paging for accessing RAM, instead of concatenating two registers (the "X", "Y", "Z" registers on AVR, or DPTR on 8051).
Mostly used with assembly, or a proprietary C toolchain or an endangered(?) SDCC compiler port.

MSP430, I do not know much about it, except it is a 16-bit arch which avoids some of the problems of 8-bit registers. I suppose 16-bit address space.

STM8 has interesting features such as having a single address space ("modified Harvard"?) for everything, I do not know much more.

PicoBlaze which is FPGA-Only is taking an interesting approach to use fewer LUT. https://en.wikipedia.org/wiki/PicoBlaze#Architectural_notes
But noone producing any of these on a foundry AFAIK.

Now in all of this, something I wonder: Why is AVR said to be a modern architecture than, say, 8051?

* The fact that it has 3 memory pointers? X, Y, Z vs just DPTR
* No limitation to just an accumulator register? Although does it much change anything in practice outside compiler optimization?
* The fact that it is 1T for most things? Which is also true for modren 8051
* A simpler instruction set encoding scheme? Like how RISC-V is said to be modern for having a easy to implement instruction decoding
  Although looking at AVR and 8051, I find the AVR a bit more complex and irregular
  https://en.wikipedia.org/wiki/Atmel_AVR_instruction_set#Instruction_encoding
  https://en.wikipedia.org/wiki/Intel_8051#Instruction_set
* The limited set of instructions (RISC)? Recent 8051 are hardwired rather than microcoded (C8051 from these same EFM8BB), and 128 of the 256 instructions are for specifying the register to work on.

Last but not least, any hint in what makes an 8-bit power-efficient (how to get the core smaller, spend less time doing more things to go back to sleep quicker using less power, etc...) and simple to implement (say, on an FPGA) is welcome!

Thank you!

[Picuino]

The microcontroller core is important, but not the only thing. The peripherals matter just as much, if not more.

AVR has a superb core and a port of GCC as a compiler.

The PIC core doesn't look as good to me, but PICs have a great peripheral system.

Now that Microchip has bought Atmel, some AVRs are appearing with PIC peripherals, which in my opinion is the best combination that could be made.

[rhodges]

The STM8 is my favorite 8-bit MCU. It has a nice set of addressing modes, and the SP + offset is very handy. Writing to Flash and EEPROM is super easy. The interrupt vector table makes ISRs straightforward. Programming the chips is also fast and easy with ST-LINKv2 USB adapters.

[Kleinstein]

The actual CPU part of a µC is usually only a small part of the chip. So the core size is less relevant. This makes it possible the 16 bit MSP430 to compete with the 8 bit ones. often ADC and similar mesured values are more like 10-12 bit wide and thus need 2 byte operations. So the 16 bit CPU makes absolute sense and indeed saves some hassles, e.g. with 8 bit acess to 16 bit counters or 10/12 bit ADC results.
With plenty of transistors available a small CPU is less important, it is now more about relatively small code a compiler can produce.
How much power a µC uses depends a lot on the production technology used - the instruction set is more like a small part here, though it does have some effect.
As an odd side effect the chips with more transistors may consume less power as they tend to use smaller strucktures.

The AVRs are relatively new (the 8051 and PICs (e.g. PIC16) are quite old and from times when the transistors in the actual CPU mattered a bit more).
To make a GCC port easy the AVR CPU has quite a lot of similar CPU registers and avoids the RAM / register paging of the 8051 / PIC than can make programming quite a hassel.
However the separate code / RAM adress spaces also causes some hassels for the compiler / programmer.
The more RISC like commands (seprate load/store and ALU commands) also works well with the opimization of modern compilers made with Risc processors in mind. The compilers have also changed: In the early days the stack oriented TMS1000 was easy for a compiler.

AFAIK a large point to make the AVRs quite popular was also that they were relatively fast to include flash memory instead of OTP memory.
Having to save the registers by the ISR code is a common solution, it keeps the CPU simple and in most cases only some registers are needed and this part is more like easy for the compiler.

[PCB.Wiz]

A lot of interesting approaches, and promising low-power, low-price, high-reliability (in various amount) in the 8-bit world.
I did not find a place that was comparing the architecture of these MCU, so I am collecting wisdom and trying to dive down this littlebig world and report what I found.

Note: I am not curious about which is better, just trying to understand what makes an 8-bit instruction set modern, and what is the way followed by 8-bit MCUs.

It is useful to include the opcode size range, for MCU comparisons.
eg PIC's vary with 12 and 14 and 16b opcodes, for various models, and the tiny PICs have very limited HW stacks.
8051 have 1,2,3 byte opcodes

The oldest NMOS 8051's had a 6 clock phase, and needed 12 or 24 clocks for most opcodes.
The more modern 1T 8051 started with mostly 1 clock per byte, but some recent models use a 32 bit fetch, and on those models a 24bit opcode can execute in 1 sysclk. 


You could also include parts like Zilog Z8 (8b) and Siemens (infineon) C166 (16b)   FWIR those both used a register frame pointer, which allowed register banks to be moved across RAM space.
That expanded on the 8051's 4 register banks for interrupts.

Last but not least, any hint in what makes an 8-bit power-efficient (how to get the core smaller, spend less time doing more things to go back to sleep quicker using less power, etc...) and simple to implement (say, on an FPGA) is welcome!

power-efficient is mostly the process.
getting the core smaller usually means getting it simpler.  eg Multiply and Divide are nice to have but costly in core area.

[josuah]

Glad to see this thread sparkling with answers. Thank you all!

The microcontroller core is important, but not the only thing. The peripherals matter just as much, if not more.

The actual CPU part of a µC is usually only a small part of the chip.

So this would mean quite some room to implement something efficient even while trying to save as much space as possible.
To quote another thread:

Quite honestly, I don't really care what core a MCU have. It's like having 30 peripherals on it, and the 31st is the core.

Now that Microchip has bought Atmel, some AVRs are appearing with PIC peripherals, which in my opinion is the best combination that could be made.

I forgot to evoke how many peripheral the PIC range had! It feels like they are adding a CPU core to an ASIC more than adding a peripheral to a MCU.

The STM8 is my favorite 8-bit MCU. It has a nice set of addressing modes, and the SP + offset is very handy. Writing to Flash and EEPROM is super easy. The interrupt vector table makes ISRs straightforward. Programming the chips is also fast and easy with ST-LINKv2 USB adapters.

Really looks like https://jaycarlson.net/ said, an ARM in disguise.
I need to try them some day!

modern compilers made with Risc processors in mind. The compilers have also changed [...] only some registers are needed and this part is more like easy for the compiler.

Interesting: the user of assembly is moving from text editor to compiler, so the MCU would likewise evolve target their new user (copmilers).

It is useful to include the opcode size range, for MCU comparisons.

Which I totally forgot.

You could also include parts like Zilog Z8 (8b) and Siemens (infineon) C166 (16b) FWIR those both used a register frame pointer, which allowed register banks to be moved across RAM space.
That expanded on the 8051's 4 register banks for interrupts.

Thank you! I was wondering about how it 8051 came-up with that strategy.

power-efficient is mostly the process.

So power efficiency would then not be so tightly coupled with architecture... Interesting.

Maybe along with some features to permit jumping to low-power modes and staying as long as possible https://ww1.microchip.com/downloads/en/Appnotes/AN2515-AVR-Low-Power-Techniques-00002515C.pdf#page=13

getting the core smaller usually means getting it simpler.  eg Multiply and Divide are nice to have but costly in core area.

So a matter of trade-off at least as much as implementation.

[mariush]

Not sure if you've heard/read  Jay Carlson's "The amazing 1$  microcontroller" article / post : https://jaycarlson.net/microcontrollers/

He goes through 21 microcontrollers under 1$ and discusses performance, differences, ease of compiling and other things.

[josuah]

Not sure if you've heard/read  Jay Carlson's "The amazing 1$  microcontroller" article / post : https://jaycarlson.net/microcontrollers/

He goes through 21 microcontrollers under 1$ and discusses performance, differences, ease of compiling and other things.

It was a very good read! I read it about 2 times. Well 3 times now.

China doesn’t care about patents, just accessibility and cost. Can’t implement a core when you don’t have the design files. So everything was 8051 because the designs were readily available and free.
-- https://hackaday.com/2022/11/09/chinese-chips-are-being-artificially-slowed-to-dodge-us-export-regulations/#comment-6530447

[PCB.Wiz]

You could also include parts like Zilog Z8 (8b) and Siemens (infineon) C166 (16b) FWIR those both used a register frame pointer, which allowed register banks to be moved across RAM space.
That expanded on the 8051's 4 register banks for interrupts.

Thank you! I was wondering about how it 8051 came-up with that strategy.

My wording may have been poor. The 8051 is older and used 4 register banks, and 2 bits to select which bank was used, but those 4 banks were fixed in RAM space.
The Z8 and C166 are newer, and the improved technology allows them to have larger multi-port RAM and so they have more bits to position the registers in RAM space.

power-efficient is mostly the process.

So power efficiency would then not be so tightly coupled with architecture... Interesting.

Maybe along with some features to permit jumping to low-power modes and staying as long as possible https://ww1.microchip.com/downloads/en/Appnotes/AN2515-AVR-Low-Power-Techniques-00002515C.pdf#page=13

Yes. Modern MCUs have multiple internal Oscillators with lower frequency ones helping snooze control.
Many also have on chip regulators, and those have control bits to trade off quiescent power vs core speed.
They also specify wake-up times from the low power modes.

On the topic of power, one of my peeves has been the poor pre-reset power specs of even recent MCUs - many take mA when held in reset, as the default high speed oscillator is on and a lot is running.

I see just recently, Nuvoton has defined a 8051 variant part with low power pre-reset. Tho their 200uA does not seem too low, maybe that is simply a power optimised HFOSC trade off  ?
they spec 7.3728 MHz MIRC oscillator with variation ±10 % within all temperature range
These days 10% is not great and 7MHz is lowish, but could be chosen if power were the main focus

https://www.nuvoton.com/products/microcontrollers/8bit-8051-mcus/low-power-mug51-series/

•   Power Management
    -   Integrated with Power-on Reset, Brown-out Reset and Low voltage Reset
    -   Normal run: 1.08 mA at 7.3728 MHz
    -   Power-down: 1 μA
    -   Power-on before Flash memory is initialized：200 μA
    -   Supports wake up from Power-down mode by: ACMP, GPIO, WDT, TIMER, UART, I²C, SPI, PDMA, PWM

 they say Target application： Stylus pen RFID card  so they must have a specific large customer ?

It would also be useful for mains-dropper designs, and solar cells where source is effectively current only.

[brucehoult]

MSP430, I do not know much about it, except it is a 16-bit arch which avoids some of the problems of 8-bit registers. I suppose 16-bit address space.

Essentially a DEC PDP-11 expanded to 16 registers instead of 8, and more instructions, in the same fixed-size 16 bit opcode by cutting the available addressing modes, especially for the result of the instruction.

Apparently they are very low power. And for sure more compact code and a lot faster than 8 bit CPUs if you actually have to deal with 16 bit quantities or larger amounts of RAM.

If you are talking about stand-alone microcontroller chips then the actual silicon inside is a relatively small part of the total cost, vs for example packaging and testing, and even 32 bit CPUs are now price-competitive. e.g. see the other thread about $0.10 RISC-V microcontrollers from WCH.

[NorthGuy]

This is not a reasonable comparison. You list different totally unrelated features for different MCUs. You need to get a single feature which you dim is important and compare it accross different MCUs. Then proceed to the next important feature and so on ...

Modern is something that appeared recently, as opposed to ancient which appeared in some relatively distant past. There's nothing to consider here except the release date.

[josuah]

My wording may have been poor. The 8051 is older and used 4 register banks, and 2 bits to select which bank was used, but those 4 banks were fixed in RAM space.
The Z8 and C166 are newer, and the improved technology allows them to have larger multi-port RAM and so they have more bits to position the registers in RAM space.

Thank you for clarifying, it was me who read it the other direction.

On the topic of power, one of my peeves has been the poor pre-reset power specs of even recent MCUs - many take mA when held in reset, as the default high speed oscillator is on and a lot is running.
[...]
It would also be useful for mains-dropper designs, and solar cells where source is effectively current only.

If the chip keeps resetting due to brown-out, it would drain more power if pre-reset uses a lot of power. I never thought about measuring the power before chips start.

I see just recently, Nuvoton has defined a 8051 variant part with low power pre-reset.

They also look like having DMA support. That reminds me of http://elm-chan.org/docs/avr/avrdma_e.html which uses DMA through an external RAM chip.

These days 10% is not great and 7MHz is lowish, but could be chosen if power were the main focus
[...]
they say Target application： Stylus pen RFID card  so they must have a specific large customer ?

After all, if a custom design just for one client can be interesting to someone else, why not sell it to everyone...

[josuah]

This is not a reasonable comparison. You list different totally unrelated features for different MCUs. You need to get a single feature which you dim is important and compare it accross different MCUs. Then proceed to the next important feature and so on ...

This is true.

Modern is something that appeared recently, as opposed to ancient which appeared in some relatively distant past. There's nothing to consider here except the release date.

I was fooled into thinking this: earlier designs used the best of what could be done, with newer designs adding "well-known improvements", with a rather linear evolution where date tells mostly which step of the common evolution we are at.
But discovering further, it looks like it is not as homogeneous and consensual, and different approaches are getting interesting performances in different cases.

[josuah]

Essentially a DEC PDP-11 expanded to 16 registers instead of 8, and more instructions, in the same fixed-size 16 bit opcode by cutting the available addressing modes, especially for the result of the instruction.

A PDP-11 fitting in a small corner of a 3mm x 3mm package!

Apparently they are very low power. And for sure more compact code and a lot faster than 8 bit CPUs if you actually have to deal with 16 bit quantities or larger amounts of RAM.

The MSP430 parts are 16-bit processors  They are von Neumann architecture, and very low power.

I realize I mistook 8-bit for low-power. Afterall, it is only a matter of data bus size, not sure that takes so much power just for that point.

If you are talking about stand-alone microcontroller chips then the actual silicon inside is a relatively small part of the total cost, vs for example packaging and testing, and

It looks like I was mislead about the importance of the CPU core within an MCU.

even 32 bit CPUs are now price-competitive. e.g. see the other thread about $0.10 RISC-V microcontrollers from WCH.

Astonishing.

[bson]

MSP430, I do not know much about it, except it is a 16-bit arch which avoids some of the problems of 8-bit registers. I suppose 16-bit address space.

Many are actually MSPX, which has a 20-bit address space but is fully backwards compatible.  The MSPX core also has a slightly improved instruction set.  But, personally, if a project doesn't fit in a 16-bit address space I'd rather use an ARM based processor than deal with address space segmentation hassles.  But I usually enable the MSPX extensions for improved code generation, only with the plain 16-bit space.

I think where the 20-bit space might make sense is if you have an existing design and just have to squeeze a bunch more stuff in there that simply can't fit.  With some adhocery sprinkled with ugly hacks it can be made to fit and work - and ship in a timely manner.  It'll be on a downward slope of code maintenance hell though, so better only happen to something that is not going to need more than the occasional bug fix ever.

[NorthGuy]

I was fooled into thinking this: earlier designs used the best of what could be done, with newer designs adding "well-known improvements", with a rather linear evolution where date tells mostly which step of the common evolution we are at.
But discovering further, it looks like it is not as homogeneous and consensual, and different approaches are getting interesting performances in different cases.

By any means, modern doesn't necessarily mean better. Comparing performance of different MCUs is at best difficult. There are little things here and there which may affect performance under certain conditions, but may be totally irrelevant in other cases. There's lots of opinions, but they're usually worthless - people like what they use (whatever it is) and they will tell you that this is the best.

[westfw]

Why is AVR said to be a modern architecture than, say, 8051?

It's designed to better support high level languages.  Multiple index registers is part of that.  Many general purpose registers (as in - ALU operations can occur between any two registers) is part of that.  A large stack is part of that.  Taking ideas from RISC research is part of that (in particular: all alu operations are between registers, only load/store operations access memory.)  Not having a limit of 256bytes of "internal RAM" (with "external RAM" access limited to a couple of instructions) is part of it.

8051 have multiple register banks
most modern 8051 are 1T
Recent 8051 are ...

Better be careful.  You sound like you're looking at "some" modern 8051 variants and assuming that "many" 8051s have those features.  My impression is multiple register banks and 1T 8051s are relatively rare (6T seems common, though.) (oh wait.  You mean R0-R7, and not AC, PSW...)
(Hmm.  Anyone know of a table somewhere that lists current 8051-based processors along with which "modernizations" they support?)

A simpler instruction set encoding scheme? ...
  Although looking at AVR and 8051, I find the AVR a bit more complex and irregular

The AVR instruction set is annoying to encode/decode by human (sometimes a register field in an instruction is 4 bits, sometimes it's 5 bits, sometimes a field is spread across non-contiguous bits, etc.  But it's no more complex than others to a compiler or to the hardware.  (another RISCy principle: design the instruction set and encoding for compilers, not for humans.)

[PCB.Wiz]

8051 have multiple register banks
most modern 8051 are 1T
Recent 8051 are ...

Better be careful.  You sound like you're looking at "some" modern 8051 variants and assuming that "many" 8051s have those features.  My impression is multiple register banks and 1T 8051s are relatively rare (6T seems common, though.) (oh wait.  You mean R0-R7, and not AC, PSW...)
(Hmm.  Anyone know of a table somewhere that lists current 8051-based processors along with which "modernizations" they support?)

All 8051's since the original intel release, have multiple register banks, (4) and 2 bits in the PSW select the bank.
Intel did a revision to support 4 interrupt priority levels, up from 2, and that has propagated into many/most new designs too.

1T 8051's are by far the most common new devices in 2022, and 6T parts are very much NRND and EOL.

The differences in the 1T parts in 2022 now comes down to how wide their fetch from CODE is, so they are not all the same

STC  1T parts do a 32 bit fetch, and that means most 2,3 byte opcodes are also all 1T
SiLabs EFM8BB52 parts have 4 level interrupt priority, and their speed is mostly 1T per byte, but they have a somewhat higher SysCLK rating than STC.

However, not all vendors are fully clear on their opcode timings. 
I can find SinOne timings for old SC91F parts (~ 1T/Byte) but not for their new SC95F series, that the prose suggests is faster.

[josuah]

By any means, modern doesn't necessarily mean better.

I will have to get used to it.

Comparing performance of different MCUs is at best difficult. There are little things here and there which may affect performance under certain conditions, but may be totally irrelevant in other cases. There's lots of opinions, but they're usually worthless - people like what they use (whatever it is) and they will tell you that this is the best.

Thinking again, criterion for selecting 8-bitters it is more likely a combo of peripherals, price, power, memory, and availability than actual core architecture and performance.
All things equal remains personal preferences and knowledge already invested to understand each.

It's designed to better support high level languages. [...] is part of that [...].

I have read about some 8-bit MCU architectures being harder to optimize by C compilers, and I would not wish to write too complex application logic in assembly.

encoding for compilers, not for humans

This sounds like avr-gcc is accounting for a good part of AVR popularity.

Essentially a DEC PDP-11 expanded to 16 registers.

All 8051's since the original intel release, have multiple register banks, (4) and 2 bits in the PSW select the bank.
Intel did a revision to support 4 interrupt priority levels, up from 2, and that has propagated into many/most new designs too.
1T 8051's are by far the most common new devices in 2022, and 6T parts are very much NRND and EOL.

I am surprised to see how far ago these design root from, most likely very different to today's iterations.
It might be about giving solutions people were already used to.

[brucehoult]

I am surprised to see how far ago these design root from, most likely very different to today's iterations.
It might be about giving solutions people were already used to.

Many old designs are rubbish, but some ideas have stood the test of time.  Equally, new designs from people who have not carefully studied the best of the past are very likely to be rubbish too.

PDP-11 is one of those classic designs that works very well with single-issue non-pipelined machines with fast RAM (e.g. microcontrollers with SRAM). If you know it you will see very strong echoes in M68k (and more recent ColdFire), MSP430, and SuperH. Those successors use various techniques to expand it from 8 to 16 registers:

- M68k: 8 data registers plus 8 address registers, with when set is used implicitly determined by the instruction (plus some doubling up of opcodes)

- MSP430: expand register fields from 3 to 4 bits, decrease addressing mode fields from 3 bits to 2 (src) or 1 (dst) bits.

- SuperH: expand register fields from 3 to 4 bits, only load/store have addressing modes, arithmetic is only on registers.

In addition, M68k and SuperH expand it from 16 to 32 bits.

You could also argue that ARM is PDP-11 influenced (maybe via M68k).


Features that the above generally share (not all have them all):

- very similar condition codes and compare/branch instructions

- similar sets of addressing modes, including especially register indirect with pre-decrement and/or post-increment

- including SP and PC as general purpose registers, with addressing modes applied to them to get functionality such as push, pop, immediate and absolute addressing that require dedicated instructions in other ISAs.

[westfw]

This sounds like avr-gcc is accounting for a good part of AVR popularity.

Yes, I would say so.  Also avr-g++ (as used in ... Arduino.  But there aren't many 8bit CPUs that have C++ compilers available.  Not that everyone agrees that it's a good idea.  Or that what avr-g++ implements is "real C++")

It's pretty nice to program in assembly language as well.  But I wouldn't want to have to decode a binary dump of the program memory!  (fortunately, that's rarely necessary these days.)
I'd say that the instruction set is about half "real" RISC, and half "extensions for microcontroller use" (like single-instruction pin setting and bit twiddling.)
(And you look at the code produced by the compiler, and say "that's almost as I could do in assembly", instead of "eww!  Well, it fits and it's good enough, I guess."  (which is how I feel about C produced by PIC compilers, and I suspect 8051 is similar.))

1T 8051's are by far the most common new devices in 2022

Who cares about "new devices"?  My impression is that the vast majority of 8051-architecture chips sold these days are actually Asian System-on-Chip things like MP3 players, or Mouse controllers, or flash memory controllers, or ... etc.  Or "legacy" chips that aren't new at all (Microchip, for example, sells a whole range of 8051 chips that they've acquired via corporate acquisitions.  12T, 6T, 1T...  Lots of choices.  But I don't think they're doing any "new" 8051 chips.

[brucehoult]

Or that what avr-g++ implements is "real C++")

Why on Earth not? What do you/they think is missing?

Certainly, the Arduino environment uses a non-standard library, but that's an entirely different question. You can perfectly well use standard C++ with all features and the standard C++ library as well if you want to -- RAM size permitting: better be on a Mega2650, preferably with external RAM (you can get 64k total).

[voltsandjolts]

I suspect 8051 is similar.

Depends on the compiler, but you ain't gonna do much better than Keil C51.
It's the main reason the 8051 arch has survived this long IMO.
I recall finding one Chinese 8051 mcu manufacturer that had a free download of hacked Keil C51 on their website

[HwAoRrDk]

I think where the 20-bit space might make sense is if you have an existing design and just have to squeeze a bunch more stuff in there that simply can't fit.  With some adhocery sprinkled with ugly hacks it can be made to fit and work - and ship in a timely manner.  It'll be on a downward slope of code maintenance hell though, so better only happen to something that is not going to need more than the occasional bug fix ever.

The STM8 has optional 24-bit addressing, necessary to make full use of the larger parts with more than 32 KB flash (you might think it should be 64 KB, but flash starts at 0x8000). With SDCC at least, it's for the most part completely transparent. Just a question of setting the appropriate memory model command line option.

I say for the most part because there is one gotcha: function pointers are only ever 16-bit, so if you try to take a pointer to a function stored in flash at an address greater than 0xFFFF, things won't work properly.

[ale500]

At least the Keil C51 supplied with cypress parts seems to be very suboptimal in comparison with IAR.
Some problems with the '51 are that the code gets large for anything using pointers, shuffling chunks of memory around is painfully slow. The compiler (IAR) has quite a bit of smarts but pointer related, just forget it. The banks, yeah well not the panacea. Yeah, we produce chips with '51 cores in them (1), a pain to program for, too many desired features too little ROM/OTP . Ours is also a 1 T job with prefetch...