-
The term "module" is conditional. The executable part of the firmware is a monolithic elf-file. The firmware includes several additional data files (graphics, firmware fpga, etc.), but all the CPU code is in the elf.
Therefore, "extract and insert" - is a non-trivial task, weakly amenable to any automation.
It's easier to write your own module on your own, than to cut out a binary image from another firmware, manually adjust the addresses (by a binary editor) and paste result into the existing firmware.
I'm currently considering the option to add firmware functionality by automatically downloading additional modules from the system or user partition (when you turn on the oscilloscope) or from a flash drive. If you do not try to write "Windows For Rigol", then free internal memory is enough.
-
but can it play DOOM??
-
install for testing that firware that modifies among other things "Changed USB Buffer Size (40-> 200) - test, please.I do not use USB IF" I connect the rigol to the usb and the data transmission is still slow, realy that I should improve.
-
I would like to see additional protocols decoding added if that's possible.
-
I connect the rigol to the usb and the data transmission is still slow, realy that I should improve.
Speed should not change. This patch was supposed to fix the problem of non-compliance with the standard.
The implementation of SCPI commands from Rigol is very suboptimal. They tried to formalize the splitting of the program into components, and as a result, reading one parameter word (basically accessible and for direct access to memory) requires a dozen calls, including calling the module via a text name with a table search. It turned out even worse than OLE in Windows. -
but can it play DOOM??
Maybe. The first versions required only a few megabytes of memory.
When saving pictures, the Rigol calls malloc on 4 megabytes:
P4_rw:400D1BB4 MOV R0, #4096000 ; size_t
P4_rw:400D1BB8 BL malloc ; Branch with Link
The operating memory of the CPU is 64MB, of which the firmware takes about 10 at startup. Another 64 MB are connected to FPGA.
You have not decided to write Windows yet?
-
i dont write windows, i erase it!
-
I decided to test in practice how much Rigol memory can allocate via malloc().
I was pleasantly surprised by the result: the limit is about 23 MB (0x1660000). The attempt to get 0x1680000 bytes - failed.
The number of channels turned on, the selected memory depth or the recorded samples did not affect the result.
Thus, more than a third of the CPU memory is free and can be used for any extensions. -
I decided to test in practice how much Rigol memory can allocate via malloc().
I was pleasantly surprised by the result: the limit is about 23 MB (0x1660000). The attempt to get 0x1680000 bytes - failed.
The number of channels turned on, the selected memory depth or the recorded samples did not affect the result.
Thus, more than a third of the CPU memory is free and can be used for any extensions.
That is a very interesting finding, good to know, thank you!
Yet, those 23MB might not be _always_ free to use. I am saying this because, when I tested how many RAW data points can be requested at once by the SCPI command ":WAV:DATA?", sometimes it was possible to extract up to 1 700 000 data samples at once (with only one channel active, byte mode). When more functions were activated, the maximum data points fell down to about 1 000 000 points (for only one channel active). The programming manual for DS1054Z specify the limit should be maximum 250 000 points at once.
That makes me think that there are situations were _all_ the RAM is used, when additional functions like Fourier/filters/decoders/whatever are activated. If we were to have 23 MB free at all time, then I don't see the reason for not being able to extract the same 1.7mil data points when additional functions are enabled.
Of course, this is not such a big inconvenient, because it can be tested if a malloc() was successful, but I won't bet on having all those 23MB available all the time. -
I am surprised that this has not gone viral. What am I missing?
-
I am saying this because, when I tested how many RAW data points can be requested at once by the SCPI command ":WAV:DATA?", sometimes it was possible to extract up to 1 700 000 data samples at once (with only one channel active, byte mode). When more functions were activated, the maximum data points fell down to about 1 000 000 points (for only one channel active). The programming manual for DS1054Z specify the limit should be maximum 250 000 points at once.
It is possible that this is due to some other reasons. I watched (though not in great detail): WAV: DATA? - the function uses a permanently allocated (at the startup) buffer of 2 MB. The same buffer is used in the part of the functions of the decoder. I did not find allocate additional memory of noticeable volume. That, however, does not guarantee that this capture is not performed in any function at the thirty-seventh level of nesting.
Of course, the oscilloscope sometimes uses this memory (a fragment from the image saving function I previously mentioned). The question is in the other - no one ever calculates the required amount of memory to the accuracy of percents. Probably, the developers Rigol went the following way: 32MB of memory for the CPU is not enough - we will put 64.
As a result of the study, we can draw the following conclusion: it is possible to freely load plugins or patches with the size of hundreds of kilobytes. With some caution - the size of megabytes. Considering that Rigol's firmware is a microkernel operating system with a network stack, a host USB and device USB, libraries for saving in png, jpg and tiff, with fonts and many many others, take less than 4MB, my imagination is not enough to imagine a megabyte plugin.
-
Thanks for all the work you put into this project!
I love where this is heading
-
The implementation of SCPI commands from Rigol is very suboptimal. They tried to formalize the splitting of the program into components, and as a result, reading one parameter word (basically accessible and for direct access to memory) requires a dozen calls, including calling the module via a text name with a table search. It turned out even worse than OLE in Windows.
Can you trace the calls?
I have a long standing suspicion that the vertical position controls are slow because somewhere internally they're converting them into multiple SCPI commands. Is it possible to confirm that?
-
I watched (though not in great detail): WAV: DATA? - the function uses a permanently allocated (at the startup) buffer of 2 MB. The same buffer is used in the part of the functions of the decoder.
Great info, thanks!Considering that Rigol's firmware is a microkernel operating system with a network stack, a host USB and device USB, libraries for saving in png, jpg and tiff, with fonts and many many others, take less than 4MB
Could that be BusyBox, or do you think it is rather a proprietary Rigol microkernel?
The thread https://www.eevblog.com/forum/projects/rigol-ds10xxz-firmware-re-write/ seems very interesting, too.
While it is not directly related to writing plugins, tmbinc has a working U-Boot for DS1000Z, and in the pages 3 and 4 there is some info about the keyboard pinout. The pinout might help for disassembling (or writing) code for the keyboard. -
I am surprised that this has not gone viral. What am I missing?
DOOM isn't running yet.
-
I would like to repeat my naive question, is this cool firmware compatible with MSO1074Z ?
Cheers,
DC1MC -
Can you trace the calls?
I have a long standing suspicion that the vertical position controls are slow because somewhere internally they're converting them into multiple SCPI commands. Is it possible to confirm that?
I see calls in disassembler.
I will give an example of how the simplest command for obtaining the channel bandwidth is produced. (SYSTEM:BW?)
In the firmware there is a huge table (more than 1000 elements) for all SCPI commands. The main handler parses the command to separate components, finds the desired element in the table, and calls the subroutine. Up to this point everything has been done normally.
First subroutine:Code: [Select]LXI_SYSTEM_BW_Ack ; DATA XREF: P4_rw:402086C4o
P4_rw:4030F3F4 STMFD SP!, {R4,LR} ; Store Block to Memory
P4_rw:4030F3F8 MOV R4, #0 ; Rd = Op2
P4_rw:4030F3FC BL LXI_SYSTEM_BW_Ack_ ; Branch with Link
P4_rw:4030F400 MOVS R4, R0 ; Rd = Op2
P4_rw:4030F404 CMP R4, #0 ; Set cond. codes on Op1 - Op2
P4_rw:4030F408 BEQ loc_4030F414 ; Branch
P4_rw:4030F40C ADDS R4, R4, #0xB0000 ; Rd = Op1 + Op2
P4_rw:4030F410 B loc_4030F418 ; Branch
P4_rw:4030F414 ; ---------------------------------------------------------------------------
P4_rw:4030F414
P4_rw:4030F414 loc_4030F414 ; CODE XREF: LXI_SYSTEM_BW_Ack+14j
P4_rw:4030F414 BL LXI_SetBackAnswerFlags ; Branch with Link
P4_rw:4030F418
P4_rw:4030F418 loc_4030F418 ; CODE XREF: LXI_SYSTEM_BW_Ack+1Cj
P4_rw:4030F418 MOVS R0, R4 ; Rd = Op2
P4_rw:4030F41C LDMFD SP!, {R4,PC} ; Load Block from Memory
Second subroutinr:Code: [Select]P4_rw:402E1344 LXI_SYSTEM_BW_Ack_ ; CODE XREF: LXI_SYSTEM_BW_Ack+8p
P4_rw:402E1344
P4_rw:402E1344 BWCode = -0xB0
P4_rw:402E1344
P4_rw:402E1344 STMFD SP!, {R4,LR} ; Store Block to Memory
P4_rw:402E1348 SUB SP, SP, #0xB0 ; Rd = Op1 - Op2
P4_rw:402E134C MOV R4, #0 ; Rd = Op2
P4_rw:402E1350 MOVS R0, SP ; Target
P4_rw:402E1354 LDR R1, =dword_4000DE68 ; Source
P4_rw:402E1358 MOV R2, #0xB0 ; '-' ; Len
P4_rw:402E135C BL memcpy ; Branch with Link
P4_rw:402E1360 ;--
P4_rw:402E1360 MOVS R2, SP ; Buffer
P4_rw:402E1364 MOV R1, #6 ; Id
P4_rw:402E1368 LDR R0, =aConfig_16 ; OptCONFIG
P4_rw:402E136C BL GetSettings ; Branch with Link
P4_rw:402E1370 ;--
P4_rw:402E1370 LDR R0, [SP,#0xB8+BWCode] ; Load from Memory
P4_rw:402E1374 MOVS R4, R0 ; Rd = Op2
P4_rw:402E1378 MOVS R1, R4 ; Rd = Op2
P4_rw:402E137C ADR R0, aU_3 ; "%u"
P4_rw:402E1380 BL LXI_PrinfOut ; Branch with Link
P4_rw:402E1384 ADD SP, SP, #0xB0 ; Rd = Op1 + Op2
P4_rw:402E1388 LDMFD SP!, {R4,PC} ; Load Block from Memory
Here you can see a call to the GetSettings with three parameters: the module name (char []), the request code to the module, and the query format descriptor. The specified subroutine searches for the desired module (by direct search through the table via strcmp!!!), creates a special block of query, puts it in the queue for processing and and falls asleep.
The system thread periodically looks at the request queue and calls the desired subroutine (each module also has the ID and subprogram matching tables, the search for them is also linear). Here's a subroutine for BW:Code: [Select]P4_rw:402521F8 Cfg_GetMaxFreq ; DATA XREF: P6_rw:40601EC0o
P4_rw:402521F8
P4_rw:402521F8 var_18 = -0x18
P4_rw:402521F8
P4_rw:402521F8 STMFD SP!, {R4-R6,LR} ; Store Block to Memory
P4_rw:402521FC SUB SP, SP, #8 ; Rd = Op1 - Op2
P4_rw:40252200 MOVS R4, R0 ; Rd = Op2
P4_rw:40252204 MOVS R5, R1 ; Rd = Op2
P4_rw:40252208 MOVS R6, R2 ; Rd = Op2
P4_rw:4025220C BL RetMaxFreqX ; Branch with Link
P4_rw:40252210 CMP R0, #100 ; Set cond. codes on Op1 - Op2
P4_rw:40252214 BNE loc_40252220 ; Branch
P4_rw:40252218 MOV R4, #100 ; Rd = Op2
P4_rw:4025221C B loc_40252224 ; Branch
P4_rw:40252220 ; ---------------------------------------------------------------------------
P4_rw:40252220
P4_rw:40252220 loc_40252220 ; CODE XREF: Cfg_GetMaxFreq+1Cj
P4_rw:40252220 MOV R4, #70 ; Rd = Op2
P4_rw:40252224
P4_rw:40252224 loc_40252224 ; CODE XREF: Cfg_GetMaxFreq+24j
P4_rw:40252224 MOVS R0, R6 ; Dsc
P4_rw:40252228 BL InitStrParamDescr ; Branch with Link
P4_rw:4025222C MOV R0, #0 ; Rd = Op2
P4_rw:40252230 STR R0, [SP,#0x18+var_18] ; Store to Memory
P4_rw:40252234 MOV R3, #0 ; Rd = Op2
P4_rw:40252238 MOV R2, #1 ; Rd = Op2
P4_rw:4025223C MOVS R1, R4 ; Index
P4_rw:40252240 MOVS R0, R6 ; int
P4_rw:40252244 BL RetValFmt_02 ; Branch with Link
P4_rw:40252248 LDMFD SP!, {R0,R1,R4-R6,PC} ; Load Block from Memory
P4_rw:400F5F00 RetMaxFreqX ; CODE XREF: Cfg_GetMaxFreq+14p
P4_rw:400F5F00 ; ConfigMaxFreq2Operate+14p
P4_rw:400F5F00 ; ConfigMaxFreq2Operate:No70Mhzp
P4_rw:400F5F00 ; ConfigMaxFreq2Operate+6Cp
P4_rw:400F5F00 LDR R0, =ManufacturerName ; Load from Memory
P4_rw:400F5F04 LDR R0, [R0,#(FreqVersion2 - 0x40761C9C)] ; Load from Memory
P4_rw:400F5F08 BX LR ; Branch to/from Thumb mode
Having received the required parameter, the subroutine fills the response field, after which the data goes in the opposite direction - the system thread copies the data to the source buffer, frees the previously occupied memory blocks and awakens the requesting. And this is all for the sake of reading one word. In some cases, one module may give nested queries to other modules when processing a request. It reminds me very much of SendMessage / PostMessage from Windows, but can this mechanism be fast?
-
Could that be BusyBox, or do you think it is rather a proprietary Rigol microkernel?
No BusyBox. Rigol use MQX.The thread https://www.eevblog.com/forum/projects/rigol-ds10xxz-firmware-re-write/ seems very interesting, too.
I saw this topic. in my opinion, project has no real prospects. Completely rewrite the firmware, in which more than 12000 functions, without having complete hardware documentation - are practically unrealistic, especially considering the need to work with two FPGAs (one of which firmware is unavailable in principle, and the second one does not have a disassembly mechanism).
While it is not directly related to writing plugins, tmbinc has a working U-Boot for DS1000Z, and in the pages 3 and 4 there is some info about the keyboard pinout. The pinout might help for disassembling (or writing) code for the keyboard.
As for the keys and LEDs, I do not need to know their wiring. I know the functions from the firmware, which are already debugged by programmers Rigol.
Key codes with mnemonic names are available from the SCPI commands:Code: [Select]enum Keys
{
Keys_None = 0x0,
Keys_HMENU = 0x11,
Keys_REF = 0x12,
Keys_HELP = 0x13,
Keys_LA = 0x14,
Keys_TMENU = 0x15,
Keys_TLEVEL = 0x16,
Keys_ZTLEVEL = 0x18,
Keys_CURSOR = 0x21,
Keys_MEASURE = 0x22,
Keys_F2 = 0x23,
Keys_DISPLAY = 0x24,
Keys_F4 = 0x25,
Keys_HPOSITION = 0x26,
Keys_ZHPOSITION = 0x28,
Keys_UTILITY = 0x31,
Keys_STORAGE = 0x32,
Keys_SINGLE = 0x33,
Keys_SOURCE = 0x34,
Keys_MNEXT = 0x35,
Keys_VSCALE = 0x36,
Keys_ZVSCALE = 0x38,
Keys_QOFF = 0x41,
Keys_MSF2 = 0x42,
Keys_CLEAR = 0x43,
Keys_MATH = 0x44,
Keys_TMODE = 0x45,
Keys_VPOSITION = 0x46,
Keys_ZVPOSITION = 0x48,
Keys_MSF6 = 0x51,
Keys_MSF1 = 0x52,
Keys_QM1 = 0x52,
Keys_AUTO = 0x53,
Keys_ACQUIRE = 0x54,
Keys_TFORCE = 0x55,
Keys_HSCALE = 0x56,
Keys_ZHSCALE = 0x58,
Keys_QPREVIOUS = 0x61,
Keys_MSF3 = 0x62,
Keys_F1 = 0x63,
Keys_CH1 = 0x64,
Keys_F6 = 0x65,
Keys_KFUNCTION = 0x66,
Keys_ZKFUNCTION = 0x68,
Keys_MSF5 = 0x71,
Keys_CH3 = 0x72,
Keys_F3 = 0x73,
Keys_CH2 = 0x74,
Keys_F5 = 0x75,
Keys_CH4 = 0x78,
Keys_QNEXT = 0x81,
Keys_MSF4 = 0x82,
Keys_MPREVIOUS = 0x83,
Keys_RSTOP = 0x84,
Keys_QPRINT = 0x85,
};
LED codes are easily determined experimentally:Code: [Select]enum LEDS
{
LED_KFUNCTION = 0x1,
LED_CH1 = 0x2,
LED_CH2 = 0x4,
LED_CH3 = 0x8,
LED_CH4 = 0x10,
LED_LA = 0x20,
LED_GENERATOR = 0x40,
LED_RUNSTOP_GREEN = 0x80,
LED_SINGLE = 0x100,
LED_REF = 0x200,
LED_MATH = 0x400,
LED_RUNSTOP_RED = 0x800,
LED_TRIG_SINGLE = 0x1000,
LED_TRIG_NORMAL = 0x2000,
LED_TRIG_AUTO = 0x4000,
}; -
I would like to repeat my naive question, is this cool firmware compatible with MSO1074Z ?
I did not test it on this model, but there should not be any differences in behavior on different models. Rigol has one firmware for all models of the DS1000Z series. -
I would like to repeat my naive question, is this cool firmware compatible with MSO1074Z ?
I expect so. Rigol only supplies one firmware for both models.
-
After cracking last bits of GEL file, I think this FLASH signature sounds interesting.
https://www.eevblog.com/forum/testgear/rigol-ds1000z-series-buglist-continued-(from-fw-00-04-04-03-02)/msg1466288/#msg1466288
I made even windows XXTEA program for crypt and decrypt but didn't know how and where
to write this to see it in action. Is this in some file or in specific sector somewhere there?
I'm not so bright to understand everything (anything) in that complex (boring) disassembly stuff... -
After cracking last bits of GEL file, I think this FLASH signature sounds interesting.
I made even windows XXTEA program for crypt and decrypt but didn't know how and where
to write this to see it in action. Is this in some file or in specific sector somewhere there?
source fragment:Code: [Select]static const unsigned char DecryptKeyString[]={
0x01, 0x00, 0x92, 0x03,
0x41, 0x08, 0x41, 0x08,
0x04, 0x21, 0xC3, 0x18,
0xC7, 0x39, 0x86, 0x31
};
static const char Sign[]="RIGOL TECHNOLOGIES,DS1000Z,SPARROW,201212";
#define DELTA 0x9e3779b9
#define MX (((z>>5^y<<2) + (y>>3^z<<4)) ^ ((sum^y) + (key[(p&3)^e] ^ z)))
static void pascal XXTEA(DWORD *v, int n, const DWORD *key)
{
size_t y, z, sum;
unsigned p, rounds, e;
if (n > 1){
rounds = 6 + 52/n;
sum = 0;
z = v[n-1];
do {
sum += DELTA;
e = (sum >> 2) & 3;
for (p=0; p<n-1; p++){
y = v[p+1];
z = v[p] += MX;
}
y = v[0];
z = v[n-1] += MX;
} while (--rounds);
}
else if (n < -1)
{
n = -n;
rounds = 6 + 52/n;
sum = rounds*DELTA;
y = v[0];
do {
e = (sum >> 2) & 3;
for (p=n-1; p>0; p--){
z = v[p-1];
y = v[p] -= MX;
}
z = v[n-1];
y = v[0] -= MX;
} while ((sum -= DELTA) != 0);
}
}
void TestTea(void)
{
unsigned char Buf[0x200];
FILE *OutFile;
memset(Buf, 0, sizeof(Buf));
memcpy(Buf, Sign, sizeof(Sign));
// ????????????
XXTEA((DWORD *)&Buf, 0x80, (DWORD *)DecryptKeyString);
if(NULL != (OutFile = fopen("SIGN", "wb"))){
fwrite(Buf, 1, 0x200, OutFile);
fclose(OutFile);
}
// ?????????????
XXTEA((DWORD *)&Buf, -0x80, (DWORD *)DecryptKeyString);
Buf[0x1FF]=0;
}
The SIGN file must be written to the last reserved sector of FAT32.
It should be taken into account that this signature gives access to all destructive functions, such as the formatting of the built-in drive and the erasure of the bootloader in NVRAM. I would not want to be responsible for the consequences in not very experienced hands. I will assume that the ability to compile this code and write down the result is a minimal test for experience.
-
So ... when I can run DOSBox in there ? ... j/k guys ...
Really appreciate all the works here, thank you.
Subscribed. -
I had this plain signature file and crypted signature file and xxtea.exe in zip ready.
Good I didn't post them all in there... -
Yes, You can format local 90M disk and SYS directory is there with all files,
You can delete them... But where is the bootloader?
And yes, this menu is super secret and won't be visible on screenshots.
That's why here is my professional webcam images...