Author Topic: 121GW bluetooth protocol  (Read 2591 times)

0 Members and 1 Guest are viewing this topic.

Offline nuclearcat

  • Supporter
  • ****
  • Posts: 37
  • Country: lb
121GW bluetooth protocol
« on: January 12, 2018, 08:28:34 pm »
I prefer to create separate thread, as (hopefully) here we can discuss bluetooth protocol of 121GW.
It looks like quite trivial to integrate to various software.

As far as i see specification is not published, and honestly github C# sources are not clear(no comments whatsoever, but i am usual OSS/Linux fanboy,
and C# is something i just dont like)

So till now what i reached, i am subscribing over BLE, to indication, over handle 0x9
This is what i get:

n@n:~$ gatttool -b 88:6B:0F:41:0A:8F --char-write-req -a 0x0009 -n 0300 --listen
Characteristic value was written successfully
Indication   handle = 0x0008 value: f2 31 37 30 38 30 30 30 30 30 30 32 30 30 30 32 45 34 30 36
Indication   handle = 0x0008 value: 31 32 30 30 30 30 30 30 30 30 31 36 34 30 30 30 30 36 31 32
Indication   handle = 0x0008 value: 30 30 30 30 30 30 30 30 30 30 30 30 42 32 0d 0a f2 31 37 30
Indication   handle = 0x0008 value: 38 30 30 30 0a f2 31 37 30 38 30 30 30 30 30 30 32 30 30 30
Indication   handle = 0x0008 value: 32 31 43 30 0a f2 31 37 30 38 30 30 30 30 30 30 32 30 30 30
Indication   handle = 0x0008 value: 32 31 43 30 0a f2 31 37 30 38 30 30 30 30 30 30 32 30 30 30
Indication   handle = 0x0008 value: 32 31 43 30 36 31 32 30 30 30 30 30 30 30 30 31 36 34 30 30

As i suspect, BLE indication is limited to 20 bytes, and probably real data needs more, so it come over several BLE indication packets.

Or as ASCII something like that (over simple python script):
A notification was received: ?1708000000000000064
A notification was received: 0100B30B000240000000
A notification was received: 0000000000009F
?170
A notification was received: 8000
?17080000000000
A notification was received: 0006
?17080000000000
A notification was received: 000640100B30B0002400
A notification was received: 0000
?17080000000000
A notification was received: 000640100B30B0002400

Probably 0xF2 signaling start of data? (very likely according C# code)

So string to decode is "17080000000000000640100B30B0002400000000000000000009F"
I will try first:
17,08,00000,0,00,

Then according to source, first 2 bytes should be year:
public int Year => DecimalNibbleToValue(0, 2) + 2000;
I have 2018 set correctly, but it seems showing 2017?
Next is month 2 - 08? august?
Serial 00000 - well, no idea why zeroes, maybe not implemented yet.
00 Voltage Low Z - which is correct

I switched to voltage:
17,08,00000,02,0001B8061200000000164000061
And 02 is correct, "Voltage AC (V)", thats what i see on screen.

So what you think should be done next? Maybe, i can do some easy to hack python script based on bluepy library.
This will open bunch of opportunities to hook multimeter to raspberry, use it for triggering some specific events (like voltage threshold reached, and something should be done)

Suggestions?
 
The following users thanked this post: Iagash

Offline nuclearcat

  • Supporter
  • ****
  • Posts: 37
  • Country: lb
Re: 121GW bluetooth protocol
« Reply #1 on: January 12, 2018, 10:07:52 pm »
With simple python script to accumulate data from notifications from 0xf2 to 0xf2, with fairly trivial script, here is what i got:
It looks like crc errors?
I dont exclude it is some python issues (usually i use C/c++/Go, but programming BLE with them is pain and not very cross-platform)

Accumulated data(55): array('c', '17080000001000001640100BD00000E4000000000000000000096\r\n')
Accumulated data(43): array('c', '17080000001000001640100BD00000E40000000000\n')
Accumulated data(55): array('c', '17080000001000001640100BD00000E4000000000000000000096\r\n')
Accumulated data(43): array('c', '17080000001000001640100BD00000E40000000000\n')
Accumulated data(55): array('c', '1708000\x81001000001640100BD00000E4000000000000000000096\r\n') <-- this one for example 0x81 appeared in serial number
Accumulated data(43): array('c', '17080000001000001640100B"00000E40000000000\n')
Accumulated data(55): array('c', '1708000000100000064&100BD00000E4000000000000000000097\r\n')
Accumulated data(43): array('c', '17080000001000000640100BD00000E40000000000\n')
Accumulated data(55): array('c', '17080000001000000640100BD00000E4000000000000000000097\r\n')
Accumulated data(43): array('c', '17080000001000000640100BD00000E40000000000\n')
Accumulated data(55): array('c', '17080000001000001640100BE00000E4000000000000000000095\r\n')
Accumulated data(43): array('c', '17080000001000001640100BE00000E40000000000\n')
 

Offline Iagash

  • Regular Contributor
  • *
  • Posts: 65
  • Country: de
Re: 121GW bluetooth protocol
« Reply #2 on: January 12, 2018, 10:35:11 pm »
Hi,

With simple python script to accumulate data from notifications from 0xf2 to 0xf2, with fairly trivial script, here is what i got:
It looks like crc errors?
I dont exclude it is some python issues (usually i use C/c++/Go, but programming BLE with them is pain and not very cross-platform)

wow this is cool. I tried your gattool command and can also receive data from my 121gw.

I just tried with my laptop but will look through my hardware box if I can find a BLE capable usb bluetooth receiver for my desktop machine.

Could you add your python script to your post?
« Last Edit: January 12, 2018, 10:37:06 pm by Iagash »
 

Offline nuclearcat

  • Supporter
  • ****
  • Posts: 37
  • Country: lb
Re: 121GW bluetooth protocol
« Reply #3 on: January 12, 2018, 10:38:36 pm »
It is still VEEERY early debug, i am just searching for a way to retrieve data reliably and then will start parsing it
Code: [Select]
#!/usr/bin/python
from bluepy import btle
from bluepy.btle import UUID, Peripheral,DefaultDelegate
import binascii
import array
from struct import *

def parse_data(x):
    xlen = len(x)
    for offset in range(0,xlen):
        if (ord(x[offset:offset+1]) == 0xF2):
            # Process accumulated data
            print ("Accumulated data(%d): %s" % (len(parse_data.accumulated_data), str(parse_data.accumulated_data)))
            parse_data.accumulated_data = array.array('c')
        else:
            parse_data.accumulated_data.append(x[offset:offset+1])

class MyDelegate(btle.DefaultDelegate):
    def __init__(self):
        btle.DefaultDelegate.__init__(self)

    def handleNotification(self, cHandle, data):
         parse_data(data)


parse_data.accumulated_data = array.array('c')
# Put your multimeter mac here
p = btle.Peripheral("88:6B:XX:XX:XX:XX", btle.ADDR_TYPE_PUBLIC)
p.setDelegate( MyDelegate() )

print('Ready!')
p.writeCharacteristic(0x9, "\x03\x00")

while True:
    if p.waitForNotifications(1.0):
        continue

    print("Waiting...")

 
The following users thanked this post: Iagash

Offline Iagash

  • Regular Contributor
  • *
  • Posts: 65
  • Country: de
Re: 121GW bluetooth protocol
« Reply #4 on: January 13, 2018, 01:50:35 am »
Thanks a lot, I found an usb-adapter that seems to support BLE and I got your script running after compiling and installing bluepy. 

I also get these strange escapes:

Accumulated data(39): array('c', '17080000004\x0300000061200000100164000061\n')
 

Offline nuclearcat

  • Supporter
  • ****
  • Posts: 37
  • Country: lb
Re: 121GW bluetooth protocol
« Reply #5 on: January 13, 2018, 02:55:02 am »
Probably we need to wait official comments and documentation for protocol(ot at least ideas, why data is sometimes corrupted), then i can continue development.
 

Offline hwti

  • Contributor
  • Posts: 15
  • Country: fr
Re: 121GW bluetooth protocol
« Reply #6 on: January 13, 2018, 03:22:46 am »
Using text doesn't seem very efficient !
But even on BLE, 5 measurements (ie 100 bytes ?) by second should be doable, but it depends on the connection interval.
 

Offline nuclearcat

  • Supporter
  • ****
  • Posts: 37
  • Country: lb
Re: 121GW bluetooth protocol
« Reply #7 on: January 13, 2018, 03:29:08 am »
On my experience with BLE (but with NRF51), it can do much more.
But in that case i developed a bit custom protocol, while in 121GW is bluegiga module, probably used in "serial cable extender" mode, i'm not very familiar with this module, and not even sure it is possible to upload custom firmware there.
 

Offline Iagash

  • Regular Contributor
  • *
  • Posts: 65
  • Country: de
Re: 121GW bluetooth protocol
« Reply #8 on: January 13, 2018, 04:17:12 am »
I played around with the gatttool a bit using:
Code: [Select]
gatttool -b 88:6B:0F:XX:XX:XX --char-write-req -a 0x0009 -n 0300 --listen | ./121gw.pl
The perl script triggers on the 0xf2 like Dave2's app does. However the app calls the PacketProcessor with a length of 52.

Code: [Select]
private PacketProcessor MyProcessor = new PacketProcessor(0xF2, 52);
I don't understand where the 52 is from. From gatttool I get most of the time 78 hex chars or 39 bytes until the next 0xf2 follows..

The perl script is that one (sorry, I only know perl, yes I'm that old):
Code: [Select]
#!/usr/bin/perl

$start = 0;

while(<>) {
        if ( /^Indication   handle = 0x0008 value: (.*)/ ) {
                @in = split(' ',$1);
                foreach $b (@in) {
                        if ( $b eq "f2" ) {
                                print "\n";
                                $start = 1;
                        } else {
                                if ( $start == 1 ) {
                                        print "$b";
                                }
                        }
                }
        }
}
print "\n";

The output looks like this:

Quote
31373038303030303030313031343041451b34303130304537303130383038343030303030300a
3137303830303030303031300c343041453634303130304537303130383038343030303030300a
31373038303030303030313031343041453634303130304537303130383038343030303030300a
31373038303030303030313031343041453634303130304537303130383038343030303030300a
31373038303030303030313031343041453607303834303030303030303030303030303030300a
31373038303030303030313031343041450d74373031303830383430303030303030303030300a
3137303830303030303031303134304145360a
31370638303030303030313031343041453634303130304537303130383038343030303030300a <-- here is a "06" at pos 5,6 where I expected "30"
31373038303030303030313031343041453634303130304537303106383038343030303030300a
31373038303030303030313031343041453634303130534537303130383038343030303030300a
31373038303030303030313031343041453634303130304537303130383038343030303030300a
3137303830303030303031303134304145360c304537303130383038343030303030303030300a
3137303830303030303031303134304145360630303030303030303030303032440d0a
313730383030303030303130313430414536343031300a
31373038303030303030313031343041453634303130304537303130383038343030303030300a
31373038303030303030313031343041453634300330304537303130383038343030303030300a
31373038303030303030313031343041453634303130304537303130383038343030303030300a
31373038303030303030313031343041453634303130304537303130383038343030303030300a
31373038303030303030313031343041453634303130304537303130383038343030303030300a
31373038303030303030313031343041453681303030303030303030303030303030303030300a
313730383030303030303130313430414536303032440d0a
3137303830303030303031303134304145363430313030453730313038303834300a
31373038303030303030313031343041453634303130304537303130383038343030303030300a
31373038303030303030313031343041453634303130304537303130383038341830303030300a

So there seem to be incomplete records. You  can also find flipped bits in the "complete" records.
It looks like the BLE transmission is either inherently unreliable (I don't know anything about BLE) or the meter is sending garbage and incomplete records.

In Post http://www.eevblog.com/forum/testgear/eevblog-121gw-multimeter-issues/msg1397729/#msg1397729 I also quoted bluetooth logged VA values, which where way of.

Did you find anything in the app that looks like a CRC or something else that makes sure the data is correct? I didn't find anything.

If there is no data integrity check and the BLE protocol doesn't make sure the data arrives correct, then this bluetooth feature is completely useless.
« Last Edit: January 13, 2018, 05:09:26 am by Iagash »
 

Offline hwti

  • Contributor
  • Posts: 15
  • Country: fr
Re: 121GW bluetooth protocol
« Reply #9 on: January 13, 2018, 04:57:13 am »
There should be a CRC on the lower layers.
BLE indications are also acknowledged, so nothing should be lost.

If the meter is more than 1.5m away from my laptop, the gattool command sometimes fails, and even when it works the connection drops after a few seconds.
Perhaps the transmission power is very low (but it could be my laptop's fault too).
« Last Edit: January 13, 2018, 04:59:19 am by hwti »
 

Offline Iagash

  • Regular Contributor
  • *
  • Posts: 65
  • Country: de
Re: 121GW bluetooth protocol
« Reply #10 on: January 13, 2018, 05:14:23 am »
Yes, I did some reading on BLE and the GATT profile. The lower layers should take care about transmission errors.

This makes the variable length and "unexpected" bytes in the "records" even more suspicious. Unfortunately the 121gw is my only BLE device, so I can't test if I get correct data from other devices.

I'd also be interested in the reasoning of sending the data basically ASCII coded when transmission speed is already low.
 

Offline riyadh144

  • Supporter
  • ****
  • Posts: 47
Re: 121GW bluetooth protocol
« Reply #11 on: January 13, 2018, 05:29:43 am »
That is pretty great work, I will start testing and playing with it once I get my meter.
 

Offline beanflying

  • Frequent Contributor
  • **
  • Posts: 703
  • Country: au
  • Toys so very many Toys.
Re: 121GW bluetooth protocol
« Reply #12 on: January 14, 2018, 04:58:48 pm »
Using one of these ANT+ dongles I use on my indoor Trainer (yes I am a Lycra wearer too ;D ) https://www.ebay.com.au/itm/USB-ANT-Stick-for-Garmin-Forerunner-310XT-405-405CX-410-610-910-/232297420959?hash=item361601589f

Added it on the shack PC and it hooked up to the 121GW without a problem.

One of the first things to do if possible now is to get access to the internal SD card so log files can be read, deleted etc!

Log from the install
"Driver Management concluded the process to install driver bthleenum.inf_x86_3bbedb3abba072b6\bthleenum.inf for Device Instance ID BTHLE\DEV_886B0F731C99\6&3338274&0&886B0F731C99 with the following status: 0x0."
Coffee, Food, R/C and electronics nerd in no particular order :)
 

Online EEVblog

  • Administrator
  • *****
  • Posts: 25632
  • Country: au
    • EEVblog
Re: 121GW bluetooth protocol
« Reply #13 on: January 14, 2018, 07:26:10 pm »
Probably we need to wait official comments and documentation for protocol(ot at least ideas, why data is sometimes corrupted), then i can continue development.

We saw data corruption too, David dealt with it in software somehow, he may be able to clarify.
 

Online EEVblog

  • Administrator
  • *****
  • Posts: 25632
  • Country: au
    • EEVblog
Re: 121GW bluetooth protocol
« Reply #14 on: January 14, 2018, 07:26:55 pm »
One of the first things to do if possible now is to get access to the internal SD card so log files can be read, deleted etc!

It's not. The firmware has no support to do this that I know of.
 

Online EEVblog

  • Administrator
  • *****
  • Posts: 25632
  • Country: au
    • EEVblog
Re: 121GW bluetooth protocol
« Reply #15 on: January 14, 2018, 07:28:54 pm »
On my experience with BLE (but with NRF51), it can do much more.
But in that case i developed a bit custom protocol, while in 121GW is bluegiga module, probably used in "serial cable extender" mode, i'm not very familiar with this module, and not even sure it is possible to upload custom firmware there.

The Bluegiga 112 module uses the standard Cable Replacement mode. There is an internal header to reprogram this is you really want to.
 

Offline nuclearcat

  • Supporter
  • ****
  • Posts: 37
  • Country: lb
Re: 121GW bluetooth protocol
« Reply #16 on: January 14, 2018, 07:50:44 pm »
The Bluegiga 112 module uses the standard Cable Replacement mode. There is an internal header to reprogram this is you really want to.
Probably if i will do anything, only if it is useful for rest of community. For myself - existing functionality is more than enough, just data corruption a bit irritating.
Yes, it is solvable (by checking length of data, printability of characters in the middle of data chunk, etc), but such trivial issue should be fixed in firmware, if possible.
I'm realistic about this subject as well, sometimes it is not possible, i have experience working with proprietary OEM blobs where vendor just don't care to fix such issues, and only way left - ugly workarounds.

And about retrieving data from sdcard, it is for sure on stm32 and i believe it is definitely possible (unless bluegiga hooked on TX only), but again - this depends if someone might get luck with reverse engineering it(no idea how welcome it is), or get into some NDA and make alternative firmware.
 

Online EEVblog

  • Administrator
  • *****
  • Posts: 25632
  • Country: au
    • EEVblog
Re: 121GW bluetooth protocol
« Reply #17 on: January 14, 2018, 07:56:01 pm »
And about retrieving data from sdcard, it is for sure on stm32 and i believe it is definitely possible (unless bluegiga hooked on TX only), but again - this depends if someone might get luck with reverse engineering it(no idea how welcome it is), or get into some NDA and make alternative firmware.

It was discussed early on and UEi said it was not easy to add, so we decided to leave it out, I would be stunned if it's been included.
 

Offline nuclearcat

  • Supporter
  • ****
  • Posts: 37
  • Country: lb
Re: 121GW bluetooth protocol
« Reply #18 on: January 14, 2018, 08:02:24 pm »
It was discussed early on and UEi said it was not easy to add, so we decided to leave it out, I would be stunned if it's been included.
The only respectful reason, i believe, if they are out of resources on stm32 :)
Any chance data corruption will be solved in upcoming firmware releases?
If not, i finish python examples with workarounds for this corruption.
 

Online EEVblog

  • Administrator
  • *****
  • Posts: 25632
  • Country: au
    • EEVblog
Re: 121GW bluetooth protocol
« Reply #19 on: January 14, 2018, 08:22:26 pm »
Any chance data corruption will be solved in upcoming firmware releases?

Report it on the Issues thread and it'll go on the list.
 

Offline Kean

  • Supporter
  • ****
  • Posts: 581
  • Country: au
  • Embedded systems & IT consultant
    • Kean Electronics
Re: 121GW bluetooth protocol
« Reply #20 on: January 15, 2018, 05:02:03 am »
I wonder if the data corruption mentioned above is a slight mismatch in baud rate between STM32 and Bluegiga module.  I'm guessing they're both using crystal oscillators.
Alternatively, is there any circuitry between the STM32 serial transmit and the Bluegiga receive - e.g. level converter?
 

Offline savageautomate

  • Supporter
  • ****
  • Posts: 41
  • Country: us
  • Technology Entrepreneur, Consultant, Enthusiast
    • Savage Home Automation (Blog)
Re: 121GW bluetooth protocol
« Reply #21 on: January 16, 2018, 06:43:38 am »
I have written a sample NodeJS (Javascript) program that can parse and decode the 121GW data protocol if anyone is interested.
I gathered most of the info needed from David's C# sources.

I also have a rough prototype working on an ESP32 board (http://amzn.to/2mydmoq) that can capture the BLE packets and forward them to the USB connected serial port.  The BLE implementation is still pretty wonky and the platform is a bit unstable.  I'm really looking to implement this logic on an Adafruit nRF52 board (http://amzn.to/2FECcvP) as its much more stable but I'm waiting for a bug to be addressed (https://github.com/adafruit/Adafruit_nRF52_Arduino/issues/113) before I can move forward with it.

 Thanks, Robert
 

Offline Iagash

  • Regular Contributor
  • *
  • Posts: 65
  • Country: de
Re: 121GW bluetooth protocol
« Reply #22 on: January 16, 2018, 07:45:15 am »
Here is the further developed version of my simple perl-script. It now parses the output of gatttool and displays it in readable form.

You should rename the file from 121gw.txt to 121gw.pl.

Then you can run it via:
gatttool -b 88:6B:0F:73:XX:XX --char-write-req -a 0x0009 -n 0300 --listen | ./121gw.pl

It reads the output of gatttool and parses the packet the multimeter sends.
It detects a lot of error but not all of them, so it's something to play with but until the bluetooth transmission is fixed in the firmware I would not rely on the data.

If the meter is in VA mode most of the data is wrong. This can't be fixed in the script and is also the case in the Android app.

The output looks like this:
Quote
1516048737.255 2017-08 00042   Voltage DC (V)  4.4652 V      _TempC    24.0
1516048737.795 2017-08 00042   Voltage DC (V)  4.4648 V      _TempC    24.0
1516048738.065 2017-08 00042   Voltage DC (V)  4.4648 V      _TempC    24.0
1516048739.010 2017-08 00042   Voltage DC (V)  4.4643 V      _TempC    24.0
1516048739.280 2017-08 00042   Voltage DC (V)  4.4643 V      _TempC    24.0
1516048739.550 2017-08 00042   Voltage DC (V)  4.4640 V      _TempC    24.0
1516048739.820 2017-08 00042   Voltage DC (V)  4.4640 V      _TempC    24.0
1516048740.157 2017-08 00042   Voltage DC (V)  4.4639 V      _TempC    24.0

At the beginning is a unix timestamp when the record was received.
The year and month is always the same since the meter doesn't seem to send the current one.
The third column is the "serial number" of the meter. You can set this value on the meter by cycling through the setup until you reach the display with 00000. If you long press SETUP you can change the digits to some other number. This number is send by the meter in every bluetooth frame.

« Last Edit: January 16, 2018, 07:48:12 am by Iagash »
 

Offline amspire

  • Super Contributor
  • ***
  • Posts: 3633
  • Country: au
Re: 121GW bluetooth protocol
« Reply #23 on: January 16, 2018, 09:23:00 am »

The third column is the "serial number" of the meter. You can set this value on the meter by cycling through the setup until you reach the display with 00000. If you long press SETUP you can change the digits to some other number. This number is send by the meter in every bluetooth frame.
Thank you. I have been wondering what that number is for.
 
The following users thanked this post: nuclearcat

Offline Iagash

  • Regular Contributor
  • *
  • Posts: 65
  • Country: de
Re: 121GW bluetooth protocol
« Reply #24 on: January 16, 2018, 07:43:09 pm »
Hi,

I have written a sample NodeJS (Javascript) program that can parse and decode the 121GW data protocol if anyone is interested.
I gathered most of the info needed from David's C# sources.

that's were I also gathered my information for the script.

But did you find a way to get correct readings in VA mode?
 
In VA mode I get every now an then a correct reading but most of the readings are way off, but not invalid in the packet data..
 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf