Where to start trying to decode a data stream

[elnino]

Hi all, I'm new here but reasonably fluent in basic electronics and circuits as well as a programming and computer background too.

I've had a big ambition for a while now to be able to control my ducted home air conditioner with my home automation system.  The problem is, the protocol it uses to communicate between the 2 controllers and the main indoor unit is unknown.

What I do know:
it uses a single wire data signal at 12v signalling. It has an extremely weak pullup on the data line.  It would appear that the controllers pull the signal pin low to control the data stream but it is a single bus - The data line is common to the 2 wall controllers and the indoor unit and can operate in a multi-master mode.

In isolation, the controller will regularly spit out data when powered on, the data stream changes with different settings active.  However, after about 2-3 mins, it will come up with an error on the screen.  I assume this is because it is not getting responses/data from the main unit.

I don't have a usable scope but i did hook up an opto isolator to the output and pumped it into my sound card on my PC and I have a grab of a few of the outputs.

What I need help with (even if someone can point me to some resources) is some assistance trying to determine the protocol.  Is this impossible?  Where is a good place to start?

The air conditioner is an LG and uses the 'Deluxe wall controller'.  It is based on a Toshiba TMP87PP21DFG 8 bit UC.  It has an SPI port but even with that I am not sure where to start. There is no off the shelf option for any sort of network interface for this model. The unit has an option for a Wireless RF remote but they are impossible to buy/find, otherwise i would have gone down that path..

Any assistance would be greatly appreciated.

[ebclr]

Do you have a osciloscope?

[Ian.M]

You start with a scope that can handle DC coupled input signals.   It needs to have a bandwidth of at least 5x the baud rate of the data signal you are trying to sample.  A sound card scope almost always has AC coupled inputs and with the maximum possible sample rate of 96KHz for a high end card will be unsatisfactory or even useless for any baud rate above 9600 baud.  A crappy 44.1KHz sample rate card will be no good for more than 4400 baud.   As its AC coupled, long busts will drift till they are balanced about 0V further messing up your captured data and making automated processing to decode it more difficult.

Depending on how you wired the optoisolator, and the resistors and output side supply voltage you used with it, you may have further smeared the signal.  Zip up some of the data you've captured and attach it , or post two screenshots, one of the whole data burst and the other zoomed in to show about 20 edges (if you aren't seeing reasonably square edges when zoomed in, your capture method is definitely FUBARed) and we'll see if there's anything recognisable there.

@ebcr: O.P. said "I don't have a usable scope ...."

[brozier]

I'd say you don't need an scope.

You need a logic analyser (cheap salae clone and sigrok will do) and a level shifter / op amp buffer to bring the +12V to +5V without affecting the system...

Sigrok is nice as it has many different protocol decoders so you might strike lucky.

Cheers
Bryan

[Ian.M]

OTOH if you don't have a usable scope, it makes it  difficult to determine if your level shifter is distorting the signal significantly.   You could try a fast comparator  with a fixed potential divider for the input signal and the reference level from a pot between the logic supply and ground, and twiddle it to find the maximum and minimum reference levels that still appear to produce a valid datastream, then set it midway.

[brozier]

OTOH if you don't have a usable scope, it makes it  difficult to determine if your level shifter is distorting the signal significantly.   You could try a fast comparator  with a fixed potential divider for the input signal and the reference level from a pot between the logic supply and ground, and twiddle it to find the maximum and minimum reference levels that still appear to produce a valid datastream, then set it midway.

True everything is easier if you have a scope :-)

Looks like its a common problem to solve :-
https://hackaday.io/project/19473-reverse-engineering-a-fujitsu-air-conditioner-unit/log/52023-baseline-communication

Might be worth looking at the driver chip mentioned in the above project. Should give an insight into slew rates needed. Might even be a similar protocol so an FTDI chip + level shifter might be the answer!

[elnino]

Thanks for the responses everyone!

Here are some snaps of the data.  Yes, you can see that the soundcard is messing with the signal being held high or low but I think it leaves enough to see what is going on.

The Baud rate seems fairly slow.

[Ian.M]

Assuming the second image's timescale is in seconds, it looks like 93.5 baud asynch serial with inverted RS232 framing - 1 start bit, 8 data bits and 1 stop bit.  There's a 13 byte block then a short gap before the next block.   Your trace annotated with framing attached.

Of course I'm guessing about the data bits being inverted or not and whether its low bit first like RS232 or high bit first like SPI, but the above would be good enough to read it with a level shifter and a FTDI USB<=>logic level serial cable set to a custom baud rate divisor of 32100, then the rest could be fixed up once you've grabbed the dat5a.

[elnino]

Wow, that is awesome!  Thank you so much - It makes so much more sense now! That's way more than what I was hoping for.  I'm just not used to analysing data like this though.

I believe the signal here is actually inverted because of the method I used to acquire the waveform so its entirely possible its a more standard serial stream, just at a very low bit rate.

I'll have a tinker with an FTDI adapter and see how I go.

[Ian.M]

Logic level serial is inverted with respect to RS232 anyway, so its already the right way up for a logic level serial adaptor. However I've just checked the FTDI documentation and the maximum divider is 16384 giving 3MHz/16384 = 183.1 baud as the lowest a FTDI adapter can do.   

You are going to need something to translate the baud rate up to something a modern PC can handle.  I believe an Arduino with a hacked version of its softwareSerial library (to add a new entry 93 baud in the baud rate table, with the appropriate delays), would be the best option.  Its got plenty of time between characters so you can do the rest of the decoding there and send ASCII HEx to the PC.

[martinator]

[elnino]

I should be able to mod the software serial library ok - It's a bit beyond my knowledge in relation to serial timings etc but I should be able to nut it out.  Thanks for the assistance!

[Ian.M]

It looks like newer versions of the library compute the baud rate on the fly when you use the .begin() method.  It takes an integer, so assuming you've already created a SoftwareSerial object called mySerial, try:

Code: [Select]

  mySerial.begin(94);Its better to be fractionally faster so its ready for the next start bit so 94, not 93.
You can try sending repeated 0x55 (ASCII 'U') which is alternate '0' and '1' bits, and if you send it continuously, should give you a frequency of 47Hz on your chosen TX pin (use the . print method to send a long string of repeated U characters for minimum gaps).

[elnino]

Wow, you have gone so far beyond my expectations you've been a massive help, thank you so much Ian!

I do have a scope but it's some old CRT thing from the 80's, hence my statement no 'usable' scope but perhaps this might actually be useful here.  I'll load up some code onto an arduino a bit later and check it out. With any luck this might be way simpler than what I first imagined.  One can only hope!

[Ian.M]

Gack!  I took the time for 11 bytes - 110 bits not 100.  Corrected baud rate 102.8 - still assuming that the timescale in the plot you posted is in seconds.
Try .begin(103) and you'd be looking for 51.5Hz out sending 'U'.

A reasonably good analog scope is fine for this sort of stuff except for the flicker you get at low sweep speeds.  If it's got two channels + ext trigger capability, a dual (delay) timebase and 50MHz or better bandwidth, its plenty good enough for all hobbyist stuff up to about 10MHz.

If you set up the Arduino sending 'U' on Ch2 and the original signal on Ch1, in 'chop' mode, triggering on Ch1, with the tmebase at about 0.1s/div you can see how close the baud rate is by the speed the Ch2 signal drifts past the CH1 one.

[elnino]

That time scale is in seconds, you are right.

I think the scope I have is 2 channel but I can't remember, it does have a trigger input though. TBH, I've only used it a couple of times.  It was one of those things someone was throwing out and it seemed too much a shame to let it go to the bin.

Comparing some of the other data streams I captured in various states of the controller, the data makes much more sense now.

EDIT: looking closer over the captured waveforms (and having the ability to zoom in nicely) and pairing it with your info Ian, it pretty much works out to bang on 100bps.  The pulses work out to pretty much 10ms but a few are down to 9ms.

[elnino]

Well I have analysed the data stream and comparing 9 samples I took from the controller I seem to have been able to at least start to decipher the data.

I can see bits changing for the Fan speed, Set temp and the Zone control.  The zones are obvious, so is the fan speed but the set temp still has me a tad baffled. It does not seem to align with anything yet.  Either the raw binary equivalent of the set temp value, nor an 'offset' (Some reading suggests that for some units, the value is an offset from the minimum or maximum the system is able to be set at)

What it seems like though is the 13th byte is some sort of checksum as it seems to vary a lot which might take some extra work but at least it's progress!

I will hook up the arduino over the weekend to assist with the data capture and comparison.  As i said though, I'm a bit new to this so reverse engineering a checksum is probably going to be a challenge.

[elnino]

Well thanks to Martinator and Ian I have been able to reverse engineer the protocol and I am successfully reading the codes from the controller - I am ecstatic!

I have been wanting to do this for ages and now it is finally within my grasp.

I have set up some arduino code, reading the output from the controller and printing it in normal human readable strings now.  I also managed to determine that the checksum for the data (13th byte) is a sum of the 12 bytes XORed with 0x55 and then the just using the lower byte of the resulting 16 bit int.

Still some challenges to tackle such as the fact that it uses a single wire for Tx and Rx but it's awesome to see it even reading the codes!

Thanks so much guys!

[NivagSwerdna]

I don't have one of these but I'm a bit surprised it is a single data line.... wouldn't it be more normal to have a pair of differential signals e.g. RS485?

... I did a google... indeed GND, SIG and 12V.   I'm surprised

[Ian.M]

The concept of 12V (or there abouts) active low single wire serial bus is surprisingly common.  Its even made it into the automotive environment as the LIN bus standard. At its simplest the interface is just a couple of transistors, one PNP, emitter to +12V, base to a tap on the bus pullup, with a potential divider on its collector to get the desired logic levels at the RX pin, and a NPN, emitter to ground, base driven (via a resistor) from the TX pin.   A more sophisticated  interface might use a dual comparator with open collector outputs.

The next thing that needs to be done is to examine a data packet closely on a DC coupled scope.  The soundcard scope wont do because it doesn't have enough bandwidth and is AC coupled.   What you are looking for is signal levels and edge slew rates so you can design a suitable bus transciever (or if you are lucky, use an off-the-shelf one)   Specifically, the  rising edge, which if its an exponential curve will tell you the bus uses passive pullups, If its a straight sloping line then it must be a constant current pullup.  If its a near vertical edge, the bus is actively driven high.  Check the whole data packet as the master may use an active pullup.  Try connecting a 1nF then a 10nF capacitor to the bus as additional loads and each time measure the risetime, which will let you calculate the existing bus capacitance and pullup resistance or current.  The fall-time gives you some idea of how hard the existing units pull the bus down or if their drivers are slew-rate limited to reduce EMI.

Alternatively tear-down a device on the bus and trace the bus driver circuit to get a partial schematic, part numbers and values.

[nctnico]

I'd say you don't need an scope.

For this kind of reverse engineering tasks you will be completely lost without an oscilloscope because the first thing you need to know is what the signal looks like.

[elnino]

This might shed some light.

It's a bit more technical than I prefer but it looks like there are 4 transistors and a diode that handle the signal coming from the MCU.  I can take closer up pics but there does not seem to be too much multi-layer stuff going on.  what you see is basically it.

I got the SMD parts under the microscope and the markings for the transistors and diodes are (from left to right)

NB10 PBAI AV49 NB10 PBAI

To me that looks very much like the PNP/NPN driver you are suggesting Ian.

It looks like it might be as simple as hooking on to the 2 pins for transmit and receive.  I'm currently using a level shifter for reading the 12v signal direct but I think that If I put my rx/Tx lines on these pins i can probably easily control the bus.

What do you think?

[Ian.M]

You can't tap in there for transmit as it will conflict with the MCU on the board.  What you can do is duplicate the bus interface circuit.  However its probably better to simply analyse it (start by drawing up a schematic with component values), then pick a compatible off-the-shelf transceiver.  If you cant find the transistors in a SMD codebook you'll probably need to power it up and check signals anyway, as checking in-circuit with a diode tester that outputs more than 2V or 1mA, or removing one to put on a component analyser is relatively high risk.

[elnino]

This is now really pushing the edge of my knowledge and ability to comprehend whats going on in that circuit.

Would something like the attached perhaps work?

Pulled from https://www.maximintegrated.com/en/app-notes/index.mvp/id/214

[Ian.M]

No. as that would apply 12V to the MCU UART input pin.    However it would do for output.   

[elnino]

Ok, cool.  I have a logic level converter sitting between the 12v signal and the arduino so if I used this circuit on the low side of that, it might just be a solution then.  The logic level converter I have is a Jaycar/Duinotech one which I believe is uni-directional so it could just work but this is where my knowledge is limiting me.  I try to learn as much as I can but my head gets around programming better than it does the more complex electronic stuff.

I'm working on a Schematic for the existing controller but this is limited because I can't find definitive information on the little SOT-23 devices.

[Ian.M]

Which logic level converter?  Part number, order code or link please.

[elnino]

I'm currently using one of these - https://www.jaycar.com.au/arduino-compatible-logic-level-converter-module/p/XC4486

I've managed to map out the circuit as best I could - Attached.  However, i am perplexed by the capacitors C08 and C07.  I have tripple checked that is correct. Edit - Somehow that junction between the two capacitors is grounded - Makes a lot more sense now.  Updated schematic.

Thanks again Ian, your help has been awesome.

[elnino]

I also found the transistors are KRC102S and KRA102S which are a switching type transistor but with built in bias resistors.

Your input would be greatly appreciated if you know of a way I can simplify this down with more common parts (SMD is fine).

[Ian.M]

I thought they must be 'digital' transistors.
Here's what it looks like in the LTspice simulator.  I'm guessing the capacitor values and the Zener voltage.


Depending on the low state current required its probably easiest to use a dual comparator (with open collector outputs):

which is good for about 15mA (20mA abs. max.),

Another possibility is to use a LIN transceiver, for a single chip, minimal external components solution.

You can actually get away with using the Jaycar level translator bidirectionally, (Just connect the Arduino's software serial TX pin to the RX pin via a small Schottky diode, cathode to TX so it can only pull the combined TX/RX line low, but the problem is the Arduino's logic low threshold is then used for receive so if the bus doesn't pull down to within a volt or so of ground you dont get valid levels, and the Arduino TX pin has to be able to sink the whole bus pulloup current.  I wouldn't recommend it - if anything goes wrong, your Arduino will let the holy smoke out.

[BubbaMc]

Just out of interest elnino, which home automation system are you using?

[elnino]

I've built my system around OpenHab.  I just recently upgraded it to openhab2 and I am trying to tick off some of the things on the list again I want to be able to control.

Just the  other week I managed to connect my coffee machine to the system which was actually quite easy.
I have RFID enabled front door (I have an RFID implant), automatic stair lights, roller doors, watering system etc.  I'd love to do my lights too but there is not really anything on the market that is economical and safe/approved. Sonoff devices are awesome but not Australian standards approved.

Unfortunately, the AC is just on the edge of the newer generation ones and has very limited / no practical possibilities for integration.

I played with it a bit more tonight but I am struggling with a simple circuit to drive the Tx line. I got to the point tonight where I felt like it should be working but the controller did not update the screen with the settings I sent it.

All of the testing I have done so far has been just with the controller and an arduino - Not actually connected to the AC unit.  It's possible that there is another bit that is set and re-transmitted from the main indoor unit to update the displays.  They have a master/slave switch and this does change the serial data in the first byte so it's also possible that I might need to transmit a 'master' code for the slave to update too.

[elnino]

success!!!  I have managed to manipulate the data and control the system!!  I am so happy! 

Now time to move the code over to ESP8266 and build in some MQTT functions and i'm set!

Ian - Do you have a paypal account I can swing you some beer money on?

[Ian.M]

success!!!  I have managed to manipulate the data and control the system!!  I am so happy!

Great.  Please post the circuit you've ended up with and anything you have discovered about the protocol so others can benefit.

Now time to move the code over to ESP8266 and build in some MQTT functions and i'm set!

I don't know how well the ESP8266 handles very low baud rate.  It looks like you've still got a lot of work left to do.

Ian - Do you have a paypal account I can swing you some beer money on?

Sorry, I don't use Paypal - please make a donation to Dave (EEVblog) on my behalf.

[elnino]

Thanks Ian,  I ended up using an opto isolator for the Tx line.  It was super simple.  Just pulling the 12v signal pin low via the open collector on the isolator.  The led side is tied to 5v and the tx pin on the other side (all with current limiting resistors).  I'll draw it up and post back.  I'll probably also post this as an instructable too.

I bashed my head for a while on it but it was not until I put the new Tx line back into the logic analyser that I noticed the timings were off a bit. Changed the baud to 104 and it was all good.  That moment when you send data and the display changes - Priceless!

The Rx is still via the level converter but I think I will just tie in to the Rx on the exiting controller.

If the ESP module will not do a baud that low, I can handle it, I'll just use a pro-mini with a ENC28J60 ethernet module on it ESP just makes it simple because of Wifi. As you know - Many ways to skin a cat

Will donate to Dave on your behalf.  Many thanks again, I really appreciate your input.  It was very valuable to find the solution!

[BubbaMc]

What you've done is really cool, however if anyone else is looking to solve the same problem I can confirm the Intesis Modbus unit (FJ-RC-MBS-1) works beautifully as an interface with Fujitsu systems using this remote control.

I'm controlling mine now via a Raspberry Pi using Node-RED, with plans to turn it into a fully automated zoned system.

[Fran(_)]

Hi,
I try to read my LG wall air conditioning controller with this https://www.sparkfun.com/products/12009. I connected the low voltage side to a m5stack (GND, 3,3V and pin 16). The high voltage side was connected to the three wires of the wall controller (GND, SIGNAL and VCC). After a minute, the m5stack began to smell like burned.
After that proof I try to upload code to my m5stack without success. I supose the m5stack is dead.
How I have to connect to the m5stack the low side wires of the logic level converter?
Thanks in advance.

[Fran(_)]

One more thing,
when I tried to read codes I only connect the wire of the signal with low voltage of the converter to the pin configured as RX (the 16 in my case).