Need Help with MSComm on PC to Read Serial Binary Data Stream

• Need Help with MSComm on PC to Read Serial Binary Data Stream
  Posted by SuzyC on 06 Nov, 2016 04:41
• I hope some of you code gurus are in a nostalgic legacy mood and give me a helping hand.
  
  I want to input data into a PC using COM1: serial port using MSComm within VB6 SP6. I have no trouble transmitting a binary stream from a MCU to the PC but I don't know how to read the MSComm input buffer when the binary data has being received.
  I want to transfer data as fast as possible, and COM1: port is opened and set to work with 115200, n, 8,1 MSComm.Settings
  
  If wanted to deal with the byte overhead, I could send everything using literal hex characters, but that slows things down!. I would have to make time-consuming int to ascii conversions on the MCU end and ASCII to integer conversions on the PC side.
  Of course, then the byte buffer would be all ASCII 0-F hex literals and I wouldn't have a problem even if the stream was one big slow to process string terminated by a single 0x00 null character.
  
  I am sending a fixed-length stream of integers to the PC with a 5-byte start-of-stream constant preamble followed by each bytes id number and then the hi and lo bytes of the integers one after another.
  
  I have configured the MSComm settings so that MSComm is expecting to receive binary rather than ASCII data.
  
  Example:  00 00 00 00 0x1B  preamble (to detect start of stream)
  then    I would send the first id and word,  for example id=1  then 16-bit unsigned integer (=1.)..id=2..16bit  unsigned int ..etc until 96 integers have been sent and the serial frame ends until the next transmission.
  Integers are sent in big endian order. This means the buffer should contain 5 +(96*3) bytes = 293 bytes total/frame.
  Now I can easily set the input buffer size and read input buffer number of bytes, but there is more devil to detail to get my bytes back.
  
  So if I were to send (preamble) id=1 integer =1..id=2 integer=100...
  So the stream would look like (in Hex)   00 00 00 00 1B   01 00 01  02 00 64 ...etc
  
  My question is how to read the MSComm buffer  to retrieve my binary data?
  
  Thanks for your help!
• #1 Reply
  Posted by JPortici on 06 Nov, 2016 09:15
• if i understood what you want to do..
  why do you think you need to convert binary to ascii on the mcu side?
  reading mscomm gives you ascii one byte at a time, ok. so just use CInt(bufferblah) and you have your raw 8bit back. then you concatenate them into other int or long or whatever.. i do it all the time
  buffer(0) = CInt(rxbuffer)
  ...
  buffer(1) = CInt(rxbuffer)
  
  data = buffer(0) * 256 + buffer(1)    <--- thi actually depends on the endianness of your data
• #2 Reply
  Posted by SuzyC on 07 Nov, 2016 01:51
• In my serially endiendless fight for answers such as to seek Cintches in my search of sequestered information, Jportici, your comm-ents stalls short of sure help.
• #3 Reply
  Posted by cv007 on 07 Nov, 2016 06:19
• https://msdn.microsoft.com/en-us/library/aa259414(v=vs.60).aspx
• #4 Reply
  Posted by SuzyC on 07 Nov, 2016 12:29
• 'Thanks a Gigabit, cv007, but I have already set:
  B$ = Trim(BaudRate)
  MSComm1.Settings = B$ & ",n,8,1"
  MSComm1.InputMode = comInputModeBinary
  MSComm1.InBufferSize = 2048
  MSComm1.InBufferCount = 0 'Clears buffer
  MSComm1.InputLen = 0
  MSComm1.PortOpen = True
  
  'and I (attempt to) get  my data with:
  
  '.............
  TxtNumBytesInBuf = MSComm1.InBufferCount
  If (MSComm1.InBufferCount = FrameSize) Then
    B$ = ""
    B$ = B$ & MSComm1.Input
    Updates = Updates + 1
    Txtupd = Updates
    MSComm1.InBufferCount = 0 'Clears Buffer
  End If
  GoTo ExitComit
  '........
   B$ contains garbage,  and surprisingly, always has only half the number of bytes of FrameSize.
  
  For instance, if I send 8 bytes  (values shown in decimal)  0  0  0  0  27  1  0  1
  Then I parse and look at each of the total of four bytes:
  for j=1 to 4
     A$ = Mid$(B$,j,1): bite=Asc(B$)
    Print bite;"  ";
  next j
  
   Result: Always yields the same bite values:   0  0  103  63
  
  
  No Workee.
• #5 Reply
  Posted by Keicar on 07 Nov, 2016 13:03
• Have you tried using an array of bytes instead of a string variable to store the contents of MSComm1.Input? Not sure it'll help, but it is the actual data type that being returned when InputMode is set to comInputModeBinary.
• #6 Reply
  Posted by PA0PBZ on 07 Nov, 2016 13:16
• Quote from: SuzyC on 07 Nov, 2016 12:29

    B$ = ""
    B$ = B$ & MSComm1.Input
  

  
  Replace that by
  B$ = MSComm1.Input
   (just because...)
  
  

  Quote

  B$ contains garbage,  and surprisingly,t always has only half the number of bytes of FrameSize.
  For instance, if I send just (values shown in decimal)  0 0 0 0 27 1 0 1 , reading Asc(B$) yields 0 0 103 63
  

  
  Wrong baudrate?
• #7 Reply
  Posted by Keicar on 07 Nov, 2016 13:25
• The native string type in VB6 is the OLE Automation BSTR, which is a null-terminated array of 16-bit Unicode characters, preceded by a 32-bit length marker (which resides in the 4 bytes prior to the address pointed to by the BSTR).
  
  The fact that VB6 strings are Unicode may well be related to your getting half the number of characters you're expecting - it would seem that assigning an array of bytes to a string doesn't work the way you might expect. Since each character is 16 bits, it assigns two bytes to each one, I suspect.
• #8 Reply
  Posted by Buriedcode on 07 Nov, 2016 16:27
• Second on the byte encoding.  I would keep your incoming bytes as just.. bytes and not ASCII.  The old MScomm as well as the newer (terribly implemented) .NET serial controls are all geared towards true terminal comms - ASCII, linefeed, carriage return etc.. rather than raw bytes, but of course can be configured for such.
  
  Whilst its nice to be able to use string functions, I have had to use very specific coding to get actual single byte ASCII values, rather than the standard 16-bit encoding.  I believe it was.. crudely converted from C#...
  
  

  Code: [Select]

  s = System.Text.Encoding.GetEncoding(28591).GetChars(dataBytes.ToArray, 0, EOLlocation) 'gets all the bytes up until linefeed character  Note the dodgy number, which I got from an hour of googling.
  
  In the embedded world ASCII is simple, but there are many forms of ASCII encoding on PC's.
  
  As for dealing with frames and fixed-size packet data, I never relied on built in events to fire when a terminal character was found, or when a certain number of bytes has been received.  This is mostly related to the newer .NET controls (3.5+) but I vaguely remember the same deal with MScomm (when I was 12 :/).  It sometimes didn't fire when a certain number of bytes was received, or it did, but only returned half of what it had.  Also, both versions rarely reported the correct number of bytes in its buffer:  you get the number of bytes it claims it has recieved, create an array of that size and copy over, only for an exception to be thrown because in the time it took to move the data through many levels of buffers - you've received a few more bytes, and the array you created is too small.
  
  Crude solution which might not be so great on VB6 was to set up a background worker thread on a timer.  This simply, periodically grabbed all the data from the receive buffer (by seeing how many it claims were received, adding to a large array, resizing the array based on how many were actually received, then copying), and added that chunk to a byte list, which then triggered an event in the main thread to grab that, add to to the end of the main buffer, check for start/termination values and spit out packets when it found them.  Effectively doing it manually because it was the only way I could get all the bytes in at > 19.2k. (seems to work at >1Mbps using USB-serial adapters).  It shouldn't have to be this way, but also was less prone to hanging.
  
  That said people still use MScomm today because for legacy apps its more reliable than the later .NET and...ahem.. Java... for serial comms.
• #9 Reply
  Posted by hendorog on 07 Nov, 2016 19:11
• Quote from: Buriedcode on 07 Nov, 2016 16:27

  
  That said people still use MScomm today because for legacy apps its more reliable than the later .NET and...ahem.. Java... for serial comms.
  

  
  Java serial comms worked fine when I used it many years ago - we did several multidrop RS-485 installations with it. We moved to java as it was easier to avoid memory leaks than C/C++ or Delphi, and easier to maintain than VB (OCX hell). The other bonus was it would run on linux which avoided the pain of having a headless windows box sitting in an industrial cabinet.
  
  I've been using Java serial comms over USB recently, and it has been working perfectly.
  
  
  
• #10 Reply
  Posted by Buriedcode on 08 Nov, 2016 00:54
• I guess I am a bit out of touch then, haven't used java that much in the past er.. 8 years? Well, not directly..
• #11 Reply
  Posted by SuzyC on 08 Nov, 2016 05:55
• Thanks Keicar, that means MSComm binary input mode is useless or just very clumsy for receiving binary bytes with VB6 SP6.
  
  Thanks PAOPBZ. but your change didn't change the result. The baudrate is what it needs to be: 115200,n,8,1
  
  Thanks Buriedcode. I am using a Timer Control that is set to fire every 3-mSec to invoke reading the input buffer. It must be working fine because my received bytes are wrong but entirely consistently wrong each time in values.
  
  Thanks henderog, but at least I know using the VB6 MSComm .OCX code will not have to face compatibility issues with other Windows versions that might belch with any .OCX.  For the automated machine control job program I am coding(only to run on Windows XP SP6 ), my choice of VB6  programming is ideal, it is just so easy and so readable, yet powerful and versatile enough.
  
  VB6 is so simple to code with compared to any newer .net, C++. C#. Java on Unix etc.
  
  
• #12 Reply
  Posted by hendorog on 08 Nov, 2016 06:11
• Quote from: SuzyC on 08 Nov, 2016 05:55

  
  Thanks henderog, but at least I know using the VB6 MSComm .OCX code will not have to face compatibility issues with other Windows versions that might belch with any .OCX.  For the automated machine control job program I am coding(only to run on Windows XP SP6 ), my choice of VB6  programming is ideal, it is just so easy and so readable, yet powerful and versatile enough.
  
  VB6 is so simple to code with compared to any newer .net, C++. C#. Java on Unix etc.
  

  
  Quite, and I was not suggesting you need to change, I was replying to BuriedCode. I actually did a bit of VB back in the day - starting with VB 1.0 and some later versions which I can't remember.
  
  Anyway, if I was going to suggest an alternative (for simpler VB apps) it would probably be Processing.org. Same IDE as Arduino and really easy to get into.
• #13 Reply
  Posted by Keicar on 08 Nov, 2016 10:49
• I guess if you really need the received data in a string you could always assemble it yourself, just iterate through the array of bytes received from MSComm.Input and concatenate each one to your string with CHR$() - not exactly elegant, but it should at least work.
• #14 Reply
  Posted by SuzyC on 08 Nov, 2016 13:02
• Thanks keicar, but your solution falls flat with the third double-byte = 103..
  The recovered code should be the same as sent(in decimal)  0 0 0 0 27 1 0 1   
  Your method changes the received 4 16-bit ascil words from 0 0 103 63 (decimal) into 0 0 0 0 0 163 0 63
  This is giving up figuring out the problem to hacking it and not getting the proper results.
  It can't be that complicated!
  
  Thanks hendorog, but switching to some type of Arduino codiing is just throwing away the baby with the bath water,  again skirting around the problem which obviously must have a simple solution, yet to be revealed here.
  
  It is sad to note that Microsoft's documentation did not present just a few VB lines of code clearly showing how to use MSComm with binary data.
  
  https://msdn.microsoft.com/en-us/library/aa259414(v=vs.60).aspx
  Clearly says to set data input mode to binary to receive binary data, but some other code details are needed to rescue my bytes from the receive buffer.
  
  It can't be that MSComm is an incompetent buggy .OCX  or that it hasn't been successfully implemented by thousands of coders working with binary data who happen to somehow know the proper coding to get it to work perfectly. 
• #15 Reply
  Posted by SuzyC on 08 Nov, 2016 13:30
• Very interesting...
  
  If I switch input mode from binary to ASCII
  If (MSComm1.PortOpen = True) Then MSComm1.PortOpen = False 'Clears any Errors
  MSComm1.CommPort = 1
  B$ = Trim(BaudRate)
  MSComm1.Settings = B$ & ",n,8,1"
  MSComm1.InputMode = 0 ' changed to ASCII from comInputModeBinary
  MSComm1.NullDiscard = False
  MSComm1.InBufferSize = 2048
  MSComm1.InBufferCount = 0 'Clears buffer
  MSComm1.InputLen = 0        'Allows all of the buffer to be read into a single string
  MSComm1.PortOpen = True
   
  Then all 293 binary bytes are consistently received perfectly and in their proper transmitted order into B$ with:
  
  TxtNumBytesInBuf = MSComm1.InBufferCount
  If (MSComm1.InBufferCount = FrameSize) Then  'Frame Size is 293 for preamble of 5-bytes followed by 96 integers in the form id..integer..id.. integer ... all in binary.
  
    B$ = ""
    B$ = B$ & MSComm1.Input
    MSComm1.InBufferCount = 0 'Clears Buffer, ready to receive next Xmitted frame.
  End If
  
  Problem solved...buy why could Microsoft be so misleading!