High-performance brain-to-text communication via imagined handwriting

[DrG]

Nice!




Nature article cite with abstract https://www.nature.com/articles/s41586-021-03506-2

Preprint (not the reviewed final MS) available here https://www.biorxiv.org/content/10.1101/2020.07.01.183384v1

Mainstream https://www.usatoday.com/story/news/health/2021/05/13/stanford-researchers-implant-microchip-paralyzed-man-communicate/5070495001/

[BrianHG]

90 cpm!
99% accuracy!

Wish this was available decades ago, but hopefully it will eventually become easily available.

[DrG]

/--/
Wish this was available decades ago, but hopefully it will eventually become easily available.

The EEG spellers work better and are more established. What blows me away here is that it is an implanted microcomputer in the motor cortex and the associated code to analyze the neuronal activity there. IOW people who know how to write, but can't write because of trauma or disease can write by *trying* to write....writing mentally, just like they used to do before paralysis. That is some nifty code running on some cool hardware.

I remember as an adult, a BYTE magazine handwriting analysis contest (if anyone else remembers that contest please correct my recollection details, if necessary). Basically, they provided writing (or letters) samples - something like in .bmps or similar. You wrote your software to analyze those. Then they tested your software against other samples  - and nothing outrageous at all. My recollection was that the results were...umm..disappointing and incredibly slow...and those were the top entries...the winners

That is how far we have come in less than one adult's lifetime and it is pretty cool. So, I figure, it is compelling to give a lot of attention to the latest ransomeware debacle and the like, but I sometimes forget how much cool stuff is going on.

[RoGeorge]

The discovery was made many years ago, by a monkey.   

Not kidding.  It was an experiment to listen and interpret neural pulses with some electrodes implanted in a monkey's brain.  Seen that in a video from the dawn of youtube era, couldn't find it any more, probably reported for animal cruelty and taken down.  At some point the human researchers started to understand the neural pulses, and made the game to respond to the monkey's brain instead of responding to the joystick controller.   

The monkey figured out pretty fast that the game can be controlled only by "thinking" at moving the joystick, without actually moving the joystick.  Imagine the monkey chilling in front of the screen, and winning every game without moving a finger.   

The awe on the researcher's face must have been priceless.   

About the neural pulses recognition and translating that to text, that's nothing fundamentally new, just classic machine learning trained for the given patient.

The major drawback is that it requires electrodes implanted in the patient's brain.   

[BrianHG]

The 'Ghost In The Shell' universe is coming...

[nctnico]

The discovery was made many years ago, by a monkey.   

Not kidding.  It was an experiment to listen and interpret neural pulses with some electrodes implanted in a monkey's brain.  Seen that in a video from the dawn of youtube era, couldn't find it any more, probably reported for animal cruelty and taken down.  At some point the human researchers started to understand the neural pulses, and made the game to respond to the monkey's brain instead of responding to the joystick controller.   

Still it is a weird kind of science. Years ago while I was working at a research institute my manager came into my office one day asking me if I knew anything about brain-machine-interfacing. I looked at him for a while trying to decipher whether he was joking or serious. Turned out it wasn't a joke and some researchers where trying to get a piece of half-assed software to work. Unfortunately the author gave them a non-working 'latest' version which wasn't tested at all and as a result we wasted a whole lot of time with nothing to show for in the end. It would have been cool to see it work back then but time was up.

[DrG]

The 'Ghost In The Shell' universe is coming...

/-/

I watched it all. For "pop science", it was a refreshingly reasonable take. Insofar as how close we are  to some of those possibilities, I like to remind myself where we were at understanding germ theory a few hundred years ago even though we knew that infectious disease existed a few thousand years ago. When we don't know what we don't know, it is easy for us to claim we know more than we do....until we know we don't