Author Topic: Haptic Feedback glove  (Read 9814 times)

0 Members and 1 Guest are viewing this topic.

Offline PlagueDoctorTopic starter

  • Contributor
  • Posts: 13
Haptic Feedback glove
« on: November 06, 2014, 01:12:16 am »
Good day EEVBlog!

I know this is my first post here. but I've been subscribed to EEVBLog on youtube for quite some time, and most of my projects I haven't needed help with. but now I want to go on, quite a big undertaking.

The Problem (inspiration):
when you see people put on the oculus rift. after they say "this is soooo cool!" what's the first thing they do?
They reach out their hand and try and touch their virtual surroundings.

What has been done so far (by other people):
I've seen people put sensors on gloves to put their finger movement into the virtual environment.
 - no tactile feedback
 - very imprecise (there is footage of people trying to challenge each other just to grab a hamburger)

I've seen people use strings to create tactile feedback (similar system used in children's hand grabbing toys)
 - can only stop the fingers, not pull it around. (from what I can see)
 - still imprecise
 - fragile
 - clunky.

and I've seen very little improvement in either design over the years.

What do I want?

-I want precision (ever seen a gamer who can tolerate low precision mouses or spongy controls?)
-and I want the world to be able to interact with you, and you with it.


How so I plan to do it?


Stepper motor & Servo hybrid:

-The stepper motor has a strong holding force (you can alter the input current to adjust the strength) and can pull up/down on the attached limb
*the glove will be made primarily of rigid components for a better transfer of energy to your fingers. and to give the joints a mechanical barrier to protect your fingers from bending the wrong way due to glitches/faulty programming.
- there will also be a optical encoder attached for both communicating to the computer the location of each limb. but also to control each step of the stepper motor to make it run smoother, rather then harsh individual steps.

both of witch will (ideally) integrated into a housing with a depth of less then 8mm:


(this is just a rough drawing in sketchup, I am well aware of the countless mistakes in this)

on the knuckles and thumb where a rotary device on the joint is not feasible I will use (not sure of the proper name)
what essentially is a linear stepper motor (where the rotor lies flat, think of a rail gun, or a mag-train) that has a linear encoder attached to it



and have everything located on the glove as so:


*nothing is to scale, placement location only*


Forseeble problems
(ok, there are more foreseeable issues then I'll list here, but I am confident I can overcome all the major one's)

-a door hinge pivot on the knuckles & thumb will be uncomfortable & limit movement
   - Thats why I won't be using plain hinges (though for now, it will have limited movements, for simplicity sake)
-can't feel textures
   - I won't deal with this problem until I have the current system working (though I do have multiple Idea's how to do it)
-how does it know where the hand is in 3d space?
   - A combination of digital gyroscopes (for low latency output) & RF (or similar) triangulation, which corrects any drifting from the gyroscopes.
-how will all this data be processed?
   - on the wrist portion there will be (several) microprocessors (most likely arduino's for now) that takes in all the finger point data, and hand location, compresses it all down to a few bytes, and sends it to the computer, and vice versa (takes in all the feedback data, compresses it, and sends it to the fingers)
-getting a stepper+encoder to that small of a form factor & high enough detail will be expensive, yet alone enough for each joint.
     - I have several Idea's, if worst comes to worst, I'll get the bigger alternatives and mount it on top of the fingers (sacrificing sleekness)



This is the rough outline of my plan.
-(hopefully) I will upload a video tomorrow of a very crude concept prototype of a single knuckle in action.
-I plan to kickstart this if there is enough interest, and isn't completely proven to be impossible (unless I miraculously get a ton more hours at work)
what do you think?

The Plague Doctor.

*Edit: oh! I forgot to mention. I plan to release the information to the open source community (at every major prototype stage at least)
« Last Edit: November 06, 2014, 01:32:00 am by PlagueDoctor »
 

Offline PlagueDoctorTopic starter

  • Contributor
  • Posts: 13
1st update
« Reply #1 on: November 06, 2014, 09:05:30 pm »
as promised, I built a (VERY) crude prototype/proof of concept:





I don't have it wired up yet, nor have I finished the code yet. but it's really quite simple

the Var. resistor goes to the 555 timer circuit, which feeds a variable pulse to the parallel port (yes, I know, the parallel port is not ideal for it since it's not serial, but that just means (for me) that it will be very laggy, which is ok for this sample)

then the output of the parallel port feeds into the floppy drive controller, which feeds into the stepper motor.


(just ignore the other IC's, they were from a different project)

everything else is handled via software.

I know, crude. but I'm working with a budget of $0.00 CAD :)
 

Offline PlagueDoctorTopic starter

  • Contributor
  • Posts: 13
Re: Haptic Feedback glove
« Reply #2 on: November 07, 2014, 04:30:57 pm »
I'm going to be super busy the next 2 days at work (so I won't ave any more updates until Sunday/Monday)

so, a question for you guys:

What should I call this glove/thing?
 

Offline janoc

  • Super Contributor
  • ***
  • Posts: 3781
  • Country: de
Re: Haptic Feedback glove
« Reply #3 on: November 07, 2014, 05:37:51 pm »
I would strongly recommend you to research the existing approaches first before you start reinventing the wheel (and potentially run into a patented design if you are planning a Kickstarter - Immersion holds quite a few and are suing everyone else left and right if it comes anywhere close to haptics).

Look at stuff like CyberTouch, CyberGrasp, there was  a recent one called Dexmo (http://www.dextarobotics.com/products/Dexmo) also ...

CyberGrasp (uses tethers, those are driven by motors in a box on the operator's back):



CyberTouch (sense of "touch" using cellphone-like vibrating motors on top of the CyberGlove dataglove):


CyberForce (CyberGlove + CyberGraph + external force feedback arm):


Haptic workstation (two CyberForce systems + seat):


Unless you are planning this only as a self-education project, I wouldn't bother - these devices are universally horrible. I have tried the expensive full arm + hand Haptic Workstation and CyberTouch/CyberGrasp combos - they suck and the price is crazy. The fully equipped bimanual Haptic Workstation used to cost over a half a million dollars, not counting any application PCs and tracking system for the head. There were only 3 in the world for a reason ...

The problem with touch is that most of the sensation is in your finger tips, not the fingers as such. These devices do nothing to reproduce that. At best you feel that something is pulling/pushing your fingers, kinda like a baby playing with your hand. It feels really weird and nothing at all like touching a real object.

In addition, in order to have a feeling of a firm grasp, you need strong and fast motors that can actually provide quite a lot of torque quickly. Otherwise your fingers will pass through the virtual object that was supposed to be solid before the system can deliver sufficient force to stop you. Little servos and small DC motors are right out, they simply cannot satisfy this. Without that you can easily overpower the device, leading to "mushy" feeling, oscillation, motors and gears wearing out and breaking rapidly. All these were problems on the Haptic Workstation, for example, and I am very sceptical that the Dexmo will be any better considering the way it is built.

To give you an idea of the size of motors needed to achieve the amount of force required to give a feeling of a solid wall, look at this exoskeleton from PERCRO in Italy. This is actually used as haptic (force-) feedback device for the arm. They have (smaller) ones for the fingers as well, just google PERCRO and haptics.



On the other hand, the device must not break the user's fingers/hand/arm nor itself, so you have obvious conflicting design requirements. The device from PERCRO above is certainly strong enough to cause a serious injury, that's why they have quite extensive safety protocols, observers, kill switches, etc. My former prof has tested it and he tried to smash the arm against a (virtual) wall - he has almost broken his own arm while doing so - it was literally like hitting a concrete wall. But we are talking about equipment comparable with industrial robots here ...

In order to know where the hand is you will need some sort of tracking system. You can use an IMU (gyro + accelerometer) to know the orientation, but you will need some additional system to obtain the position - e.g. Kinect, pair of cameras, ultrasonic (aka Powerglove) tracker, magnetic tracking etc. Forget RF tracking, that sort of thing is both very complex and will only get you a fine due to unlicensed radio operation.

Disclaimer: I do this stuff for a living
« Last Edit: November 07, 2014, 06:05:12 pm by janoc »
 

Offline PlagueDoctorTopic starter

  • Contributor
  • Posts: 13
Re: Haptic Feedback glove
« Reply #4 on: November 07, 2014, 06:35:14 pm »
janoc:

Thank you so much for the in-depth reply!

just some things to address your concerns;

Yes, I have done a fair amount of research on the existing devices. (I have seen all of the one''s you posted, except the Dexmo one)
and, as you said: " these devices are universally horrible"
which is exactly why I'm trying to re-invent the wheel, I don't like what I've been seeing so far, and rather then throwing in the towel before I start, and just complain. I've decided to try and tackle this challenge myself.

"Haptic Workstation used to cost over a half a million dollars"

youch, I'm aiming to try and get the consumer price to under $500 a pair. Guess I have quite the challenge ahead of me.

The problem with touch is that most of the sensation is in your finger tips, not the fingers as such
I thought so (but still glad to hear from someone with real experience). and I admit, my glove still won't be perfect.
but I have already had Idea's that add detail to the finger tip parts, to compensate for what I'm quite sure will be a bit mushy feedback in the hand/fingers. these idea's include:
-small air pockets that inflate/deflate in the finger tip portion to add localized pressure (speed will be an issue here)
-small vibrations (several different way's to do this, ranging from motors, to pizzo buzers) to give textures
-the gloves will be made with a rigid shell so your finger tips are pushing against something solid.

and I'm of course open to a lot more idea's/scrutiny (hence why I'm posting on the EEVBlog)

you need strong and fast motors that can actually provide quite a lot of torque quickly

yes, this is one of my biggest concerns at the moment. so far steppers with their own processor (so you don't have to wait for the computer) is the best option I've come up with so far. and some other problems you also mention.
-strong enough to create sensation, but not break your fingers(and be able to fit in the glove without gearing):
       this is one reason i'm considering custom building my stepper motors. because the floppy steppers are to weak (but for now(prototype) they will do. and even the smallest NEMA steppers will quite easily snap your fingers
so I'm looking for a flat stepper, but a wide enough diameter to create the torque needed, without going overboard.

in short. yes, finding a way to convey force, fast, accurate, safely,  sleekly, and affordable. will be quite a challenge. I have some less conventional idea's as well, but they are far more complex (using ferro fluids & multiple sliders, etc) so I want to try with simple steppers first.

google PERCRO and haptics
Thanks! I don't think I've stumbled across them yet :)
(but I have seen that youtube vid. it's one of the better one's I've seen)

In order to know where the hand is you will need some sort of tracking system
I suppose I could use IR tracking (though the users arms could cause interference) I could use optical tracking (like the oculus DK2)
(in short, I'll cross that bridge when I get there.

Another note:
I'm going to use the fact that this is for gaming to my advantage. rather then have the glove respond to every little thing in the virtual world. the game will load up tactile objects into the glove itself. and when the player reaches out to those objects + a grasping motion, those objects will snap to the players hand. and the gloves on-board (dedicated) processor will take over from there, until that object is let go.


Disclaimer: I do this stuff for a living
no disclaimer needed. I'm well aware that I'm probably one of the lesser experienced people on this forum.
(also, I really hope my post doesn't sound like "I know better" I just enjoy a challenge. and finding out that the challenge is more challenging then I first thought doesn't bother one one bit. Just makes it more fun ;)
 

Offline miguelvp

  • Super Contributor
  • ***
  • Posts: 5550
  • Country: us
Re: Haptic Feedback glove
« Reply #5 on: November 07, 2014, 06:50:42 pm »
For low cost, Disney research has some cool technology that can even simulate rain and textures

http://www.disneyresearch.com/project/aireal/

They also have other research that add texturing to surfaces:
http://www.disneyresearch.com/project/3d-touch-surfaces/

Blackberry also did some haptic feedback for their phones, not sure if it's similar tech but also applies to surfaces.

I recall also some kind of display that will physically change. That tech can be applied to gloves.

Electrical stimulation via a wristband that can send impulses to various parts of your hand is another approach. There is been a lot of research on this field but sometimes it takes someone to bring in a new way to do it.

I remember a while back there was a mouse that provided texturing feedback to the user, and that was back in the late 90s, may forcefeedback devices have been around too and also haptic suits have been explored.

But it's mostly for sensing more than actually making virtual objects feel solid, although a lot of research is done on that aspect as well.
 

Offline PlagueDoctorTopic starter

  • Contributor
  • Posts: 13
Re: Haptic Feedback glove
« Reply #6 on: November 07, 2014, 07:36:53 pm »
Disney aireal

yeah, I've seen this in several places. not quite what I'm after though.

Disney 3D touch surfaces

Yes! this is exactly what I aim for inside the glove (on a much later version.. but still)

I recall also some kind of display that will physically change
you mean like this one?
I've looked into it, and it's still on the talble. but the big killer is the budget.


I think I'm a bit to amateur at the moment to start inducting currents into people's bodies :)

I remember a while back there was a mouse that provided texturing feedback to the user
YES! I was mega pumped for that. it was even on tiger direct for a while. but when I could afford it, I couldn't find it anywhere!
(but it's not quite what I'm looking for in this project since it would be impractical to use in a omni treadmill or similar)

assuming this is the one you are talking about:


Thank you for your information/reply !
 

Offline janoc

  • Super Contributor
  • ***
  • Posts: 3781
  • Country: de
Re: Haptic Feedback glove
« Reply #7 on: November 08, 2014, 01:13:59 pm »
and, as you said: " these devices are universally horrible"
which is exactly why I'm trying to re-invent the wheel, I don't like what I've been seeing so far, and rather then throwing in the towel before I start, and just complain. I've decided to try and tackle this challenge myself.

Well, good luck. From what you are writing, you seem to lack quite a bit of knowledge about how the electronics and haptic rendering in general actually work, but you certainly don't lack enthusiasm :) That's what I have meant about doing your research and learning first. Haptic rendering is a fairly complex subject, building electromechanical gadgets is only a small part of it.

Quote

"Haptic Workstation used to cost over a half a million dollars"

youch, I'm aiming to try and get the consumer price to under $500 a pair. Guess I have quite the challenge ahead of me.

The problem with Haptic workstation is twofold - first, it is an extremely niche, made-to-order device that pretty much is a solution looking for a (research) problem. As-sold, there is essentially *no* application that it could be used with out of the box - you get and SDK, documentation - and that's it.

The second problem is that this is classical VR hardware - that means that there are enormous margins being charged by the vendors. Until the Oculus Rift came along for $400 or so, it was common to sell much much worse HMDs for $20k upwards, even $80k was not uncommon. The CyberGlove data glove costs around $10k per hand - for the last 20 years. The only thing that has changed over time was adding of a USB to serial converter, because RS232 connectors fell out of fashion ... Silly Inertia Cube 3 which is nothing else but a 3 DOF innertial tracker comparable with MEMS gyro/accelerometer/magnetometer combo in every cellphone today is being sold for $600-$800. Invensense MPU9250 that does very much the same costs about $5 in singles, full IMU with a PC interface is doable for about $50-100 with it.

So my point was not to say that it cannot be done cheaper (it certainly can), but that even hardware that is being sold for such exorbitant prices didn't manage to solve the problem using the approach you are attempting.

Quote
The problem with touch is that most of the sensation is in your finger tips, not the fingers as such
I thought so (but still glad to hear from someone with real experience). and I admit, my glove still won't be perfect.
but I have already had Idea's that add detail to the finger tip parts, to compensate for what I'm quite sure will be a bit mushy feedback in the hand/fingers. these idea's include:
-small air pockets that inflate/deflate in the finger tip portion to add localized pressure (speed will be an issue here)
-small vibrations (several different way's to do this, ranging from motors, to pizzo buzers) to give textures

None of the above is going to give you a meaningful tactile sensation. Human fingertips are extremely "high-resolution/high DPI" sensors. If you have seen a Braille printer, that is closer to the idea (but still extremely crude). There have been attempts to use ultrasound, various pin cushion-like devices, even small electric currents etc. to simulate various textures and edges, but the work is pretty much in its infancy. Look on Google Scholar, there are quite a few papers published on this (search for "tactile displays").

BTW, forget the vibrating motors. Unless your goal is simulating the tremors of a Parkinson disease sufferer, it is not at all realistic. Think of it like getting an electric shock when you touch something live - yeah, you have got the information that you have touched something, but it certainly wasn't delivered in a way your brain expects...

Quote
you need strong and fast motors that can actually provide quite a lot of torque quickly

yes, this is one of my biggest concerns at the moment. so far steppers with their own processor (so you don't have to wait for the computer) is the best option I've come up with so far. and some other problems you also mention.

You seem to be under the impression that the problem is the speed of the host computer. That is, in fact, not the case. You will certainly have some sort of local controller to drive the motor, but there is no point in offloading the haptic rendering to the motors. It would only make the integration of the system with a graphic engine of your choice incredibly complicated.

The issue with the motor speed is different. Imagine your hand is moving towards an obstacle. What you want to have in the ideal case is a system that is behaving neutrally (potentially compensating for the friction/mass of the assembly, but not applying any other force) until you hit that obstacle. At the moment you hit it (you see yourself doing so), it needs to become rigid instantly so that the movement is stopped in the same way as a real obstacle would stop it.

That is obviously not possible because every motor requires some non-zero time to develop the torque to overcome any slop/backlash in the mechanical parts and to accelerate/decelerate any moving parts. The more powerful and faster spinning the motor is, the quicker the reaction could happen and more rigid will the apparatus feel. Of course, there is always a tradeoff - the bigger the motor, the heavier it will be, the more momentum it will have and more energy/torque will be required to quickly get it moving or to stop it. The same applies to any mechanical parts attached to it - they add mass, thus you have larger momentum and the motor must work harder.

This is tightly coupled with the update rate of the haptic system (from collision detection to the force rendering) - rates of 1kHz or even more are common here, otherwise the response times would be just too slow (compare to 60-120fps of a common game engine!). 

This is why small motors like the ones in your pictures are not going to work - they are too weak, so you must gear them down a lot which will preclude them from reacting quickly enough. It also shows why it is pointless to evaluate the "speed" of your system while driving it from something like the 555 timer - you need a closed loop where you can actually see yourself hitting an obstacle and then you can perceive the delay before the force feedback kicks in.

A sizeable stepper could potentially work, but then you cannot mount it  on the glove - that is why the CyberGrasp system has the "backpack" and tethers.

Quote
in short. yes, finding a way to convey force, fast, accurate, safely,  sleekly, and affordable. will be quite a challenge. I have some less conventional idea's as well, but they are far more complex (using ferro fluids & multiple sliders, etc) so I want to try with simple steppers first.

Those things don't have sufficient force. You could use hydraulics, but that only moves the problem to the sizing of the required pumps.

Quote
google PERCRO and haptics
Thanks! I don't think I've stumbled across them yet :)
(but I have seen that youtube vid. it's one of the better one's I've seen)

PERCRO in Pisa (http://www.percro.org/) are one of the world leaders in robotics and haptics. Again, you should do more research before you start designing and building. I know, it is more fun to just start hacking on it, but you will waste a lot of time solving problems that are solved already.

Quote
Another note:
I'm going to use the fact that this is for gaming to my advantage. rather then have the glove respond to every little thing in the virtual world. the game will load up tactile objects into the glove itself. and when the player reaches out to those objects + a grasping motion, those objects will snap to the players hand. and the gloves on-board (dedicated) processor will take over from there, until that object is let go.

Forget it. That would be horrible for the user. So I can touch this cube, but then my fingers can go straight through the desk, because you didn't "upload" the desk to the glove?

Do you want to embed a PC in your glove? Do you realize that haptic rendering is tightly integrated with collision detection/collision response of a typical game engine? A normal game could have hundreds of objects with tens of thousands of polygons that you will need to test for collisions against. Again, you are trying to address a wrong problem here.

I suggest that you google "haptic rendering" first, there are quite a few introductory papers and lectures available online that explain the basic concepts and show the requirements for a typical system. Only then it makes sense to actually start designing something. That something is "for gaming" doesn't mean it can ignore basic physics or how a typical VR/game engine works.

What you can embed on the glove/controller is closed loop control that will help to deliver the commanded amount of force in response to the finger movement of the user (so that you don't break neither the finger nor your gears). That needs to be very fast and going to the host would be way too slow for it. But the calculation of the forces themselves, which fingers to close, etc has to be done only at the host - you would need a very beefy embedded computer with a fast link to the host machine if you wanted to do all the calculations on board.
« Last Edit: November 08, 2014, 01:37:30 pm by janoc »
 

Offline janoc

  • Super Contributor
  • ***
  • Posts: 3781
  • Country: de
Re: Haptic Feedback glove
« Reply #8 on: November 08, 2014, 01:26:57 pm »
A suggestion:

If you have never used haptic devices, I strongly suggest that you get your hands on the Novint Falcon (the delta-like manipulator you have in the photo above).

It is about $250 (http://www.novint.com/index.php/store/novint-falcon-3d-touch-controllers/1) , but you can get it much cheaper on eBay.

There is an SDK for it, so I recommend you that you try to drive it and integrate it with a game engine - e.g. Unity3D or Unreal or whatever you like. The Falcon is conceptually much simpler than a human hand, it incorporates its own tracking system, so you have everything in one place and known to work. That will give you an idea of what is needed and how it needs to work for obtaining some sort of meaningful force feedback.




« Last Edit: November 08, 2014, 01:38:42 pm by janoc »
 

Offline PlagueDoctorTopic starter

  • Contributor
  • Posts: 13
Re: Haptic Feedback glove
« Reply #9 on: November 14, 2014, 08:49:13 am »
Sorry for the late reply. I've been busy with both my project & my day job.

Janoc

First off I'd like to thank you. you've helped me realize that the goal I originally set out for wasn't realistically achievable. especially not on my budget.
so, I've changed my game plan to building a haptic feedback glove, that people can easily tear apart & modify, using as much open source stuff as possible (and of course, anything i design myself will also be open source) that way people that are interested, have a cheap option to get them started, so they don't have to start from scratch (if that makes sense)

made a little video about it on my YouTube channel:
http://youtu.be/YP3ON9rSn1k?list=UU2t5s_zdMJpIFKuQC31vUCA


Quote
first, it is an extremely niche

I plan to have at least the tracking part work with any software that works with the razor hydra (they might not like it, but I don't think it's illegal either)
and I'm absolutely not in it for the money, and have no interest of making this into a company. I just love VR, and believe I have sufficient knowledge to add a bit to the growing community around it. (though by no means am I an expert, I do have experience in programming, machining, circuitry designing & building etc)

Quote
forget the vibrating motors

yeah, the motors will probably be to slow, I was thinking some type of device to send timed vibrations at precise timing to add texture to virtual objects (a bit like, how when the phone gives a small vibration when you hit a button, to make it feel like you are actually touching a something other then a flat screen)

Quote
You seem to be under the impression that the problem is the speed of the host computer

That is indeed my first concern. because the motor, unlike when it's in a CNC mill, where it's moving a heavy router back and fourth & loosing steps is a really bad thing.
in the case of the glove, it only has to so one or two steps per movement, and with the torque I want, I don't think it will be a huge problem (but of course, I could easily prove ,myself wrong. wich is why I want to keep my glove modular, so I can easily bolt on different things, and see how well they work (ok, it won't be fully plug & play modular, but at least have it fairly easy to change it up)

Quote
Those things don't have sufficient force

I'll find something. even if I have to make i myself.

Quote
I know, it is more fun to just start hacking on it

Yes it is :p. and also, a more practical reason why I don't dig too deeply, is that I find if I look to closely at how other people do it, I have a harder time thinking outside the box. I just want to look around and get a really rough Idea. but I don't want to look much closer until at least I have tried several different things.

Quote
That would be horrible for the user. So I can touch this cube, but then my fingers can go straight through the desk

more likely to be the opposite. flat planes are easy enough to write collisions for, and can easily be handled by the computer
Quote
tens of thousands of polygons that you will need to test for collisions against
and that's exactly why I was thinking of taking some of that load off.. but then again, looking at Nvidia's turf thing, it seems newer computers are much better at handling collisions then anything I have programmed before.

so, I guess, for now, for simplicity's sake, I'll let the computer deal with all the feedback data. and just have the glove process it's positions, and translating messages sent by the computer.

Quote
I suggest that you google "haptic rendering" first

That defiantly is something I need to do (something I didn't realize there was so much info out there already, I was planning to start from scratch :p)
but I see no reason why I can't work on the mechanical construction first, since I can easily re-program the glove to send & receive it's signals differently if need be.


Thank you so much for the in-depth reply. and yes, I've taken a lot more from your message then what I've replied here :)
Thanks!

janoc (part 2)

That is indeed a great Idea! and something I will do once I have the money to do so (right now, with winter coming, I barely have enough to keep food on the table & a roof over my head :p (hence why I'm only using parts I have kicking around, and why I'll be asking kickstarter, and see if people are willing to help me with some funds for this project (if not, that's fine too, it'll just make progress go extremely slow)
 

Offline janoc

  • Super Contributor
  • ***
  • Posts: 3781
  • Country: de
Re: Haptic Feedback glove
« Reply #10 on: November 14, 2014, 03:15:06 pm »
Sorry for the late reply. I've been busy with both my project & my day job.

No worries, we all know how that is.

First off I'd like to thank you. you've helped me realize that the goal I originally set out for wasn't realistically achievable. especially not on my budget.
so, I've changed my game plan to building a haptic feedback glove, that people can easily tear apart & modify, using as much open source stuff as possible (and of course, anything i design myself will also be open source) that way people that are interested, have a cheap option to get them started, so they don't have to start from scratch (if that makes sense)

I am sorry if I have put you off. My feeling was that you had very little experience with the subject and fairly unrealistic expectations. Haptics in general is tough subject, requiring good knowledge from electronics, robotics, mechanical engineering and VR. It is certainly possible for a hobbyist to enter into it, but you need to start small, gain experience and knowledge first and only then move to more ambitious projects.

I plan to have at least the tracking part work with any software that works with the razor hydra (they might not like it, but I don't think it's illegal either)

If you don't plan to sell it, you can do whatever you want. On the other hand, if you plan to develop a magnetic tracker like the Hydra and sell it, better budget a lot of money for patent licenses and lawyers. Immersion, Ascension Tech, Sixense and Polhemus own many patents pretty much covering the field here and are happy to sue.

I would probably advise against trying to develop a magnetic tracker - optical (using cameras), ultrasonic or hybrid (e.g. optical + IMU) would be much simpler. Magnetic tracking is very complex, the signal processing involved requires both complex electronics and some highly non-trivial math and physics. Out of the existing tracking technologies it is probably the most complicated one to implement, especially if you don't have pre-existing background in the field.

and I'm absolutely not in it for the money, and have no interest of making this into a company. I just love VR, and believe I have sufficient knowledge to add a bit to the growing community around it. (though by no means am I an expert, I do have experience in programming, machining, circuitry designing & building etc)

That is very cool, VR is a very interesting and rewarding field. Just be prepared for a very steep learning curve :)

BTW, I recommend  you to join the VR-Geeks mailing list: http://www.vrgeeks.org/ It is a group of people like you, interested in the technology and what can be done with it. There are some fairly high-profile names on the list as well, such as Palmer Luckey (Oculus founder), people working on the French Ariane rockets (their IMU stuff is just completely nuts compared to what we do in VR!), VR pros from places like Dassault or Immersion, etc. It is a good place to throw this sort of idea like you have around, get feedback and even find collaborators.

Quote
forget the vibrating motors

yeah, the motors will probably be to slow, I was thinking some type of device to send timed vibrations at precise timing to add texture to virtual objects (a bit like, how when the phone gives a small vibration when you hit a button, to make it feel like you are actually touching a something other then a flat screen)

The problem with the vibrating motors is not really the speed. They can be made to react reasonably quickly - especially the tiny cellphone vibration motors. The issue is that the sensation is not at all resembling anything real - try to put your finger on a vibrating cellphone and imagine this is the feeling you would have when hitting some surface. Vibrators are OK for things like force feedback in a gamepad or steering wheel simulating a bumpy road, but not really somewhere where you need to feel object textures. Unfortunately, there isn't really a good solution for this problem yet.

One interesting workaround is the "pseudo-haptic feedback" invented by Lecuyer
(http://people.rennes.inria.fr/Anatole.Lecuyer/vr2000.pdf and http://people.rennes.inria.fr/Anatole.Lecuyer/chi2004.pdf Youtube: )

Quote
You seem to be under the impression that the problem is the speed of the host computer

That is indeed my first concern. because the motor, unlike when it's in a CNC mill, where it's moving a heavy router back and fourth & loosing steps is a really bad thing.
in the case of the glove, it only has to so one or two steps per movement, and with the torque I want, I don't think it will be a huge problem (but of course, I could easily prove ,myself wrong. wich is why I want to keep my glove modular, so I can easily bolt on different things, and see how well they work (ok, it won't be fully plug & play modular, but at least have it fairly easy to change it up)

You are approaching this from the wrong angle. The CNC motors move relatively slowly and move significant masses.  On the other hand, in a CNC everything is tightly controlled and you are operating at slow speeds. That is not the case in the haptic case. There you have no control over what the user will do, so you must be able to react quickly so that the illusion is maintained for the user. Also issues like missed steps are a non-issue, because you must use some sort of position/angle encoder anyway to track what is the user doing with the device. It cannot be open-loop control like with cheap CNCs (the large ones use servos - motor+encoder, not stepper motors) - if not for anything else, then to not break the user's fingers (or fry the motors)!

The host machine can easily detect collisions even 1000x a second (using a simplified, low polygon scene - one never uses fully detailed meshes here), however the motor position has to be updated at 1-10kHz update rates if you want to be able to react rapidly to the collisions. That typically requires lightweight actuators (no large moving masses!), lots of torque (to be able to quickly change speeds and directions), tight control loop using a dedicated controller. No PC can handle 10kHz update rate with real-time constraints. This is typically a job for some sort of a micro-controller. In that case your PC identifies the possible collision (too complex job for a micro except for very trivial scenes - not enough CPU power and mainly RAM) and either commands the controller to produce X newtons of force immediately or, if the controller is a bit smarter, it will do X newtons when the actuator's position is e.g. <N degrees and Y newtons when it is >N degrees (assuming rotating joint here, but it can be generalized to other types). The position is affected by the user moving the device to or from the obstacle. Or something similar, depending on the exact architecture of your device.

Yes it is :p. and also, a more practical reason why I don't dig too deeply, is that I find if I look to closely at how other people do it, I have a harder time thinking outside the box. I just want to look around and get a really rough Idea. but I don't want to look much closer until at least I have tried several different things.

That is ok, but you should be aware of what the requirement are, what exist and what approaches were taken - if not for anything else, then to avoid wasting time by repeating past mistakes of others. Learning from your own mistakes is a very effective way to do so, but also the most expensive one :)

Quote
That would be horrible for the user. So I can touch this cube, but then my fingers can go straight through the desk

more likely to be the opposite. flat planes are easy enough to write collisions for, and can easily be handled by the computer

My point was that if you only upload some objects to the "haptic controller" (whatever that is), then it will inevitably happen that the user will encounter an inconsistent experience - some objects are solid and others are "ghosts", for no discernible reason. That is very jarring. Uploading the entire scene on a microcontroller (see the discussion above) is just not feasible, not to mention that the controller needs to communicate fairly quickly with the host PC in order to respond to tracking, user's actions, environment changes, etc. And a "big" computer (PC, RaspberryPi or something similar) is not suitable here - you need realtime capabilities for the motor control which these cannot provide (again see above).
« Last Edit: November 14, 2014, 03:18:24 pm by janoc »
 

Offline PlagueDoctorTopic starter

  • Contributor
  • Posts: 13
Re: Haptic Feedback glove
« Reply #11 on: November 25, 2014, 07:57:01 pm »
Wow, I've been away for longer then I meant to, ah well, I've set myself a scheduale now that I should be able to adhere to.

(to see my latest update you can see it here https://www.youtube.com/channel/UC2t5s_zdMJpIFKuQC31vUCA)


Just the first 'prototype' base to witch to mount the electronics on to.

janoc

You haven't put me off, you've just aligned my wheels a bit :) and I have some experience in electronics, robotics, mechanical engineering, and programming (though I admit, I lack experience in VR, but I intend to remedy that)

Quote
I would probably advise against trying to develop a magnetic tracker

I had no intentions to, I was thinking along the lines of optical & gyro's. I just meant that I wanted either the driver, or the raw data output to partially mimic the hydra, so that (for example, Half life 2) could work with it. but it was just an idea I threw out there, I haven't put to much serious though in that department yet.

and does giving it away as a kickstarter support perk count as selling it? (that's the only time I would be excepting money, and people getting the glove in return)

Quote
Just be prepared for a very steep learning curve :)

I always dive right in to the deep end, I usually find a way to at least tread water in the various subjects. (it's easier to tackle the learning curve that way, I find)
ex. when I taught myself programming, my first program was a 2d fighting game :)

Quote
The problem with the vibrating motors...

it was just an Idea I wanted to try and see the results :)
and thank you for the reading material, I will likely reproduce some of their experiments.

Quote
You are approaching this from the wrong angle...

that was partially the reason why I was thinking of having objects stored in the glove, when the computer notices the hand getting close, it sets the glove to take over the control in a closed loop to speed up the updates, but true, the more I think of it, the less practical that seems.

and I by no means expect a perfect response time in the feed back, solid surfaces will probably feel a bit spongy as the feedback motors compensate for the delayed response times. but I recon that's a problem I can solve with the help of the open source community once I get to that point. (again, I hope :p)

Thank you for your continuing support!
Thanks to you I feel like I'm making my end goal more realistic and achievable (by not aiming for perfection).
 

Offline janoc

  • Super Contributor
  • ***
  • Posts: 3781
  • Country: de
Re: Haptic Feedback glove
« Reply #12 on: November 25, 2014, 10:50:07 pm »
You haven't put me off, you've just aligned my wheels a bit :) and I have some experience in electronics, robotics, mechanical engineering, and programming (though I admit, I lack experience in VR, but I intend to remedy that)

Phew :) I am relieved :-p

Quote
I would probably advise against trying to develop a magnetic tracker

I had no intentions to, I was thinking along the lines of optical & gyro's. I just meant that I wanted either the driver, or the raw data output to partially mimic the hydra, so that (for example, Half life 2) could work with it. but it was just an idea I threw out there, I haven't put to much serious though in that department yet.

Ah, ok. That makes sense.

and does giving it away as a kickstarter support perk count as selling it? (that's the only time I would be excepting money, and people getting the glove in return)

I think that would pretty much depend on how well the company lawyers at Ascension or Polhemus slept that day ... However, if money changes hands, I would say it means that the patent law will be applicable, but IANAL - you would have to check with an IP lawyer in your jurisdiction.

On the other hand, why would you bother working something like that around? There are other, easier technologies. Moreover, even if you have managed to somehow avoid the patents in one country, they may still be applicable in another (yay, the wonders of different legal systems!) and, for example, in the US if you lose a patent litigation and your opponent could show that you were aware of the patent (e.g. because they have found this thread by googling), you are liable for treble damages ... Stay away like from a plague, IMO.

and I by no means expect a perfect response time in the feed back, solid surfaces will probably feel a bit spongy as the feedback motors compensate for the delayed response times. but I recon that's a problem I can solve with the help of the open source community once I get to that point. (again, I hope :p)

Don't rely on the community to fix what could be a poor hardware design.
A) most hobbyists don't have the specialized knowledge needed, b) the problem might be unfixable without reworking the hardware in such case.

The trouble isn't so much the "sponginess" (which is merely unrealistic and annoying), but if there is a significant lag between what you see (your hand hitting an object) and what you feel (when the motors finally deliver the force - weak motors have to be geared down, thus it will take longer!), it will break the immersion and sense of presence completely. If the delay is beyond some milliseconds, the brain is not going to associate the two perceptions anymore and will treat them as distinct, unrelated events - leading to a mighty confusion for the poor user. That's the reason for the high update speeds in haptic systems.

 

Offline miguelvp

  • Super Contributor
  • ***
  • Posts: 5550
  • Country: us
Re: Haptic Feedback glove
« Reply #13 on: November 25, 2014, 11:40:40 pm »
About patents.

Say I find prior art from some state funded university, so not sure how they would be able to protect their IP if it's already public domain, or has that changed lately? Or even declassified governmental research (military, aerospace, etc..) that is public because the public paid for it.

Anyways, if you find public domain prior art, you can submit that prior art with your patent application just to have it registered, no lawyers just the fee which can't be much.

It won't get granted but it will show the prior art and be on record, and if the prior art precedes what the competitor's patent is, they will have a hard time stopping you because your defense is already filed at the patent's office proving that you are not basing your work on that competitor, but off some public domain prior work that might even nullify their patent if they try to enforce it upon you.

Would that work?
 

Offline PlagueDoctorTopic starter

  • Contributor
  • Posts: 13
Re: Haptic Feedback glove
« Reply #14 on: November 26, 2014, 02:25:17 am »
Quote
About patents.

Say I find prior art from some state funded university, so not sure how they would be able to protect their IP if it's already public domain, or has that changed lately? Or even declassified governmental research (military, aerospace, etc..) that is public because the public paid for it.

Anyways, if you find public domain prior art, you can submit that prior art with your patent application just to have it registered, no lawyers just the fee which can't be much.

It won't get granted but it will show the prior art and be on record, and if the prior art precedes what the competitor's patent is, they will have a hard time stopping you because your defense is already filed at the patent's office proving that you are not basing your work on that competitor, but off some public domain prior work that might even nullify their patent if they try to enforce it upon you.

Would that work?

That sounds to me more like how you protect your copyright, only instead of going through all that registering stuff, you simply mail your idea to yourself, that way the post office tags the letter & dates it, holding up quite strongly in court.

and I'm not sure if that's as simple to do here in Canada, but it look into it a bit more.
 

Offline janoc

  • Super Contributor
  • ***
  • Posts: 3781
  • Country: de
Re: Haptic Feedback glove
« Reply #15 on: November 27, 2014, 08:04:44 pm »
About patents.

Say I find prior art from some state funded university, so not sure how they would be able to protect their IP if it's already public domain, or has that changed lately? Or even declassified governmental research (military, aerospace, etc..) that is public because the public paid for it.

Anyways, if you find public domain prior art, you can submit that prior art with your patent application just to have it registered, no lawyers just the fee which can't be much.

It won't get granted but it will show the prior art and be on record, and if the prior art precedes what the competitor's patent is, they will have a hard time stopping you because your defense is already filed at the patent's office proving that you are not basing your work on that competitor, but off some public domain prior work that might even nullify their patent if they try to enforce it upon you.

Would that work?

No, it wont. You are assuming that the patents are not granted yet. Which is not the case.

The tracking patents in question are awarded and have been enforced for many years, it is not something that was patented recently. I believe that the original research dates back to the 70s or so and the patents and their derivatives covering the various variants of the magnetic tracking are from the early 90s. The oldest patents have likely expired or are about to expire, but the more recent stuff dates to the time around the year 2000, so still valid.

I am not a lawyer, even less an American patent attorney, so take this with a grain of salt.  If you find something that you claim is a prior art and the patent was awarded already, then your only options are either to ask the patent office for a re-examination of the patent (good luck with that, but some did succeed) or have the patent invalidated in court (reaaallly good luck with trying to sue a big company over a patent ...). Unless you prevail in court or the patent gets invalidated by the patent office, your claims of prior art are pretty much worthless as a defense.

 

Offline janoc

  • Super Contributor
  • ***
  • Posts: 3781
  • Country: de
Re: Haptic Feedback glove
« Reply #16 on: November 27, 2014, 08:15:44 pm »
That sounds to me more like how you protect your copyright, only instead of going through all that registering stuff, you simply mail your idea to yourself, that way the post office tags the letter & dates it, holding up quite strongly in court.

Here in France you don't have to register a copyright in order have it - authors have automatically copyright over their creations, thanks to Berne convention. However, you can voluntarily depose material with the government (similar like depositing something with a notary). In case of a litigation it can be used as a proof of your authorship.

Concerning patents, if you want to make sure that someone else won't patent your idea, then the best defense is to actually publish it - for example in a scientific or engineering journal. In that case you have established a strong prior art for the future, should some patent troll try to extort money from you with a bogus patent based on your work or even attempt to obtain such patent - the patent examiners routinely search these journals and should deny such application.

The obvious other option is to patent yourself, but that is an enormous waste of time and money if you don't have the resources to defend the patent in court (potentially millions of $$$). The patent alone is only a scrap of paper where some governmental office certifies that you have done something first and should have some limited monopoly over the idea for a while, nothing more. However, they won't help you enforce it - that is up to you.



 


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf