Minority Report style glove

[Knar]

Minority Report style Glove

Enjoy!

[Jake]

[Knar]

Not the Nintendo glove. An upgraded version, where you can point and grab stuff. Gah, I knew this joke would have come up.

[Jayel]

Damn that power glove thing looks uncomfortable.

Looks like you'd have to dislocate your thumb just to wear it.

[vimes]

Last year, in the course of a computer science project, I decided to use a dataglove with vibrator attached to a 3D captor...

the problem was to find a kind of application in which the use of a glove would clearly bring an added value.

Believe le it's hard, because, as much as I fancy this kind of device, 90% of the application would be far easier to use with a mouse and a keyboard.

The original purpose of the team I worked with was to create a" Minority Report' application but, in the end, manipulating data in a 3D environment with a mouse is far easier and also more user-friendly... The only intelligent use of that device was to include it in a VR application that would help behaviour therapists in making their patient grow more 'confident' with the physical symptoms of their phobias ( vertigo, arachnophobia, boulimia)...

Well, that was a wondeful idea but I was the only guy who had computer science background, I was named project leader and I sucked at this task so the project was a disaster... nice idea though*sigh*

[Knar]

I would think gloves in a 3d environment would be more intuitive than a mouse and a keyboard.

Naturally, when I see a bunch of stuff on top of each other, I would rather pick them up and move them around. A mouse and keyboard may be great for most stuff, but wouldnt' you want to be able to rotate data or point things out?

Hands are a lot more precise than a mouse and keyboard. I mean, heck I've tried drawing with a mouse, and it looks like crap.

So can you explain more about your project? I'm intrigued.

[Salka]

ah but for drawing you would use a wacom. i can draw with a pencil, but i wouldn't draw very well with a piece of lead attached to the forefinger of a glove...

[Savage Cabbage]

Actually, you probably would. A pencil is really nothing more than an extention of your pointing finger.

Drawing on a Wacom could never be as natural as drawing direct to screen with a glove.

There are very few things I could see it being used for but I think it would definately be more precise for an artist for precision work, or like Knar said, for flipping things etc in 3D.

[vimes]

I would think gloves in a 3d environment would be more intuitive than a mouse and a keyboard.

One of the first thing that we realized is that for one to use a dataglove, a VR helmet or a giant screen is needed. Why ? Because the things you act on must be at the same scale as the device you interact with... what I'm trying to meant is that using a dataglove with a screen - even if 30 inches wide one - couldn't work because the movment of the user wouldn't be in the same frame as the user. We tested it... it sucks.

That is the first limitation.

The second one is that if moving a virtual hand around in a virtual world is easy and very efficient.. interacting with the world is a different thing. Here, I'm not talking about physics (grapsing an object releasing it or the effect of the user's move on the dynamic objects)... that's easy stuff, once you have the appropriate technology.

What I'm saying is that if you want to grab a file, then going in its properties and messing with them.. or open it with a specific program.... you have tu use either hand-sign identification or buttons. If you use the latter, you end up with a system similar to the mouse device but that brought forward other problem : where to put the button is the greatest. The location must be one that must at the same time guaranty :

1- that the user won't have to search for it

2- the user won't have to move his gloved hand to reach it [i.e. by doing so changing the state of the virtualhand within the virtual world]

If you use hand-sign, it might sounds cool at first, but in the end, it's less efficient than the mouse system. It's cool, and I'd like it to work but from what I've experienced that's not worth it.

But there is other problems :

- browsing menu (or doing precise pointing) menu with a dataglove is a pain in the ass

- representing the virtualhand is problematic because it more or less hide 1/4 of the content you're pointing at.

So can you explain more about your project? I'm intrigued.

Yeah, of course!

So, in the 2nd year of my boring generalist engineer school, we have to lead a project with a team on a topic of our choice. The highlight of the school is mainly on team managment....meaning, they focus on you're ability to meet official deadlines rather than the final product or the way you evolved during the 6 month of the project... anyway, I had done this year once already and had participate in a project called JediOSE. This was a geeky concept of a lightsaber simulation: we would put a 3D captor on a handmade woodstick so that the movements would be reproduced in 3D and we would had drones that would fire laser and the player had to deflect them by moving the lightsaber in the right direction. And the drone would explose of they were hot by a reflected laser.

I promess, we had an empty environment, we hadn't worked that much on the 3D model but it was AWESOME!

The end of the story is that I failed at my final exam in Physics so I had to do it all over again...

Doing VR was fucking interesting and i decided that I would try to recruit some people to work on a non-gaming VR application... 3 guys were interested and we went through all the stuff that was made with a glove : the project of nintendo, chirurgical operation, VR training for people suffering from trauma after having an accident at work...and we ended up watching a show on the very good french-deutsch channel ARTE about several nordic Universities that were developing VR applications to treat physical symptoms of phobias.

The goal of those therapies is not to search for the source of the trauma : why you hate spider, why you can't stand porcelain... that sort of thing, that 's the job of the therapist called 'psychanalystes" in french[freud and such]. The goal of those sessions is to make the patient bear with the fear or the nausea he's feeling while exposed to a anxigene situtation. That helps him/her to live with his/her problem.

So, because we wanted to use the dataglove we went for arachnophobia because if patient's crisis are triggerred by such simple things as 'thinking there is a spider in the room".. the final stage of confrontation is making the patient take a spider in is hand without feeling the anxiety anymore.

The main problem was that we needed medical backup to make our project sound serious and, as we went from clinic to clinic we would find people that were very interested in our approach but didn't have time to give us and feedback.

The only ones clearly motivated were therapists from a addictology clinic that were dealing with patient suffering from anorexia and drg addictio... and they wanted us to design a program for the boulimia group. As they were the only one kind enough to support us, we went that way. The thing is that they didn't want to work with us on the design doc , on the analysis and they didn't deven bother sending us back the we sent them... because they mainly wanted to correct us on errors related to medical approach (Them:" You should simulate, like, I don't know.... puking!" Us: " Really, that's... hummm.. disgusting but, well, if that helps..." Them: " Yeah, do it and then we'll see if it works!" Us: "We are students, it's not like we have the time to develop useless feature" Them :" Cooome-oooon! Puking!.. it will be fun!").

They didn't have the time to give us a main orientation.

So, how the project failed, that's an interesting story, but quite related to that.

[Erkki]

I think a webcam that is able to identify hand gestures sounds more plausible (and it's been done, too -- http://www.cs.ucsb.edu/~matz/HGI/HandVu.html). Lots of people have webcams, but I don't think a lot of people would buy these gloves.

I think gesture technology is important though, but I don't see it as a complete replacement of the mouse/keyboard. I work as a programmer and I'd like to see gestures integrated into the IDE I use. I have already tried a simple mouse gesture plugin and it works very well -- Instead of memorizing lots of shortcut keys or which menus to open, I can execute some of the most common tasks by drawing a certain gesture (the plugin only supports orthogonal movements) -- which is a lot easier to remember for me.

An even better way to run some of these commands would be to could just lift my hand slightly from the keyboard and draw the figure with a finger or make a sign or something, instead of having to grab the mouse.

[Knar]

, I had done this year once already and had participate in a project called JediOSE. This was a geeky concept of a lightsaber simulation: we would put a 3D captor on a handmade woodstick so that the movements would be reproduced in 3D and we would had drones that would fire laser and the player had to deflect them by moving the lightsaber in the right direction. And the drone would explose of they were hot by a reflected laser.

Strangely enough, I've seen this game in the arcade. It's actually a star wars game where I believe you use a lightsaber and sensor to simulating fighting with Darth Vader. LucasArts stole your idea and sold it.

These days, at the expensive arcades, I'm seeing more and more use of virtual reality via the use of a screen and sensors. There's this one game about being a Policeman where you dunk and shoot bad guys. Although the sensor array seems to be a bit basic (the response is slow and movement seems so jerky), it is a cool concept.

When I think of VR, I get flashbacks from Star Trek and the holodeck. Granted, the interface seemed so simple, and somehow perfect objects would appear out of mid air from a few simple commands, but it made my imagination run wild. Of course, there were numerous episodes about when the Holodeck's safties failed, nearly killing the users, but the idea of a creative interactive world is intriguing.

I can see VR being a great way of perhaps doing surgery, overlaying specifications to complicated hardware or just having a 3d environment to interact in.

I'm not quite sure how the clinic wanted to use your project though. By simulating something as complex as bulimia, they're asking you to somehow tap into someone's nervous system, to make them feel the nausea and smell their own vomit. I know simulating smells are a hard thing. And to feel sensations. I mean, images on a screen can go only do so much, but creating a unique smell for your world would mean a very complex and complicated mechanism.

_--the gloved hand--

I think we're not exactly doing away with a keyboard and mouse altogether. Wouldn't it be more efficient to use both a glove and keyboard. I mean, I understand the level of interactivity they are looking for, to be able to grab your windows and what nots. But shouldn't there also be a keyboard to type things in?

I can see this glove system being used by the military. I can see a 3d model of fighter jets and trying to move and change their trajectory from that. Actually, that's one type of interface I would love to see being used in Sci-Fi cinema, the use of 3d screens. Wouldnt' it make more sense?

[Thrik]

This type of technology desperately wants integrating into software such as 3D Studio Max. Imagine being able to scuplt 3D characters and objects as if they were physical objects, rather than endlessly creating and tweaking individual vertices/planes for hours on end. I mean, seriously. It'd be one of the most important and significant advances in anything ever.

[vimes]

This type of technology desperately wants integrating into software such as 3D Studio Max. Imagine being able to scuplt 3D characters and objects as if they were physical objects, rather than endlessly creating and tweaking individual vertices/planes for hours on end. I mean, seriously. It'd be one of the most important and significant advances in anything ever.

That already exists in a very archaic form in a norvegian institue : theyr created an application that display a toolset of very simple primitives [ box, sphere, pyramid] that you can stuck together to create a general shape. Then you can use a 'pick' that allows you to sculpt this shape. It's quite interesting but there - as in many VR application - the main problem is that it lacks the physical feedback of the touch which I understand is very important in sculpture.

I think a webcam that is able to identify hand gestures sounds more plausible

Yes, but there is still a great limitation which is that, by printing VR object on real environment, there's no way you can define background object, foreground objet etc : for example you can't say that this VR objets will disappear behing that element from the real picture.

I'm not quite sure how the clinic wanted to use your project though. By simulating something as complex as bulimia, they're asking you to somehow tap into someone's nervous system, to make them feel the nausea and smell their own vomit. I know simulating smells are a hard thing.

Arf, I wasn't very clear on that point. Crisis of boulimia or any phobia in fact, are triggered by the mere suggestions of the object of phobia; so, so we didn't have to create nausea or tap into the nervous system 'cause even if the graphs weren't realistic the fact that it represented the object of fear triggered the anxiety. VR was just a mid-point between total suggestion and dirrect confrontation.