[frankvl]

I'm creating a Jeff Han-style multitouchscreen table using a 15" lumenlab-style projector, for my everyday needs as a software architect/programmer (i.e. to replace my laptop interface when I'm home), to have a very nice aim in games (I think it's called cheating), perhaps enter the wonderful world of 3D editing now that it can actually be made intuitive, and to be able to paint electronically with real brushes (in 3D of course)

If I can find a good 1-way diffuser (or alternative) I might also add a webcam to track the color of objects after they have been detected on the screen, e.g. to recognize a playing card or perhaps even printed text if I can get a really really good camera setup. Or a video conferencing system might be possible where you can watch directly in the camera and the screen watches back.

Touchlib is (one of the) open source libraries for this technology, but I'll be writing my own program in Java (it's very easy actually). I want a minimalistic implementation where the user can teach the computer which gesture is connected to which data. OCR also becomes very easy if you can track the movement. Of course you dont have to use actual handwriting; you can use something that's a lot faster (not sure whether it will actually be faster than typing though). An on-screen keyboard could follow your hands or you could put tiny knobs on the screen to type blindly, with or without the (full) keyboard displayed. You could write/draw on real paper on top of the screen (have a paper roll next to the screen) and still see the screen + digitize the drawing. I think. So let's find out...

Here's a link and some pictures of existing designs; Google multitouchscreen for more if you're interested (lots of different projects on YouTube and Flickr)






The parts are on their way:

exclusiv.com:
- 400w light set: 94 Euros
- 15" optics: 75 Euros
- front surface mirror 154x93x4 mm: 16 Euros
ikbenstil.nl:
- Thermaltake Silent Wheel (120mm 16dB cooler): 12 Euros
- soft silicon bolts: 3 Euros
(bought a 2nd cooler with bolts just in case)
Led1.de: (free shipping)
- 100 8 mW IR leds: 40 Euros
(20-40 should do but it's nice to have spares)
Trust:
- 120 Sp@cecam 380x240 30fps: 20 Euros
(converted to IR cam by replacing IR filter with black photo negative)
- acryllic plate 8mm 150x100 cm: free (normally +/- 200 Euros)
(you can save 100-150 on the plate by using a smaller and/or thinner one)
- small acryllic plate to test mounting + software on: free
- paper to diffuse image on the plate: free
Stuntstore (local shop that also sells & ships on eBay):
- Eizo FlexScan L365 15" 1024x768

Total: 370 Euros!! (incl +/- 20 Euros shipping).

I can get the other electric + aluminium casing/table materials for free and I have access to advanced tools and an expert mechanic (my dad), so this is going to be interesting

I think I'll have a separate box for the ballast/fan adapter, mount the mirror on the outside and create a 2D keystone adjustment for optimal flexibility. I want to have the projector on the ground behind the (adjustable) table and the table to be nice and low (at 45 degrees) so I can just put my legs underneath it (instead of a large 2nd mirror). The table should become easily transportable so I can show off (and sell some to local businesses/institutions/gadget freaks in the process).

So far I've only played around with software and stripped the Eizo.

Pros:
- 450:1
- VGA+DVI
- Strips easily
- This particular model came with a 2kg stand that can rotate in all directions (with some alterations it can reach 2x360 degrees + 180 degrees tilt). It can rise 10cm and it won't be pushed over easily under normal conditions. Last but not least, it has a system to hold your incoming wires so there's no stress on the connectors. Absolutely perfect stand/mount for a diy projector!! Judging from the pictures on the Internet, not all 365's are equipped with it though (its feet are placed in a distinct V-shape).
- The buttons are fixed to the PCB, so they can be easily installed in the projector
- 100-120/220-240V, 50-60 Hz built-in adapter (with fuse)
- Came with 2m VGA extension cable (2 males) + removable 2m power cord + 2m audio extension cable (2 male 3.5" stereo)
- Came with 2 minispeakers that can be reused for your mp3 player
- Backlight consisted of 4 mini-tubes that you could use to lighten up a painting or something.
- The frame can be put back on the LCD glass after the backlight is removed; the frame contains a few holes above the screen which can be used for easy mounting.
- I don't know if this is the case for all LCDs, but the backlight contained a 15" 6 mm acryllic plate which you can use for multitouch (testing) after you'd polished off the diffusing dots.
- The plastic front pane can be reused to mount a fresnel and/or the filter.
- You could also dremel the center of the back pane to yield a fresnel mount.
- You could reuse parts of the polished aluminium panels to reflect the light on the sides of the lamp.
- The backlight component contained 2 diffused but see-through sheets, 1 white diffusing sheet and 1 fresnel sheet - nice experiment material...

Cons:
- XGA, 25 ms
- LCD needs a 16-pin FCC extension
- More difficult and dangerous to mount the built-in adapter
- Adapter cannot be replaced with external adapter or mounted elsewhere because it has multiple connections(/output voltages?)

I'll go off and remove the anti-glare now and find a cheap alternative to extend the FCC. I'd love to hear other people's multitouchscreen experiences/plans/ideas!! If you don't have them yet, now is the time

This post has been edited by frankvl: 12 January 2007 - 11:46 PM

[Billy A]

I am really interested in seeing how your project turns out. I always like to see innovation.

[athomasbeck]

I made one of these touchscreens a couple months ago. I was able to get webcam to do blob detection, but I know next to nothing about programming so I wasn't sure how to go about implementing it. I used processing which is supposed to be a java-like beginner language but still too much for me. The best I could figure out was using the java robot to controll the cursor. That seemed like it was a shoddy way to do it so I never tried.

The hardware is incredibly easy to put together. Once I had the materials I had it together in two days. I don't have a projector though so that’s strike two. I'm going to build a lumenlab one soon though so I'll be watching yours closely and maybe I could bum some software off of you once you get going.

[frankvl]

It's very easy if you have programmed some things before or are willing to spend some time learning it; it can be done in any language. In Java it would be about 500-1000 lines of code, or 1 day of work if you have some experience. Avoid existing API's as much as possible because they're often ill-designed and more difficult to work with than actually implementing the thing yourself.

This is the basic detection algorithm:
- capture the webcam data in a pixel matrix (preferably 256 grayscales)
- look for colors brighter than a certain threshold
- once found you check to see if the neighbours are also bright, and repeat this step until there are no more
- count the nr of bright pixels to determine the area/pressure of the contact and determine the middle, average or topleft coordinate
- use that data in your own applications or send point and click events to your operating system

If you get weak results, adjust the webcam properties and/or the threshold to look for. You could also add simple pattern matching algorithms to determine the shape/rotation of the contact, and use the data of multiple contacts and frames to determine which gesture was/is being made (incl OCR) and to estimate which contact might represent which finger (in certain gestures). These can be done in real time, probably still getting 30 fps @ 380x240 on a less than 8 year old computer. Above that all, my particular goal is to have a simple gesture configuration/interactive learning interface so that future users don't have to muddle around with code and annoying interfaces.

Calibration can be done by letting the user touch the corners of a rectangle on the screen; this data can then be used by the software to position, scale and keystone correct the webcam data.

I'll post some code as soon as I get something useful.

This post has been edited by frankvl: 10 January 2007 - 11:46 AM

[frankvl]

Soaked the anti-glare of the Eizo using a rag and tapwater; like stealing candy from a baby!
Ordered 16 way 0.5mm ribbon cable + connector to solve the FFC problem; 4 Euros at Farnell In One.
Coolers should arive this evening, optics and lamp are @ UPS.

Now all I need is patience...
and a good 2d/3d optics applet to fiddle with the design!

I will also buy 3D gamer glasses (+/- 40 Euros) for extra fun

This post has been edited by frankvl: 10 January 2007 - 07:53 PM

[frankvl]

The Thermaltake Silent Wheel is definitively what it promises!! you can only hear the buzzing sound of the motor, if you are closer than 1m that is... Also very cool; probably not the best lateral sucker since it has openings on the side, but I think one of 'em will do the job just fine.

Also polished a 19" 10mm thick acryllic testing plate and hooked it up to a usb-powered ir led from an old remote. This is what I got; the pictures were taken under normal artificial room lighting (2x60 watt dimmed light shining on the ceiling)

Finger touching but not pressing the plate:
 2.jpg (22.81K)
Number of downloads: 72

Finger firmly pressing the plate:
 1.jpg (23.64K)
Number of downloads: 66

Overview shot (location of the plate indicated with white lines):
 pressingplate.jpg (34.4K)
Number of downloads: 57

This post has been edited by frankvl: 10 January 2007 - 07:51 PM

[frankvl]

 notpressingplate.jpg (35.36K)
Number of downloads: 54

View of the outgoing light (this side had not been polished; the other side was done with car polish btw, which worked very fast and easy)
 outgoing.jpg (23.26K)
Number of downloads: 48

Me and my lamp. Something tells me it doesn't do its lighting job very efficiently You probably have to turn all the lights off/shut all the windows that are shining through the pane into the ir webcam.
 lamp.jpg (17.98K)
Number of downloads: 53

Here you can clearly see the effect of anti-glare on my (unmodified) laptop screen; the incoming light is diffused away to avoid reflections within the TFT panel itself. As you can see my head blocked some of the light so the IR is not coming from the backlight.
 antiglare.jpg (23.96K)
Number of downloads: 55

I don't think I'll be needing 100 powerleds to make this work on a 150x100cm plate, more like 10 on one side and a strip of aluminiumfoil on the other (the side where I'll be sitting).

This post has been edited by frankvl: 10 January 2007 - 10:00 PM

[frankvl]

Also tested the 6mm acryllic plate with diffuser (from my Eizo backlight); works perfectly too. The diffuser can be polished away if you must. I'd say the only consideration you need to make for the thickness of the screen is how much it bends when you touch it (the 6mm is totally unbendable so perhaps even as little as 2mm would suffice for a large screen). The thickness is a large factor in its price, but with 2mm you don't have to worry about a thing (25 Euros per m2 while 8mm is 140 Euros per m2).

This post has been edited by frankvl: 10 January 2007 - 10:01 PM

[frankvl]

Here is a description how you might create a diy multitouchscreen. It suggests that you drill holes in the sides to mount the leds (and that you should polish the sides, but that seems to be pretty useless in that configuration). However, I tested it and there seems to be a significant decrease in light because of the rough and diffusing drilled holes. That might also explain why he got bad results with a 7.5 mm plate while Jeff Han used a 6.7mm plate and survived.

Here are the test results:

The other end (unpolished) of a 10cm long plate with a led mounted in a drilled hole:
 hole.jpg (32.46K)
Number of downloads: 51

Fair, but this is more like it:
 extern.jpg (21.81K)
Number of downloads: 47

In the 2nd picture the led was placed against the polished edge. This method will most certainly increase the sensitivity and/or decrease the amount of leds needed.

I'll be using this aluminium bar to mount the leds externally; it's cheap (6 Euros/meter), strong, fairly reflective, very light, easy to drill and already equipped with the necessary profile (to block out the sides of the leds). It also has room for the wiring which you can then hide with some tape. Well actually I just chose this one because I can get it for free... I think it'll also be perfect for the (2) legs of the drawing table. Gives it a professional look, too!

This post has been edited by frankvl: 11 January 2007 - 09:11 PM

[frankvl]

Here's the design for the drafting table; 50 Euros for the 4kg framework (using the aluminium profile from my previous post) + 100 Euros for the 6kg 6mm panel. Not bad compared to those conventional heavy drafting tables starting at $700... Very adjustable, sturdy, lightweight and mobile design + lots of room for your legs and/or the projector. I'll also add a paper roll so I can still quickly draft on paper when desired. I don't think I will use a roll of 150cm wide though. Perhaps it's also possible to add a simple plotter or printer.

The vertical bars could be made shorter if you wanted, but I want to be able to stand up and use it. I will also create a continuous cut into the profile instead of the holes I depicted here, so that it can be height-adjusted a lot more easily.

 table.jpg (59.65K)
Number of downloads: 56

I want it painted black!

This post has been edited by frankvl: 11 January 2007 - 10:00 AM

[frankvl]

PS you could also heighten the panel and place it vertically, shove your bed under it, ceil/wallmount the projector and watch movies/do ordinary computer stuff while laying down completely relaxed... Nice!

If you place the projector in a rail on the wall you could probably switch positions in <1 min.

This post has been edited by frankvl: 11 January 2007 - 10:17 AM

[athomasbeck]

Do you plan on sanding down the tips of your LED's in order to get them flush with the surface of your acrylic? That's what I did and it turned out pretty well. The only problem is that once I got it put together there was a lot of light leakage. I don't know if I messed up the angle of incidence or what. I'll try and post a picture when I get home tonight so you can see what I'm talking about. Theoretically all of the light should be contained in the plastic but I don't know if that's practical.

The ftir effect is still much brighter than the light leakage though so it is still workable. I only have 16 LED's though and it might get worse as you increase that number. Just wanted to let you know that is one of the issues you might run into.

Your really starting to get me back into this though and hopefully everything goes well for you.

[PhDeviant]

I saw a video of a multi touch screen not too long ago and was pretty impressed. But the idea that I could actually build one blows my mind. I'm anxious to see what kind of results and functionality you can get from this thing. Good luck on your build.

[frankvl]

Thanks for your replies! It's really nice to hear more people are sharing this excitement
Atthomasbeck, what do you mean with 'turned out pretty well' and 'once I put it together'? Do you mean it worked on a test plate but not with the actual led mount?

The thing is that the more frequent the lightwave has to bounce up and down, the more incidental loss you get, as illustrated below:

 theory.jpg (81.21K)
Number of downloads: 128

The leakage is due to contamination of the surface and the air (i.e. materials that are denser than the acrylic rather than less dense which is needed for total internal reflection), but perhaps also due to a certain percentage defined by natural laws, I don't know.

I tested a normal led and a sanded led of the same type (with the dust cleaned off) and got significantly better results with the normal led. However, I also found out that the led has to be placed at about a 45 degree angle, so that the light is concentrated at 45 degree angles. This means that shallow angles are inefficient too; after all they almost never touch the surface. I didn't polish the other side and put reflective material on it however; when that would be the case I reckon you don't have to put the led at a 45 degree angle, because the shallow angled beams can move back and forth across the plate multiple times and eventually reach the surface just as many times as the 45 degree ones until they die out, donating the same amount of brightness to your picture (assuming perfect reflection and perfect transmission by the acrylic itself).

Conclusion: don't sand or drill! Polish all the edges and cover them with a reflective adhesive with holes cut out for the leds and cover the side of the leds so the spilled light doesn't reach the webcam. Perhaps they still need to be placed at a 45 degree angle in this setup.

This post has been edited by frankvl: 12 January 2007 - 01:11 AM

[frankvl]

Here is a 100 line program to capture webcam data in a Java byte array, to get you started. It just picks the default webcam with the last used settings (incl framerate etc). The array can be edited and the output is directly shown. Most of the code consists of dummy functions, since JMF was designed to do a lot more than what we need.

First install Java 1.5 and JMF, then test the webcam using JMStudio (google how you need to do that), remove the .txt extension from the attachments, compile Camera.java and execute Camera. In MultiSense.java you'll find a method where you can write your implementation. Ask google or me if you need help.

If you want to use my software later on, I recommend you get this to work first.

 Camera.java.txt (1.37K)
Number of downloads: 134
 MultiSense.java.txt (1.05K)
Number of downloads: 120

I'm open to suggestions and feature requests.

This post has been edited by frankvl: 11 January 2007 - 09:03 PM

[athomasbeck]

Sorry, I wasn't very clear with my post there. I didn't do any test plate at all, I kind of just threw everything together quickly. So there was only one try.



Ok, here is what I was talking about. you can see the parts of my hands that arent touching the screen very clearly. My fingertips that are touching are very bright though, so they're easily distinguishible from the unwanted background light. You can also see a few of the LED's on the bottom there. I had them covered, just not very well.

So, you are placing the LED's at 45 degrees to the surface? My LEDs have a beam angle of 40 degrees That puts the maximum angle of incidence within the screen at 40 degrees with them facing straight on. I don't know the exact critical angle were are talking about with this type of setup but based on your results I'd bet its near 45 degrees.

I just find it kind of strange cause if your tilting your LED 45 degrees then your max angle of incidence should excede this due do the angle of the cone of light that your LED puts out. Do you know that the beam angle for your LED's are? The only thing that would make sense is to me is if that angle was very small and by tilting the LEDs you got close to the critical angle.

[frankvl]

I didn't measure the angle exactly, but it was 45 degrees +/- 20 degrees, where 20 degrees would probably be the angle of the led where the light is concentrated. I don't think the exact middle of the led is much stronger than the rest, but I think you're right that it should be less than 45 (around 30-40) because then the entire concentrated beam of the led passes through the acrylic. I didn't know the max angle of incidence was 40 degrees; I just ran some quick and dirty tests. This weekend I will construct the frame, so I can test with the actual setup that I'll be using.

Nice clear results you got there btw!

[frankvl]

The high-power leds came in this morning and give a very nice narrow focused beam, almost no light at all is spilled beyond a 20 degree angle of concentrated light, unlike the ordinary remote control leds I tested with.

[frankvl]

The frame is almost finished now!! Only need to solder a few more leds and fix them at the right angle; I'll post some pictures when it's assembled again. Hopefully the lenses have arrived by then... It turned out to be a bit bigger and heavier than I imagined so I needed to improvise a bit on the design. Changing the height of the board is a bit problematic when you're on your own, so I'm thinking of using motors to lift both sides at the same time.

I mounted 50 leds on each side, which was not strictly necessary but it will greatly reduce the pressure needed to get a response. As expected it also works its way through paper (or any other ordinary diffuser for that matter).

Once I figured out the best way to drill and mount the leds it was just a matter of hours, and they all worked perfectly the first time Next time, however, I hope to use tubular fluorescent IR lamps or something, because they would greatly reduce the effort (and the pain in my back)...

[frankvl]

 front.jpg (46.69K)
Number of downloads: 181

 holes.jpg (33.81K)
Number of downloads: 152

 hand2.jpg (27.39K)
Number of downloads: 52

This post has been edited by frankvl: 18 January 2007 - 08:58 PM

[frankvl]

 behind.jpg (52.4K)
Number of downloads: 108

 hands.jpg (28.8K)
Number of downloads: 55

 hand1.jpg (28.4K)
Number of downloads: 55

The webcam images were made by pointing the webcam (at the position shown in the photos) toward the mirror on the ground. The table is fully captured in the vertical direction and about 90% in the horizontal direction. I tried mounting the cam higher to capture the entire area, but then the response wasn't very fluent and motion almost invisible.

I found that moisturising your hands greatly increases the response (the images shown here were taken of normal dry hands). The condens of your hand is captured too, however, leaving a ghost image for a second, but under normal conditions hardly any condens is formed.

The plate gets dirty pretty quickly (fat + dust); this is not a problem for the image, but it gives a slightly uncomfortable feeling when you slide your fingers over the table.

The table can be used as an ordinary desk for not too heavy objects; my laptop is on the table as we speak and the plate only bends about 1-2mm when I'm typing.

Also received an email from Exclusiv today announcing that they are about to post the package to UPS, so it will probably take another week before I can start with the actual projector

This post has been edited by frankvl: 18 January 2007 - 09:49 PM

[EdZ]

To deal with the nasty surface texture: Do coatings affect the frustration effect? Try stretching some cling-film (saran war in the US?) over the top. If it doesn't adversely effect it, try getting some sort of adhesive coating hat is more comfortable to use but still optically clear (I think the guys who sell the invisibleshields for ipods and the like sell custom sizes of the stuff, if it's not available directly from 3m. It's durable and doesn't feel particularly uncomfortable).

This post has been edited by EdZ: 19 January 2007 - 12:01 AM

[frankvl]

Thanks, I think I'll try that someday; I still have to determine which diffuser to buy anyway. It should at least be possible to make the surface wet because then you can use virtually anything as your input medium (such as brushes for a true painting experience). Perhaps I can use adhesive when I can coat the surface with some kind of IR reflective material, or perhaps that could even be applied to the adhesive; then the internal reflection should remain intact.

[relieve3d]

I will also buy 3D gamer glasses (+/- 40 Euros) for extra fun
[/quote]

Sorry 3D glasses will not work with your LCD proyector neither 5ms (I have tested with eDimensional and Xforce3D glasses and domgles and diferents combinations) That is becase LCD´s don´t refresh whole imagine at once. The pixels get and pass imagine info one to others so there are a constant refresh fluid.

Nowadays the only way to go 3D is an DLP proyector with active glasses (but many fliker, only 85hz 42,5 per eye(great headaches)) or two proyectors at once one for each eyes and using pasive polarizer glasses to get independens one imagine from other (linear or circular polarizer). If you want to move freely use circular polarizer filters if not use linears like IMAX3D, realD, Disney Digital Studios and the most 3D cinemas. In order to don´t waste light from proyectors if you choose use linear don´t add polarizers filters at all use the same which are need for the LCD, one directly and the other inverse (negative colours, correct this at drivers level and see real3d with passive glasses and full colour).

All about 3D is explained more extensive and with 3D movies links at:
http://www.lumenlab....s...15333&st=40
http://www.lumenlab....s...15333&st=60 3D glasses

Exist faster DLP´s proyectors but very expensive: lightspeed, barco.....

Great proyect yours

Attached File(s)

•  LCD_16bmp.jpg (43.1K)
  Number of downloads: 5

This post has been edited by relieve3d: 22 January 2007 - 03:03 PM

[frankvl]

Tnx, I had no idea! However, after some googling I found 3D glasses suitable for >70 Hz LCD (wired $70 /wireless $100 @ eDimensional.com). They wouldn't work on my 25ms Eizo, but do you think they would work on >70Hz Lumenlab projectors?