September, 2012. Personal Project

This is a completely original research attempt to develop an innovative and easy (and cheap) method of human-computer interaction. Aiming to replace the conventional use of mouse in situations like presentations or public demonstrations, this project features a setup of a simple webcam, laser pointer and a wireless system for trigger control/reception.

The setup requires a webcam of any standard to be placed facing the screen/projection screen. A remote trigger coupled with a laser pointer is used as a “mouse”. The user pushes the trigger when the laser points at the required target. This causes the trigger receiver at the computer to simulate pressing of a predefined key combination.

The TouchEx client program on the computer picks up the key combination and requests a webcam capture. It synchronises the computer screen to flash a complete white screen at the exact time the webcam captures the image of the screen. This synchronization requires taking into account various factors like Webcam latency, Computer operation latency and delay between consecutive shots. The image is saved to disk and a Matlab program processes the image. It finds out the co-ordinates of the laser pointer on the screen after performing various operations like target window clipping, viewing angle neutralization and monochrome conversation. Performing co-ordinate neutralization with respect to the predefined resolution of the screen, the final co-ordinates are returned to the client. The client moves the mouse to the intended target and performs a click using WINAPI.

Highlights:

• Highly cost efficient. Project budget ~ Rs 800 ($15)
• Wireless and uncompromised emulation of Mouse
• Usage of Image processing instead of live video processing, thereby drastically reducing system resource usage.
• Placement of webcam independent of viewing angle.
• Negligible latency between click requests and actual clicks
• Seamless integration of TouchEx client program with general computer usage

Critical techniques used:

• Separate programs for Key Combination pick up and Click simulation/Screen Flasher to reduce overall operation latency.
• Usage of Low level keyboard to pick up Key Combinations
• Abstracted User definable synchronisation times for intermediate operations
• Usage of Matlab script as a function within the client thereby gaining time lost to program start up and operation.

Successful demonstrations were attested by ECE staff and some its performance documented. My interest in the project later waned when we learnt that my idea and implementation wasn’t the first.

Here’s a video demonstration of how the project. Please note that this is a poorly captured video. This was meant just for personal observation and project correction. My Interest in the project waned when I learnt that this wasn’t the first of its kind. The gray screen you see is the moment the computer screen is captured by the webcam. This method was later refined to eliminate the gray screen altogether, thus reducing the overall time operation time. You can see the click gets registered quite late here.