⚠ We would appreciate if you would disable your ad blocker when visiting our site! ⚠

Innovative Hospitality Technology - Projected Image Applications

Order a reprint of this story
Close (X)

To reprint an article or any part of an article from Hospitality Upgrade please email geneva@hospitalityupgrade.com. Fee is $250 per reprint. One-time reprint. Fee may be waived under certain circumstances.


June 01, 2005
Next Generation | Technology
Michael Kasavana, Ph.D., CHTP - kasavana@pilot.msu.edu

View Magazine Version of This Article

© 2005 Hospitality Upgrade. No reproduction without written permission.

Hospitality industry leaders have long believed the ultimate, most flexible technology configuration for efficient operations had to be wireless. After all, a wireless fidelity (Wi-Fi) network provides a platform that enables rearranging furnishings, relocating terminals and rerouting workstation devices whenever desired. With Wi-Fi there is little concern for cable connectivity or most other physical constraints. What could be more convenient than portable hardware? How about eliminating hardware devices altogether.

Although still in its infancy, electronic perception technology (EPT) allows for the projection of an interactive, virtual display that mirrors the functionality of a physical component. For example, EPT can enable the projection of a typewriter keyboard onto a tabletop surface and allow the image to capture the user’s finger movements as data entry. With the magic of EPT, projecting a virtual object with sensory interpretation capabilities becomes reality. Basically, EPT combines directional technology with optical recognition to broadcast a three-dimensional representation of an object with the capability to recognize nearby or direct user activity. In the keyboard example, the device is described as observing the user's finger tapping and converting them into recorded keystrokes. Unlike physical equipment, EPT images are composed of projected light and when not in use, completely disappear. How does EPT work? What are some possible applications for the hospitality industry? Is this a near-term or long-term development?

Innovative Technology
Starting with the advent of television and auto-focus cameras, the field of machine vision is considered one of the oldest and most widely researched technological fields of the post-industrial era. Given that camera technology interprets three-dimensional landscapes in two-dimensional flat images, EPT machine vision research advances the principle to the point of separating objects in a scene from one another as well as from an extraneous background.

EPT is referred to as breakthrough technology since it is capable of providing a low cost means of translating three-dimensional features into three-dimensional object representations in real time. Such characteristics enable interactivity between a user and a virtual device that resembles transactions accomplished through physical hardware.

The main compliant with portable technologies like PDAs, cellular phones and smart phones is that they are too tiny to incorporate a full keyboard, which complicates text messaging and data entry. For this reason, cumbersome alternatives like stylus recognition, handwriting recognition, voice recognition and thumb-board recognition have become the norm. The problem with these approaches is that they alter the initial appeal of lightweight, convenient and portable devices.

Electronic projection technology, originally developed to address such concerns, provides a more user-friendly option for simplifying data capture and subsequent processing. Since EPT is capable of providing a light-projected object (e.g. virtual keyboard) that recognizes user activity it has become a product of interest. As system developers experiment with image projection as an add-on for handheld devices, several other possibilities for unique applications, on a larger scale, are approaching reality.

Electronic devices with machine vision are starting to attract the attention of creative system designers and may soon impact hospitality technology.

EPT Components
One current EPT application involves placing a PDA on a tabletop with an attached miniature projector. The light from the projector shines a keyboard onto the flat surface allowing the user to type as if the image were a real keyboard. EPT will recognize finger placements and movements translating them into keystrokes. The keyboard is made possible by a tiny sensor fitted with three chips - one to beam the keyboard image and two to pick up the movement of typing fingers. That typing movement interacts with the light and sends signals to the sensor. In addition to providing a full-size keyboard, EPT is also capable of allowing the user to perform mouse functions.

Founded in 1999, Canesta, Inc. (www.canesta.com) holds several exclusive patents related to EPT technology. In general, the patents revolve around a system that enables automated equipment to perceive and react to nearby objects. While Canesta, Inc. is not the sole supplier of EPT-based products (see sidebar listing), supplier applications mirror this sequencing:


In essence, electronic perception technology enables a machine to perceive and react to nearby objects in real time through the medium of sight. The possibility of automated applications, including hospitality industry uses, appears interesting and wide-ranging.

EPT Requirements
EPT operates through the integrated operation of low-cost, high-performance embedded sensors and specialty software. EPT relies on a chip that includes finely tuned timing circuits for measuring reflected light from individual pixels and calculating the distance to the object from the light source. The reflected waves are then used to reconstruct the image of the object, complete with a depth map extending the two-dimensional image into the third dimension. EPT does not rely on visible light, but instead uses a beam of infrared light, similar to the light emitted in an auto-focus mechanism of a camera, to create the object.

Projected image recognition is made possible by a tiny sensor equipped with three chips: one to beam the image and two to detect user movement. User action of the interface causes the projected light to transmit signals to the sensor. The sensor receives the light and reconstructs the image using built-in software. The software is embedded into the imaging chip thereby eliminating the need for a separate micro-controller.

There are four system components required for EPT application: a pattern projector, a light source, a sensor device and imaging software. Each is designed to be small enough for efficient integration with a variety of electronic devices. Taken together the four components approximate the size of a disposable lighter.

Pattern projector – a device used to project light against a flat surface in the shape of a desired design such as a keyboard or custom designed menu board. Since virtual items are images projected in light, unlike physical hardware components, EPT objects completely disappear when the light is extinguished.

Light source - an infrared light is used to provide brilliant and sufficient pixel definition enabling projection of a virtual object.

Sensor device – a critical component used to distinguish and capture user interactivity from a projected object. A low-cost semiconductor-based unit capable of recognizing user interactions up to 30 centimeters (11.8 inches) from a projection source, in a field of view about 1.5 feet square. EPT developments are expected to improve resolution recognition and significantly increase the distance from the light source. The controller chip communicates with a host device via either an RS-232, USB or Bluetooth connection.

Imaging software – special instructions designed to create a three-dimensional representation of a projected object using a relatively simplistic mathematical algorithm. Similar to radar, where the range to a remote object is calculated through various methods, chips are used to develop distance maps to points in an image of a nearby object. This information is then sent to an on-chip processor for imaging software application. Chips are used to constantly repeat this process thereby producing in excess of 50 frames of three-dimensional information per second.

EPT Operations
An EPT sensor works in a manner similar, but not identical, to radar technology. By definition, radar is a means of detecting distant objects and determining their position by analysis of high frequency radio waves reflected from distant surfaces. The distance to the remote object is then calculated by measuring the time taken for an electronic burst of radio waves to travel round trip from the transmitting antenna to a reflective object. In the case of EPT, unobtrusive light replaces radio waves.

Since light waves travel at a constant speed, a finite time can be pre-determined for travel between two distant points. Knowing the elapsed time of travel, the distance between the two points can be calculated. With EPT the light illuminating each individual pixel in an image sensor comes from a different feature in the scene and therefore can be used to measure the distance to that feature. By doing so, a three-dimensional map of a projected object’s surface can be formulated. EPT provides a mathematically accurate relief map of the surfaces of objects being imaged.

EPT technology, relying on proprietary software, is able to distinguish foreground from background objects in a landscape. EPT software features compact coding and can be embedded into small, application-specific devices.

EPT Classifications
Applications for EPT sight-enabled devices are classified into three broad categories: human-computer interfaces, navigation applications and inventorying applications.

Human-computer interfaces -- includes such user interfaces as projection keyboards, virtual video games, e-notepads and e-device controllers. The term human-computer interfaces describes a system’s ability to track pen, pencil, stylus and other pen-like object interfacing.

Navigation applications -- includes robotics, gestural interfaces for wearable computers, smart bionics (i.e. situationally-aware operations that alter actions based on proximity of nearby objects) and electronic sight assistance for the visually impaired. For example, an automobile crash alert system may include an audible warning mechanism to notify a driver whenever another vehicle or object is perceived as too close for safe driving.

Inventorying applications -- includes electronic intruder security detection, electronic babysitting devices and product preference profiling.

In essence, a sight-enabled device with the capability of reacting to and interacting with the surrounding environment. EPT devices can also be programmed for user identification and authentication through biometric (facial) recognition.

Hospitality Potential
To date, the concept of EPT has not been proposed for hotel, restaurant, club or casino implementation. Potential hospitality applications might include replacement of desktop terminals, kiosks, handheld devices and wearable PCs with projection registration processing, menu board offerings or in-room entertainment options. The use of a projected image to guide self-checkin, order entry and other limited choice options becomes more appealing as the trend toward self-service technology continues to evolve. Considerations in the areas of housekeeping, food and beverage service and casino operations may soon appear on an EPT platform.

Consider EPT menu broadcasting. In this example, EPT can be used to project the restaurant’s menu onto a flat, accessible surface and thereby enable guests to choose and self-order items without using a physical, printed menu. By offering limited menu choices on the display, EPT can enhance guest satisfaction, provide a competitive advantage, reduce labor costs and speed service. An operation that features EPT may be able to promote food and beverage sales to targeted customers at unique, non-traditional service locations (e.g. standing at the bar or waiting in the lobby). In addition, the same projection approach can be applicable to airplane foodservice through pull-down tray projection.

Beaming the list of beverages and snack items just after takeoff enables data capture in advance of service, which hopefully translates into better service. EPT technology, though in its early stages of development, possesses tremendous potential for a variety of hospitality industry applications, especially in lodging and foodservice industry guest service applications.

Electronic perception technology (EPT) is a unique and innovative technology that enables automated equipment to gain artificial sight also called machine-vision. EPT employs specialty software and embedded sensors to create a visual input field that can be used as a front-end application for transactional behavior. A projected object, based on a controller and optical components, can be used to track user movements.

What differentiates EPT from other machine-vision applications is that it is the first to recognize actionable information in real time. EPT relies on relatively inexpensive projection and sensing components and provides electronic devices with perceptual capabilities. EPT is expected to revolutionize interface technologies in general and hospitality industry specifically. EPT presents a near-term application solution that can help reduce the industry’s dependence on staffing while enhancing accuracy in processing. Potential applications within hotel, restaurant, club and casino environments are limitless.

Michael Kasavana, Ph.D., CHTP, is NAMA Professor in Hospitality Business, School of Hospitality Business, Michigan State University. He can be reached at kasavana@pilot.msu.edu.

want to read more articles like this?

want to read more articles like this?

Sign up to receive our twice-a-month Watercooler and Siegel Sez Newsletters and never miss another article or news story.