Charayaphan and Marble's research in to Image Processing for ASL

Charaphayan and Marble, in 1991-92 [CM92] examined the possibility of processing images in such a way as to understand ASL (American Sign Language). To do so, they used three criteria of movement:

• Where the hand stopped. They did this by measuring the standard deviation of the motion of the hands compared to previous positions and thus determined if the hand had come to a complete standstill.

• What ``trajectory'' -- that is the course in space -- the object followed. They did this by measuring the eccentricity between one ``stop'' and another. The eccentricity can be thought of as ``how bent'' the approach is -- it can be defined as the ratio between the straight line connecting two stop points and the the farthest distance of the path from that straight line. See figure 2.13.

    
  Figure 2.13: The eccentricity in the diagram is defined to be MaxDev/StraightPath. The value is positive if it is on our right as we follow the trajectory, and negative if it is on our left.
  

• What shape the hand was in when it stopped. When all else failed, they would look at the final handshape to determine what the sign was, using a Hough transform to compare the handshape to ones stored in a table.

They used no computer-based learning methods, but rather hand-built a system that has close similarities to an instance-based learning technique -- that is they averaged the locations of their training set and used this as the information. To test the system, they captured each sign once and simulated what they believed was typical variation in the input. This system successfully classified 27 of the 31 ASL symbols by the second stage. Of the remaining four signs, it was shown that using the Hough transform, the correct sign could be decided upon most of the time.