Contents

• List of Figures
• List of Tables
• 1 Introduction, goals and objectives
  • 1.1 Introduction
  • 1.2 Goals
  • 1.3 Objectives
  • 1.4 Potential Applications
    • 1.4.1 Inverse mapping from Auslan to English for dictionary use
    • 1.4.2 Generalised Gesture Recognition
    • 1.4.3 Another interface tool available to the Deaf community
    • 1.4.4 Communication between Deaf and speaking people
  • 1.5 Overview of the report
• 2 Previous work
  • 2.1 Introduction
  • 2.2 The field of study
  • 2.3 Physiology of the hand
  • 2.4 What is Auslan?
    • 2.4.1 Variations on Auslan
      • Finger-spelling
      • Signed English
      • Pidgin Signed English
      • Cued Speech
    • 2.4.2 Structure of signs in Auslan
      • Handshape
      • Location
      • Orientation
      • Movement
      • Expression
  • 2.5 The Device vs Vision debate
    • Device-based measurement techniques:
    • Vision-based Approaches:
  • 2.6 ``Whole Hand Input'' Devices
  • 2.7 Learning tools
    • 2.7.1 Neural networks
    • 2.7.2 Symbolic learning algorithms
    • 2.7.3 Instance-based Learning
    • 2.7.4 Grammar-based techniques
    • 2.7.5 Hidden Markov Models
  • 2.8 Previous work in sign language recognition
    • 2.8.1 Image-based approaches to sign language
      • Charayaphan and Marble's research in to Image Processing for ASL
      • Davis and Shah's work on Gesture Recognition using cameras
      • Starner's work with American Sign Language and Hidden Markov Models
      • Dorner and Hagen's work with a complete system
    • 2.8.2 Device-based approaches to sign language
      • Glove-Talk and Glove-TalkII -- Fels and Hinton's contribution
      • Pausch and Davidson's CANDY system
      • Wexelblat's work on the AHIGS system
      • Takahashi and Kishino's investigation
      • Recognition using recurrent neural nets
      • Kramer's Talking Glove Project
      • A Linguistic Approach to Recognising Gestures
      • Peter Vamplew's work with SLARTI
    • 2.8.3 Summary of previous research
• 3 Resources, Equipment and Exploration Areas
  • 3.1 Introduction
  • 3.2 Choices
    • 3.2.1 Choice of data collection technique
    • 3.2.2 Choice of computer
    • 3.2.3 Software choice
      • Originality of the Approach
      • Discrete outputs
      • Scalability
      • ``Tweakability''
      • Speed
  • 3.3 Original aspects of the investigation
    • Study of Auslan
    • The use of Symbolic Learning Algorithms and Instance-based learning for Gesture Recognition
    • Use of cheap equipment
    • Relatively large lexicon
• 4 Framework Development
  • 4.1 Introduction
  • 4.2 The Glove software family
    • 4.2.1 glovedata
    • 4.2.2 gloveread
    • 4.2.3 gloverecord
    • 4.2.4 gloveplay
  • 4.3 Selection of classes and data acquisition methodology
    • 4.3.1 Data acquisition protocol
• 5 Feature Extraction and Selection
  • 5.1 Introduction
  • 5.2 Getting a clean signal
  • 5.3 Feature extraction and validation
    • 5.3.1 Validation methodology
    • 5.3.2 Distance, energy and time
      • Principle
      • Results
      • Discussion
    • 5.3.3 Bounding boxes
      • Principle
      • Results
      • Discussion
    • 5.3.4 Positional histograms
      • Principle
      • Results
      • Discussion
    • 5.3.5 Rotation and finger position histograms
      • Principle
      • Results
      • Discussion
    • 5.3.6 Synthesised histograms
      • Principle
      • Results
      • Discussion
    • 5.3.7 Octant count
      • Principle
      • Results
      • Conclusion
    • 5.3.8 Simple time division
      • Principle
      • Results
      • Discussion
    • 5.3.9 Using synthesised features with time-division
      • Principle
      • Results
      • Discussion
    • 5.3.10 Other approaches considered but untested
• 6 Synthesis
  • 6.1 Introduction
  • 6.2 Putting everything together
    • 6.2.1 All features
    • 6.2.2 The most effective features
  • 6.3 Effects on accuracy
    • 6.3.1 Number of samples per sign
    • 6.3.2 Number of signs
    • 6.3.3 Inter-signer recognition
  • 6.4 Time issues
  • 6.5 Optimisations
• 7 Summation
  • 7.1 Conclusion
  • 7.2 Suggestions and reasons for future research
    • 7.2.1 Short-term work
    • 7.2.2 Long-term work
• A Other tests performed
  • A.1 Introduction
  • A.2 Simple, straight attributes fed directly to the learning algorithm
  • A.3 Creating additional attributes
    • A.3.1 Discussion
  • A.4 Other investigations and notes
    • A.4.1 Neural network behaviour
    • A.4.2 How does a PowerGlove compare?
  • A.5 Conclusions
• B Words selected for the project
  • B.1 Notes on the list
• C Glove package source
  • C.1 glovedata.h and glovedata.c
    • C.1.1 glovedata.h
    • C.1.2 glovedata.c
  • C.2 gloveread.c
  • C.3 gloveplay.c
  • C.4 gloverecord.c
• D A Note on the bibliography
• References
• About this document ...