Conclusion

This thesis implements the first general, accurate, comprehensible, robust, weak temporal classifier. While other systems have implemented some of these characteristics, this is the first to capture all of them.

It is a general algorithm that has been applied to two real-world and two artificial domains, one of which was specifically designed to model the characteristics of real-world domains. It has been tested on situations with up to 22 channels, 110 metafeatures, over 200 megabytes of data, 95 classes and highly skewed class distributions (8:1 ratio between the most common and least common classes in the ECG domain).

It is accurate: in all of the domains tested it was able to equal or better the two baseline algorithms, and in the case of ECG classification, performs better than a single human expert: Our system obtains 71.5 per cent accuracy versus a human expert with 70.3 per cent accuracy, despite the fact that TClass learnt with only 450 examples, including one class with only 21 examples. In the Auslan domain, it obtains an accuracy of approximately 98 per cent. Considering that there are 95 signs in the Flock Auslan domain, and the default accuracy would be approximately 1 per cent, this is no easy feat. Furthermore, it matched the performance of hand-crafted feature sets in both the ECG and Nintendo Auslan domains.

It is comprehensible: metafeatures allow the expression of learnt concepts in the background knowledge presented to the learner. Experiments show that in artificial domains, the generating concept is recovered. In the real-world domains, the produced concepts correspond to known definitions.

It is robust: in artificial domains, when the noise level was increased, classification was still possible. With the notoriously poor quality Nintendo sign language data, it was able to achieve accuracy equivalent to hand selected features.

However, it is not without its flaws. TClass has the following weaknesses presently:

• While it can be used to produce either accurate or comprehensible concepts, it has not proved possible to do both simultaneously.
• While it is a linear time algorithm, in cases with huge data it is not particularly fast. Run times for all of the domains were reasonable, under two hours in most cases on a PII-450MHz PC. However, for the ECG domain, when all data were being learnt from, this extended to approximately 8 hours.
• It is still only a weak learner: it does not yet classify sequences of classes. In this regard, it particularly falls behind hidden Markov models, although it has an edge over them in terms of comprehensibility.
• While it produces readable concept descriptions in low-noise situations, it does not do so well in high-noise situations. Noise leads to an explosion in the size of decision trees and rules.

The problems above are not insoluble; merely beyond the scope of the current research - some of the problems have quite obvious solutions, which were discussed in the Chapter 7. Hopefully, TClass will serve as a useful platform for exploring temporal classification issues.