IJCAI-99 Workshop on
Empirical AI31 July 1999, Stockholm
Since the beginning of AI, experiments have been used to demonstrate the value and potential of new techniques. While in earlier years, the demonstration of a new technique in a toy-domain was sufficient, the standards for acceptable experiments have significantly changed. In later years, more emphasis was put on "real-world problems" as well as on more stringent evaluations of experi ments. At least in part stimulated by Paul Cohen's book "Empirical Methods for AI", recent years have seen a massive increase in the use of statistical techniques to assess the validity of empirical results. However, what issues can and should be addressed by such quantitative approaches is not clear.
This one day workshop aims at clarifying the role of empirical methods in AI. Attention will be given to the traditional factors found in other empirical sciences (such as the problem of representativity of case studies, the proper use of statistics, etc.) as well as to the more traditional objectives in AI research: instead of quantifying the relative performance of certain techniques, we are often interested in the development of new concepts allowing a new and hopefully more adequate understanding of certain issues, such as our own cognitive processes, the nature of the tasks we want to solve, etc. What kind of empirical studies can help us in the development of new concepts and frameworks? To stimulate the discussion in the workshop, it will begin with an invited talk by Paul Cohen, author of "Empirical Methods for AI".
This workshop continues a sequence of workshops on Empirical AI held at previous the IJCAI-97, ECAI-96 and ECAI-98 conferences. These workshops were preceded by more specialised workshops at AAAI-94 on Experimental Evaluation of Reasoning and Search Methods and the CADE-12 Workshop on Evaluation of Automated Theorem Provers.
While previous workshops had an emphasis on performance comparisons of heuristics for combinatorial problems, the IJCAI-99 workshop promises to reflect a broader spectrum of research areas in AI.
On behalf of the organising committee I wish to thank Paul Cohen for agreeing to present the invited lecture. A big thank you goes also to all authors for their efforts and to all other participants for contributing to the success to the workshop. Finally, I wish to thank the members of the organising committee for the smooth cooperation during the preparation of this workshop.
(Chair of the Organising Committee)
|Anbulagan||Meticulous Empirical Study on Phase Transition|
|Bhattacharyya and Laird||Empirical Methods for Plan Execution Architectures|
|Endres-Niggemeyer||Grounded Theory Methodology for Knowledge Engineering|
|Bonet and Geffner||Planning as Heuristic Search: New Results|
|Gini and Benfenati||What experiments are suitable to access the prediction of toxic effects of chemicals|
|Hoos and Stuetzle||On the Empirical Evaluation of Las Vegas Algorithms - Position Paper|
|Kaindl and Kainz||Guidelines for the Experimental Comparison of Search Algorithms|
|Korb||Calibration and the Evaluation of Predictive Learners|
|Menzies||The TIM-BO model and the Fairmont Effect|
|Mitchell||A Remark on Benchmarks and Analysis|
|Thakar||Industrial experiences in empirical evaluation of domain modelling approaches|