Computer-based tools can support the development of knowledge-based systems. Such tools, however, are often complex, and achieving the optimal use of these tools can be difficult for the developer. Often, developers repeat well-known mistakes, which can be difficult to identify and to correct later in the development process.
PROTÉGÉ-II (Puerta et al., 1992) is a knowledge-engineering environment that supports the developer in the reuse of problem-solving methods and the design of domain-specific knowledge-acquisition tools. PROTÉGÉ-II provides a set of tools that allows the developer to combine problem solvers from reusable components, edit ontologies (which define concepts and relationships among concepts), and generate automatically knowledge-acquisition tools based on the ontologies. Domain specialists use the knowledge-acquisition tools to enter domain knowledge, and to produce knowledge bases (which consists of instances of concepts defined in the ontologies).
Although we have used successfully the PROTÉGÉ-II suite of tools for system development (Rothenfluh et al., in press; Tu et al., 1995), we have encountered difficulties with some aspects of the PROTÉGÉ-II approach. One of the most common problems is incorrect use of the tools. New users of PROTÉGÉ-II often make mistakes, such as inappropriate modeling decisions, that proficient users would avoid. Proficient users, however, make other types of mistakes, which could have been avoided with hindsight. Sometimes the same user repeats the same mistake several times, because the problems are documented poorly and occur infrequently. The user might be aware fully of potential problems, and yet make them anyway.
These problems are not unique to PROTÉGÉ-II. Users of most complex development languages and tools share the same frustration. Proficient users learn over time how to select appropriate design strategies, and to avoid common pitfalls. A significant proportion of the task of mastering a complex language or tool is to learn how to avoid and identify problems. Problems of this type are often costly, because it is time consuming to identify them. It is not uncommon that a large proportion of the development time is spent on relatively few bugs. By avoiding these expensive bugs as far as possible, the development cost can be reduced significantly.
Undoubtedly, many of these problems are due to inadequate language and tool design. Well-designed development environments with intuitive user interfaces are essential for good results. However, developers make mistakes even when appropriate languages and well-designed tools are available. Developers usually have incomplete knowledge of all the technical details. Moreover, developers make logical design errors.
Expert critiquing systems are programs that check problem solutions produced by humans and procedures followed by humans, and report flaws and potential problems (Silverman, 1992). Examples of such systems range from grammar checkers to knowledge-based systems for design verification. Two of the major advantages of expert critiquing systems is that (1) these systems use the knowledge of a domain specialist to check the solutions (often produced by novices), and (2) these systems perform the checking consistently.
Expert critiquing systems can be used to critique the use of knowledge-engineering tools. In our approach, the Critique Tool (CT) examines intermediate structures produced by PROTÉGÉ-II, and presents a list of critiquing points to the PROTÉGÉ-II user. Critiquing systems are useful in many phases in the PROTÉGÉ-II design cycle. We shall examine the potential use of critiquing systems in the PROTÉGÉ-II design approach, and discuss a prototype implementation of CT.