In this section we discuss configuration methods from the literature: generate-&-test, propose-critique-modify (PCM) and a specific PCM method propose-&-revise. We evaluate all these possible methods for configuring problem solvers. Although our study is not exhaustive, in this section we argue that the PCM-paradigm is an appropriate paradigm for configuring PSMs.
Generate-&-test family This family of methods generates a configuration in the first step, and subsequently tesets this configuration. There is a wide range of generation and test steps, from a simple generation step with a knowledge-intensive test step to a knowledge-intensive generation step with a simple test step. Knowledge that can be used concerns the set of possible components, the possible connections between components, the constraints and the user-requirements. Characteristic of a generate-&-test method is that when a configuration does not pass the test, the configuration process continues with a completely new configuration, without taking into account the reason why the previous configuration failed the test. In our case the generate-&test method is not appropriate because our test for the dynamic goals is expensive, since it requires performing diagnosis.
Propose-critique-modify family Characteristic of a propose-critique-modify (PCM) method is that when a configuration is not a suitable configuration, the configuration process does not continue with a complete new configuration, but uses the test results for determining a new configuration instead of generating a new one from scratch. The propose-critique-modify (PCM) family [Chandrasekaran, 1990,Brown & Chandrasekaran, 1989] consists of four steps: propose, verify, critique and modify. We discuss each step in turn.
Propose: The propose step gives a partial or a complete configuration. Methods for the propose step are: solution decomposition, design proposal by case retrieval, and constraint satisfaction [Chandrasekaran, 1990]. For our specific case of configuring diagnostic methods, another method (close to the one used in the VT-task [Schreiber & Birmingham, 1996]) seems more appropriate. In this propose-method, parts of the design (in our case some of the parameters in the diagnostic schema) are proposed on the basis of requirements. These partial proposals are then completed into full proposals by proposing values for the remaining parameters. (As will be explained later on, in our case this completion process is unguided in our current proposal).
Verify: The verify step involves checking that the proposed configuration satisfies the constraints and the user-requirements. [Chandrasekaran, 1990] distinguishes two verification steps. (1) ``attributes of interest'' that can be directly calculated or estimated by means of domain specific formulae. In our case (configuring diagnostic problem solvers) these are the constraints on the diagnostic components and on the assumptions. (2) ``behaviour interest'' that can be derived by simulation. In our case the simulation amounts to performing diagnosis. Based on these results the dynamic goals have to be verified. We use the term simulation-verification of [Chandrasekaran, 1990], but validation should be a more appropriate name, because we validate the method by execution.
Critique: The critique step is a diagnostic problem of mapping from
undesired behaviour to the parts of the configuration which are possibly
responsible for this undesired behaviour
.
This step analyses the failure of the
configuration. Therefore it needs information about how the structure of the
device contributes to the desired behaviour. In our case this is knowledge of
how properties of the components of the diagnostic schema relate to
properties of the complete schema. In this phase one can use (meta-)diagnostic
knowledge about goal violations and repairs.
Modify: The modify step uses the repair information from the critique step and executes the repair action. It changes the configuration to get closer to the specifications. In our case this is the actual adaptation of the diagnostic method.
Propose-&-revise family The propose-&-revise family is a sub-family of PCM methods. These methods are used in the VT-domain [Schreiber & Birmingham, 1996]. This family of methods is a simplification of the PCM method, because the critique step is replaced by compiled knowledge. The idea behind this family of methods is that it is possible to give an initial proposal for a configuration. This configuration is constructed by selecting values for the set of components based on the user-requirements. This configuration can be ``fixed'' (repaired) if constraints are violated. These fixes are the compiled critique knowledge. Fixes are direct associations of a constraint violation and a repair action by changing one or more parameter values [Runkel et al., 1995,Marcus et al., 1988,Fensel, 1995]. Propose-&-revise methods require these fixes as search control knowledge.
A propose-&-revise method is not appropriate for automated configuring of PSMs for two reasons. First in our problem we need a full critique step. The critique step is quite complex and it is not possible to code it in simple direct associations between a constraint violation and a repair action. Secondly, propose-&-revise methods are used because of the large search spaces, but our most important motivation is to prevent the expensive tests of dynamic goals (performing diagnoses). Our efficiency problem is not in the constraints but in verifying the dynamic goals.
Conclusion From this very brief discussion of the configuration of PSMs seen as a configuration task, we conclude that the family of propose-critique-modify is the most suitable one for a method for automated configuration of PSMs. The specific propose-&-revise method is not appropriate for our application, because (1) we need an explicit critique step and (2) the efficiency problem differs from constraints in propose-&-revise versus dynamic goals in our case. Furthermore, we made the PCM method more specific by allowing the possibility to adapt beside the diagnostic methods, also the assumptions, the diagnostic problem and the goals.