Typical application fields of Knowledge Based Systems are usually characterized by having human expertise as the only one source to specify their desired behavior. Therefore, their design, evaluation and refinement has to make effective use of this valuable source. After an introduction to the concept of collecting validation experience in a Validation Knowledge Base (VKB), the paper introduces an estimation of the significance of the cases collected in the VKB. A high significance signalizes that a VKB should not longer serve as a case-based source of external (outside the Knowledge Base) knowledge, but compiled into the Knowledge Base instead. Based on this significance estimation, a technology to compile well selected cases into the Knowledge Base of the system under evaluation is presented.