Either you know or you don’t

If evaluation was limited to True or False test then knowledge will only be limited to two stages: does know or does not know, complete knowledge or total ignorance.

There is a feeling that it should not be that simple. For instance what about those “tip of the tongue (TOT)” experience: knowledge or ignorance? On the other side and as Bertrant Russel stated in 1912: “Is there any knowledge in the world which is so certain that no reasonable man could doubt it? ” (Russell, B. (1988) The Problems of Philosophy, Prometheus Books. Page 1).

Does complete knowledge exists? When starting to learn something do human start from total ignorance?

Darwin Hunt (Human Self-Assessment : Theory and Application to Learning and Testing in Leclercq, D. & Bruno, J. (ed.) Item Banking : Interactive Testing and Self-Assessment, Berlin: Springer Verlag, 1993, F 112, 177-189) suggested to distinguish between three types of knowledge situations in which a person can be in relation to a piece of content:

1. misinformed,
2. uninformed,
3. informed.

Jans and Leclercq (Mesurer l’effet de l’apprentissage à l’aide de l’analyse spectrale des performances in Depover, C. & Noël, B. (ed.) L’évaluation des compétences et des processus cognitifs. Modèles pratiques et contextes, Bruxelles : De Boeck Université, 1999, 303-317) related these three levels with confidence degrees (or degree of certainty / certitude). Every answer is associated with a percentage of confidence in the answer. An answer wih the lowest confidence will be given with a certitude of 0%. The highest confidence will be 100%. Answer still can be right or wrong. But 4 stages can be made:

1. Wrong answer with high certitude or dangerous knowledge;
2. Wrong answer with a low certitude or unawareness;
3. Right answer with a low certitude or mid knowledge;
4. Right answer with a high certitude or usable knowledge.

In their original paper Jans and Leclercq used a logarythmic scale for students to chose their certitude from. They defined 7 stages defined by different value of the given certitude of the answer

• right answer with certitude from 85 to 100%: perfect knowledge
• right answer with certitude from 15% to 85%: partial knowledge
• right answer with certitude from 0% to 15%: doubtful knowledge
• no answer: ignorance
• wrong answer with certitude from 0% to 15%: admitted unawareness;
• wrong answer with certitude from 15% to 85%: confusion;
• wrong answer with certitude from 85 to 100%: ignored unawareness.

It should be up to the education team and based upon educational objectives to fix the limite between a high level of confidence and a low level of confidence. A medical student in surgery may need right answers with certitude of 100% to be considered as usable knowledge, all other right answer falling into the mid knowledge class. For a person taking a test for a driving licence the limit for usable knowledge might be over 50%, all right answer below 50% falling into the mid knowledge class. Of course if using True / False questions a certitude below 50% has no meaning because the odds finding randomly the right answer already are 50%.