Definition of metacognition

A definition has appeared on the Merrian Webster’s Online Dictionary for metacognition. Metacognition is defined as “awareness or analysis of one’s own learning or thinking processes”.

This definition is not accurate enough in my opinion. I would add to that definition the control and the monitoring described by Nelson and Narrens and the metacognitive regulation of those processes described by Noël.

One way to use degrees of certitude

Reading Prof. Gardner-Medwin comments (here) it obviously appears that my point of view needed some clarification. The way I use degrees of certitude will be described in this post. It is only one way, not pretending to be the best, just my way. Since degrees of certitude (or degrees of certainty) do not influence the mark it is not de facto a confidence based marking system.

Students are free to associate degrees of certitude with their answers. If they don’t use degree of certitude or if they use them being wrong or being right their score will not change. They answer MCQ choosing one out of 7 to 8 propositions, 4 being always the same: none of the proposition, all propositions together, missing data to answer the question, the question doesn’t make sense. The tariff is

• for a correct answer: +1
• for an incorrect answer: -0.5
• for not answering or omission: -0.25

Training a) to answer this kind of MCQ, b) to use degrees of certitude and c) specific training regarding the topics evaluated with these tests are organised, both on line and during simulated exams. Students are free to participate. If not using CBM they receive their scores. Students using CBM for all their answers receive detailed results. They connect to the web and using their student identity number they download their full results.

On a willing base they can answer 20 questions about their performance helping them to analyse their results and regulate their learning for the next test.

Five tests are organised during the year. None is mandatory. Student are free not to participate to any and only take the final oral exam.

1. The first lesson in September is a test about prerequisite knowledge.
2. The second test in December is a formative training test with MCQ and open ended questions covering all subjects seen at the point of the formation.
3. The third test in January is a summative test that will influence the final score if the test is taken. It is the same kind of test than the previous.
4. The fourth test in Ferburay is a formative evaluation covering all subjects of the year course.
5. The fifth test in March is a summative test covering all subjects of the year course.

 

The first summative test, if taken, will count for 15% of the final score. The second summative test, if taken, will count for 35% of the final score. The oral exam in June will take the remaining percentage, meaning

• 100% if no test was taken during the year,
• 85% if the first summative test was taken,
• 65% if the second summative test was taken,
• 50% if both summative test were taken.

If a student pass both summative tests with a score above 60% he then can choose

1. to take the oral exam for 50% of the final score.
2. to report his score to the oral exam so that his grade for both summative tests will be his final grade.

Any student who has not taken both test or not passed with 60% or more is obliged to take the oral exam. Any student who has taken all 5 tests and reached a score between 45% and 60% will be evaluated on the quality of his comments. If metacognitive analysis and regulations are “good” bonus points will be added to the student actual score. The bonus points allow the student to reach the 60% needed to be able to keep his grade and not being obliged to take the oral exam.

It is a long explanation. Let’s hope it makes things easier to understand. Next post will present the tool the help student to analyse and regulated their learning using results collected with the use of CBM.

Either you know or you don’t

If evaluation was limited to True or False test then knowledge will only be limited to two stages: does know or does not know, complete knowledge or total ignorance.

There is a feeling that it should not be that simple. For instance what about those “tip of the tongue (TOT)” experience: knowledge or ignorance? On the other side and as Bertrant Russel stated in 1912: “Is there any knowledge in the world which is so certain that no reasonable man could doubt it? ” (Russell, B. (1988) The Problems of Philosophy, Prometheus Books. Page 1).

Does complete knowledge exists? When starting to learn something do human start from total ignorance?

Darwin Hunt (Human Self-Assessment : Theory and Application to Learning and Testing in Leclercq, D. & Bruno, J. (ed.) Item Banking : Interactive Testing and Self-Assessment, Berlin: Springer Verlag, 1993, F 112, 177-189) suggested to distinguish between three types of knowledge situations in which a person can be in relation to a piece of content:

1. misinformed,
2. uninformed,
3. informed.

Jans and Leclercq (Mesurer l’effet de l’apprentissage à l’aide de l’analyse spectrale des performances in Depover, C. & Noël, B. (ed.) L’évaluation des compétences et des processus cognitifs. Modèles pratiques et contextes, Bruxelles : De Boeck Université, 1999, 303-317) related these three levels with confidence degrees (or degree of certainty / certitude). Every answer is associated with a percentage of confidence in the answer. An answer wih the lowest confidence will be given with a certitude of 0%. The highest confidence will be 100%. Answer still can be right or wrong. But 4 stages can be made:

1. Wrong answer with high certitude or dangerous knowledge;
2. Wrong answer with a low certitude or unawareness;
3. Right answer with a low certitude or mid knowledge;
4. Right answer with a high certitude or usable knowledge.

In their original paper Jans and Leclercq used a logarythmic scale for students to chose their certitude from. They defined 7 stages defined by different value of the given certitude of the answer

• right answer with certitude from 85 to 100%: perfect knowledge
• right answer with certitude from 15% to 85%: partial knowledge
• right answer with certitude from 0% to 15%: doubtful knowledge
• no answer: ignorance
• wrong answer with certitude from 0% to 15%: admitted unawareness;
• wrong answer with certitude from 15% to 85%: confusion;
• wrong answer with certitude from 85 to 100%: ignored unawareness.

It should be up to the education team and based upon educational objectives to fix the limite between a high level of confidence and a low level of confidence. A medical student in surgery may need right answers with certitude of 100% to be considered as usable knowledge, all other right answer falling into the mid knowledge class. For a person taking a test for a driving licence the limit for usable knowledge might be over 50%, all right answer below 50% falling into the mid knowledge class. Of course if using True / False questions a certitude below 50% has no meaning because the odds finding randomly the right answer already are 50%.