Abstract

---

Background:
Problem based learning (PBL ) is an instructional method which utilised clinical cases or problems as a context for students learning in basic and clinical medicine. Melaka Manipal medical College has incorpoarted PBL into its undergraduate medical curriculum. During the process we developed a system to assess the students’ performance during the PBL process. The study was aimed to understand the correlation of this method with other existing assessment methods and its reliability.
Method: The study reports the data generated during the PBL process of september 2006 batch. Students’ performance was assessed using the new assessment method. The marks obtained during the three PBL sessions were analysed for its correlation with other assessment methods. Faculty feedback regarding the assessment system was also taken.
Results: Percentages of students’ who scored above ≥ 4/6 in the brain storming scores increased from the first to the third PBL session (80.4% to 90%) , tutors were satisfied with the students’ performance. PBL evaluation provided a documented feedback to the students on their knowledge, skills (e.g. use of resources, problem analysis and solving) group work skills and attitudes.

Background

---

Problem based learning (PBL) is an instructional approach that uses problem as a context for students to acquire problem solving skills and knowledge. It embraces principles of effective learning and teaching. It is student directed and fosters intrinsic motivation, promotes active learning, encourages peer teaching, involves timely feed back, and can support student self assessment. The most important function of the assessment process is to enhance student learning. The assessment of students’ performance during the PBL is essential as the assessment is a motivator for learning. The assessment system should be reviewed and sufficient modifications should be placed based on the PBL philosophy at the time of the implementation of curricular reforms like PBL1.
Melaka Manipal Medical College (MMMC) offers the Bachelor of Medicine and Bachelor of Surgery (MBBS) program which is a twinning program with Malaysia and India. The program runs in two phases, Phase I and Phase II. Students undergo Phase I training in Manipal, India and it comprises first year (Phase I, Stage I), second year (Phase 1, Stage II A) and six months of clinical training (Phase 1, Stage II B). Phase II component comprises clinical subjects which are taught in Malaysia. About 98% of the students are Malaysian citizens and the rest constituted from different parts of the world.
The curriculum of each year of stage I is dividedinto four blocks , approximately each block of 10 week’s duration. The content coverage in each block consists :
Block I - Basic concepts and immunity
Block II – Central nervous system, special senses ,skin, muscles, bones and joints
Block III- Gastrointestinal, cardiovascular and respiratory system
Block IV- Endocrine, reproduction, Kidney and electrolytes
MMMC has adopted a hybrid curriculum including instructional methodologies like PBL, self directed learning strategies and didactic lectures. To implement PBL sessions, it became imperative to develop an assessment system used by the tutors, to assess students’ performance . Assessment system included range of competencies and functions like group dynamics, active participation in a group task, presentation skills, and the mastery of the subject. The aim of this study was to analyze the effectiveness of the assessment system in evaluating the students’ performance during the PBL sessions, to check the reliability of the assessment system ,to study the correlation of PBL assessment system with other assessment methodologies of the same batch of students and to ascertain faculty perceptions regarding the assessment system.

Materials and Methods

---

PBL was introduced to March and September 2006 batches of MMMC. The study reports the data generated during the PBL sessions of September 2006 (n=133) batch; during their second year of phase I (stage II). Students were divided into 10 groups (10-13 students in each group), and each group had a faculty facilitator. All tutors were trained to facilitate the PBL process and the use of assessment system. The problems were developed by the same group of faculty using a standardized system ‘seven principles of effective case design in PBL’ as proposed by Dolman et al (1997)2. At the completion of each session, the students were awarded the following grades depending on the performance using the newly developed assessment system (Appendix A). The grades given were E, for excellent (score6/6), D, for distinction (score 5/6), H for honors (score4/6), P for pass, (score 3/6), F for fail (score below 3/6), A for absent (score 0). The grades were awarded separately for brainstorming, presentation, participation, and content coverage based on the objectives. The assessment outcome was analyzed statistically, using SPSS package, keeping the score of 4/6 (students scoring ≥ 60% marks) as cutoff score for the improvement in performance since the minimum pass percentage in our institution was 50% . We compared the scores of the study group, for three PBL sessions conducted during three different blocks.
In order to ascertain whether the assessment system was complementing with other summative assessment systems patterns already existed in MMMC, the correlation of the PBL marks with short essay and multiple true false questions in the assessment scheme using Pearson correlation method was adopted
To understand the students’ perceptions regarding the PBL assessment process, students were asked to give feedback on the PBL assessment scheme. The responses were collected by the end of the second year course in a qualitative manner. 111 students responded by giving their written feedbacks
In order to understand the faculty perceptions regarding the PBL assessment scheme, we administered a questionnaire (Appendix B) to the faculty members of Melaka Manipal Medical College. The questionnaire had 16 items pertaining to the effectiveness of this method as an assessment strategy, whether it facilitated learning, provided effective feedback and how easy or difficult was the method to use during the PBL process. The responses were collected in a five point Likert scale format comprising Strongly Agree to Strongly Disagree. Each response was given a value from 4 to 0 for Strongly Agree to Strongly Disagree respectively.
Statistical analysis was performed using cluster analysis and Pearson correlation using statistical software SPSS version 16.

Results

---

Percentages of students’ who scored above ≥ 4/6 in the brain storming scores increased from the first to the third PBL session (80.4% to 90%),  which indicates the improvement in understanding the process of PBL. Percentage of students who scored above 4/6 in communication and presentation skills scores increased from 85 to 94% and in content scores, which indicate the active learning component, increased from 85.9% to 92.5%. Overall , tutors were satisfied with the students’ performance. The frequencies of the scores obtained by the students in different blocks were represented in the illustrations   (1-4).  Faculty perceptions regarding the PBL evaluation scheme was shown in illustration 5. The PBL scores correlated moderately and significantly with other assessment schemes illustration 6.

Discussion and Conclusion

---

Evidences have made the statement ‘assessment drives learning’ into a principle in education and it is well known that assessment influences students learning behavior. It is also evidenced by many researchers that no one assessment system can be ideal and each one has its own inherent deficiencies. So student evaluation will require the use of multiple complementary instruments.3
Tutors role is pivotal in conduction of PBL process. The role of the tutor is important in fostering the self directed learning activities students perform during a PBL process.4-6
The assessment system has to be aligned to the objectives and teaching learning activities performed during an educational activity. PBL is one of the most constructively aligned methods of active learning.7
The brain storming session of the PBL process revolves around various aspects of team work and critical analysis of problem. The students scoring more than 60% marks in the assessment of brain storming session from the PBL in block I through the PBL in block IV (illustration1) had increased , indicated an improvement in ability to propose the hypotheses, identification of learning objectives, and moreover a constructive thinking process. This also indicated more familiarity with the PBL process. The improvement in the active participation score indicated an improvement in group dynamics, demonstrating a view point to others in the group, providing constructive feedback (illustration 2). Presentation scores indicated the students’ improvement in communication skills and their effective use of visual aids. The improvement in content scores indicated the improvement in the active learning among the students by effectively utilizing the relevant resource materials, sharing and understanding them among the peers.
As per the opinion of the faculty the method was easy to use and it was possible to carefully listen and evaluate the students using this assessment method. They had opined that they do not have difficulty in facilitating the PBL process as they were performing the assessment through this method. At the same time the faculty perceptions on the reliability of this system and it being a complete valuation process were low. (illustration 5). The feedback showed that the criteria for differentiating students’ performance were not clear for many faculty members. This made us look into the guidelines of assessment and restructuring of the criteria.
In similarity to other reported studies (Dodds etal, 2001) 8, there was moderate and significant correlation between scores of other assessment modalities during the continuous assessment in each subject and PBL scores. (illustration 6). This shows that our PBL assessment system seems to give neither advantage nor disadvantage to the students’ in scoring, and complements the other assessment modalities already in use.
PBL evaluation provided a documented feedback to the students on their knowledge, skills (e.g. use of resources, problem analysis and solving) group work skills and attitudes (e.g. independence, respect for the contribution of others, leadership and self reflection).

References

---

1. Davis M H and Harden R M. An extended summary of AMEE Medical Education Guide No 15. Medical Teacher 1998; 20:2, 317-322
2. Dolmans, Diana H. J. M., Snellen-Balendong, Hetty, Wolfhagen, Ineke H. A. P. and van der Vleuten, Cees P. M.  'Seven principles of effective case design for a problem-based curriculum', Medical Teacher 1997; 19:3, 185 — 189.
3. Van Luijk and van der Vleuten.  Assessment in Problem-based Learning (PBL). Ann Acad Med Singapore 2001; 30:347-52.
4. Thomas R E. Problem-based learning: measurable outcomes. Medical Education 1997; 31: 320-9.
5. Schmidt H D. Foundations of problem-based learning: some explanatory notes. Medical Education 1993; 27: 422-32.
6. Norman G R, Schmidt H D. The psychological basis of problem-based learning: a review of the evidence. Acad Med 1992; 67: 557-65.
7.Biggs J. (2002) Aligning teaching and assessment to curriculum objectives. Retrieved September 21, 2006, from LTSN Generic Centre website: http://www.heacademy.ac.uk/resources.asp?process=full_record§ion=generic&id=154
8. Dodds A E, Osmond R H, Elliott S L. Assessment in Problem-based Learning: The Role of the Tutor. Ann Acad Med Singapore 2001; 30:366-70

Source(s) of Funding

---

Funding by Manipal university

Competing Interests

---

No competing intrests

Disclaimer

---

This article has been downloaded from WebmedCentral. With our unique author driven post publication peer review, contents posted on this web portal do not undergo any prepublication peer or editorial review. It is completely the responsibility of the authors to ensure not only scientific and ethical standards of the manuscript but also its grammatical accuracy. Authors must ensure that they obtain all the necessary permissions before submitting any information that requires obtaining a consent or approval from a third party. Authors should also ensure not to submit any information which they do not have the copyright of or of which they have transferred the copyrights to a third party.
Contents on WebmedCentral are purely for biomedical researchers and scientists. They are not meant to cater to the needs of an individual patient. The web portal or any content(s) therein is neither designed to support, nor replace, the relationship that exists between a patient/site visitor and his/her physician. Your use of the WebmedCentral site and its contents is entirely at your own risk. We do not take any responsibility for any harm that you may suffer or inflict on a third person by following the contents of this website.