1. Implementation
The Faculty of Medicine of the University of Lisbon (FMUL) has a system for marking examinations by optical reading (OCR) since the first semester of the 2010/11 academic year. This service is available to FMUL lecturers who teach the Integrated Master Degree in Medicine, the Integrated Master Degree in Biomedical Engineering and the Bachelor Degree in Health Sciences.
The management of the system is the responsibility of the Internal Evaluation and Quality Assurance multidisciplinary team, which reports directly to the Secretary- Coordinator of FMUL.
The OCR ensures:
- Fast delivery of results;
- Information security;
- Confidentiality;
- Statistical analysis of grades.
To use the service it is necessary to observe the following requirements:
- Multiple choice examinations, with five response options per question;
- Examination papers cannot exceed 120 questions;
- There must be a single right answer for each question;
- The answer sheets printed by the Internal Evaluation and Quality Assurance multidisciplinary team must be used;
- The grid with the right answers must be provided;
- Receipt and return of examinations must be monitored.
After exams have been read, results are sent on an Excel file. This file contains the individual results of each student, information about the classifications, the list of grades in alphabetical order, and the statistical analysis of results (summary). The exam images are also sent (through a link).
At the end of the semester, individual reports are sent to the lecturers responsible for that particular subject, containing the histogram with the grades of students and the frequency distribution of answers for each individual test. This last feature allows lecturers to see which questions students failed the most and the option responses selected.
2. Enforcement
Below there are some indicators of the execution of the OCR during the 1st semester of 2010/2011 (Integrated Master Degree in Medicine):
- Out of the 22 Curricular Units/Disciplinary Areas identified as users of multiple choice examinations, 19 requested the OCR service;
- These 19 Curricular Units/Disciplinary Areas requested the OCR service again in the 2nd period of examinations;
- Given that the predicted response time for returning exams to users was not to exceed 48 hours, this goal was largely overcome, as the average time was 27 hours in the 1st examination period and 12 hours in the 2nd examination period;
- There was total adhesion to the OCR system by the Curricular Units/Disciplinary Areas in years 2, 3, 4 and 5.
3. Evaluation
According to information sent to FMUL lecturers, the OCR system was evaluated by its users. In order to evaluate user satisfaction, a survey at the end of exam period was carried out. The questionnaire sent to all users (n = 19) had a return rate of 100% of responses.
The following chart shows the distribution of average satisfaction on a scale of four categories of evaluation against the indicators used in the OCR service: dissemination of the service; procedures related to requesting the service; waiting time for return of results; accessibility of the information provided (Excel files and images); functionality of the information provided (Excel files and images), and overall satisfaction with the OCR service.
Chart 1 – Satisfaction with the OCR service
.JPG)
(Dissatisfied) (Not Satisfied) (Satisfied) (Very Satisfied)
In the evaluation that was conducted, we stress the following:
¬ All users said they will use the OCR again.
¬ All users said they will recommend the OCR service to a colleague.
4. How to request the service
The OCR management unit is located in the Technical-Administrative Services on level 3 of the building shared with Santa Maria Hospital. The service is the responsibility of Pedro Marçal and António Santos.
Contact details:
leituraexames@fm.ul.pt
Internal Number: 44580 | 44581
Tel: + 351 217985179
Pedro Marçal, António Santos
Office for the Support of Governing Bodies – Internal Evaluation and Quality Assurance
leituraexames@fm.ul.pt
The Faculty of Medicine of the University of Lisbon (FMUL) has a system for marking examinations by optical reading (OCR) since the first semester of the 2010/11 academic year. This service is available to FMUL lecturers who teach the Integrated Master Degree in Medicine, the Integrated Master Degree in Biomedical Engineering and the Bachelor Degree in Health Sciences.
The management of the system is the responsibility of the Internal Evaluation and Quality Assurance multidisciplinary team, which reports directly to the Secretary- Coordinator of FMUL.
The OCR ensures:
- Fast delivery of results;
- Information security;
- Confidentiality;
- Statistical analysis of grades.
To use the service it is necessary to observe the following requirements:
- Multiple choice examinations, with five response options per question;
- Examination papers cannot exceed 120 questions;
- There must be a single right answer for each question;
- The answer sheets printed by the Internal Evaluation and Quality Assurance multidisciplinary team must be used;
- The grid with the right answers must be provided;
- Receipt and return of examinations must be monitored.
After exams have been read, results are sent on an Excel file. This file contains the individual results of each student, information about the classifications, the list of grades in alphabetical order, and the statistical analysis of results (summary). The exam images are also sent (through a link).
At the end of the semester, individual reports are sent to the lecturers responsible for that particular subject, containing the histogram with the grades of students and the frequency distribution of answers for each individual test. This last feature allows lecturers to see which questions students failed the most and the option responses selected.
2. Enforcement
Below there are some indicators of the execution of the OCR during the 1st semester of 2010/2011 (Integrated Master Degree in Medicine):
- Out of the 22 Curricular Units/Disciplinary Areas identified as users of multiple choice examinations, 19 requested the OCR service;
- These 19 Curricular Units/Disciplinary Areas requested the OCR service again in the 2nd period of examinations;
- Given that the predicted response time for returning exams to users was not to exceed 48 hours, this goal was largely overcome, as the average time was 27 hours in the 1st examination period and 12 hours in the 2nd examination period;
- There was total adhesion to the OCR system by the Curricular Units/Disciplinary Areas in years 2, 3, 4 and 5.
3. Evaluation
According to information sent to FMUL lecturers, the OCR system was evaluated by its users. In order to evaluate user satisfaction, a survey at the end of exam period was carried out. The questionnaire sent to all users (n = 19) had a return rate of 100% of responses.
The following chart shows the distribution of average satisfaction on a scale of four categories of evaluation against the indicators used in the OCR service: dissemination of the service; procedures related to requesting the service; waiting time for return of results; accessibility of the information provided (Excel files and images); functionality of the information provided (Excel files and images), and overall satisfaction with the OCR service.
Chart 1 – Satisfaction with the OCR service
(Dissatisfied) (Not Satisfied) (Satisfied) (Very Satisfied)
In the evaluation that was conducted, we stress the following:
¬ All users said they will use the OCR again.
¬ All users said they will recommend the OCR service to a colleague.
4. How to request the service
The OCR management unit is located in the Technical-Administrative Services on level 3 of the building shared with Santa Maria Hospital. The service is the responsibility of Pedro Marçal and António Santos.
Contact details:
leituraexames@fm.ul.pt
Internal Number: 44580 | 44581
Tel: + 351 217985179
Pedro Marçal, António Santos
Office for the Support of Governing Bodies – Internal Evaluation and Quality Assurance
leituraexames@fm.ul.pt
