Journal Club #6: G. Di Pietro, Bulletin of Economic Research, 2012, 65(1), 65 – 81. [Link]
It is my experience in academic discourse that when a change is proposed, those advocating the change rely on “gut instinct” and “common sense” while those opposing it seek evidence from the literature. My own institution is currently planning a significant change in the academic calendar, and while thinking about this, I came across this paper.
The author examines whether an institution’s reform involving moving semester exams to end of year exams had a negative impact on student performance. The system under study had two semesters, with exams in January and June, and the reform meant that there would be no January exams, just exams for the entire year at the end of the year (the way it used to be, I hear the chorus).
Reasons for the reform included the desire not to overburden first years with exams in January, and to allow students more time to digest their material. The author doesn’t hold back in stating that he believes the reasons for the reform were administrative and financial.
The study involved comparing the mid-term results from modules, and comparing these with semester exam results. Assuming that the mid-term results stayed constant before and after reform, the difference between the mid term mark and the exam performance mark before reform and after reform allow for a measure of the impact of reform on student grades to be determined.
The results shown demonstrate that there was a drop in student performance when the exams moved out of semesters to the end of the year, with students scoring 4 points lower (nearly half a grade).
The author concludes with a statement that sounds a note of caution to those considering changing calendars (DIT colleagues take note!)
These findings may have important policy implications. Changes in examination arrangements should ideally be tested for their impact on student performance before they are introduced. Many changes in higher education are driven not by student learning considerations, but by other reasons such as financial and administrative convenience.
Discussion
Do you have any feelings regarding when modules should be examined?
Is it true to say that grade have fallen when marking is subjective? Learning outcomes will dictate but employers will give you a different view. I deal with employers and they choose students from programmes on the basis of what the graduate is capable of doing when they are in the labour market.
There is a lot of bluster and nonsense spouted about the value of semesterisation; most by those pro the system. I have taught on both systems. Non-semesterised in my opinion, with properly designed continuous assessment system, is a far superior educational model . The opportunity for cross linking modules over a longer period of time, deep learning and development of expertise above knowledge and also being able to focus on pedagogy instead of exam boards.
Semesterised systems on the other hands become exam thread-mills often with a lack of depth and deep understanding often taught by lecturers who are products of the system that they value so much. One of the greatest values (among some sound ideas of learning and assessing in smaller increments) of semsterisation is the fear of the exam as motivation; is this really what an education is about? I work in IOT with both systems; no one has ever said or proved that one system is so superior that it is the obvious choice.
What is the value of semesteristion to a third level institution? Being cynical I could say it for Erasmus student and international profile rather than any sound pedagogical reason. This would be unfair as there are students who thrive in this system and particularly part time student.
A deeper question would be to examine the use of post-grads and teaching assistants in the University sector to deliver the vast bulk of labs and tutorials. Few if any of these have ever been assessed for their suitability as teaching staff on any system.
Re subjectivism: the author looked at three years before reform and three years after.
My point of highlighting this paper is to begin to address what you say; nobody says one or other is better – in fairness this guy is trying to do that.
Some of the other points are a bit off topic- but are you confusing modularisation and semesterisation?
Michael
All sorts of factors here. One is that (in my view) semesterisation actually makes exams easier. It is easier to learn bite-size chunks of material than large meal-size ones. Furthermore the large chunk probably provides scope for asking questions whose answers require higher order thinking. Essentially I don’t think that this is a comparison of like with like.
Agree, but the end of semester exams were still modular, so it wasn’t like the exam format changed; they were just all held together. So I don’t think that’s a factor in this case. But I agree that the modular system acts against seeing (and examining) the bigger picture.
Thanks for commenting,
Michael.
Can I clarify a couple of points?
Did the end of year assessment involve assessing material taught before Christmas but not assessed at Christmas OR did it involve material being delivered across the entire year and being examined in May/June?
Did all modules have an equal weighting?