Managing information from student perspective during COVID contingencies

My colleague Chris Mowat and I co-wrote a blog post for our university’s Teaching Matters blog, which I am linking here as it pulls together a lot of our activities into one post, so might be useful for people in a hurry!

The end of academic year is a busy time for students in normal circumstances, but this year, in addition to spending this time reviewing all of their courses and getting ready for assessment, there is a swell in information relating to managing alternative arrangements due to COVID-19 that our honours students have to process. As a combined effort, we have been working to mediate this information in a way that is clear, coordinated, and student-focussed. We share below five headline strategies that are working well for us, what we are doing next, and reflect on how the last few weeks have been. [Link to Teaching Matters blog post to continue reading]

(Graphic courtesy of @ScienceNerdSACR and used with permission) - see post linked for details
(Graphic courtesy of @ScienceNerdSACR and used with permission) – see post linked for details

Managing the open-book exam process

At Edinburgh, we are mostly moving to replacing our 3 hour exams with open book exams. We had initially intended these to be within 24 hour timeframes, but the University has mandated 48. Otherwise, things are as described in the previous post.

So students will need to access an exam paper from a  specific “start time” and submit their written answers no later than 48 hours than that start time. Easy!

Exam Process Guide

Having looked through the various options, I am going with the following 5 step plan based on Blackboard Assignments (rather than Turnitin), described below in terms of front of house and behind the scenes:

For students:

  1. Students will be able to download the paper in advance from Blackboard (called “Learn” here), but it is password protected.
  2. At the start time, the password is released on Blackboard and by email. We didn’t want 150 students trying to download a PDF at the same time.
  3. Students complete their answers on paper.
  4. Students scan their answers using Adobe Scan app to create a PDF. For us, one lecturer corrects one exam question, so we want these uploaded on a question by question basis.
  5. Students upload their answers.

I’ve made a draft video (subtitled) outlining this process, which we are testing robustly this week! We are then going to release to students and allow them play with a mock set-up.

Behind the scenes, this means:

  • Setting timings so that Blackboard courses release in line with exam timetable, and components (e.g. password, answer submission areas) release at the right time
  • Enabling anonymous marking so that the student number doesn’t appear in the file name
  • Allowing multiple submissions so that when students upload the wrong file (it will happen) they can submit the correct file – this needs careful management post hoc.
  • After time window has closed, each question will be downloaded and shared with examiners. I am not going to ask my colleagues to annotate files in any way; they will simply keep a mark tally per exam number related to marking scheme so that they can refer back to that if there are queries.
  • Marks can then be returned in an Excel sheet by exam number, and these can go into “the system” for exam boards.

EASY! :) What can go wrong? (no, really…?)

Supporting student study in the “pivot” online

As mentioned in last post, we are focussing our current efforts on two strands – maintaining and promoting academic focus, and being active in student support, the second aspect being led by m’colleague Chris Mowat.

In terms of academic focus, we are moving our closed book exams to open book. Of course this seems “easier”, but I think brings new challenges for students in their study. All that study time spent learning things off doesn’t seem as important now, and we are moving our focus to asking students to think about what questions are asking, showing their understanding concisely in answers, etc. In other words, now that students have a chance to write as much as they want, the challenge instead becomes, what should they do to show they understand? (It’s like, I don’t know, closed-book exams aren’t a good way to assess or something, huh).

Anyway, to help with this I have produced the guide below for students. Hopefully it has some useful prompts in terms of getting focussed and keeping organised, thinking about “pivoting” (ugh) to the new assessment regime, along with Chris’ ongoing guidance re student support. I’ve reproduced the text below, if any of it is of use, please reproduce as you need. CC-by-CoronaVirus 4.0. (With thanks back to the internet for various bits of guidelines that fed into this.)

Supporting Students Study graphic

Text of graphic:

As we have moved quickly to online and remote learning and teaching, you may need to work to establish new study patterns. This will take a little time to get used to, so take your time, and take care of your wellbeing first.
This guide will outline how you can make a plan to adjust your studying and help you regain control of your work.
In this guide we’ll talk about:
» Staying organised
» Adjusting to new assessment protocols
» Connecting with others
» Keeping in touch with the School

You may need to adapt your study habits. Find out what works best for you and establish a regular pattern of work, ensuring you include downtime. This is a marathon, not a sprint!
1. Organise your notes and study
There has been a lot of upheaval in the last weeks of Semester. Your task is to manage your ongoing workload in an organised and coherent way. To do this you can:
make sure you know where all the materials are for each lecture course unit. Live lecture recordings will be in the Lecture Recordings area, with online classrooms in the Course Collaborate channel linked in each course;
make sure you know any revisions to the assessment protocols, so you can plan your study accordingly;
plan and keep track of your study tasks, including scheduling downtime;
note the contacts for each course lecturer;
share your plans with peers in your study group so that you can coordinate your work.
To keep your study focussed and on target, a suggestion is to divide each study day into thirds, with a study session or a scheduled downtime in each third.
In each study session, you can:
focus on reviewing the content of particular topics, drawing on lecture recordings as you need;
work through tutorial materials, discussing with peers in your study group through whatever electronic communication means you have decided on;
test out your understanding with past paper questions;
draw up a list of questions that you wish to discuss with your peers and with your lecturers.
In each session, stay focussed on one topic. Multi-tasking (or micro-tasking) is a very poor learning strategy (only about 2% of the population can multi-task). Work on one topic, wrap it up so that you can return to it easily (clear questions that need follow up), and move on.
In your study sessions, aim to establish a rhythm. Structure the time within each study session, by using, for example, the Pomodoro technique (e.g. a series of three 50 minutes on, 10 minutes break in one study session).

Remember to schedule downtime including full days away from study. Make sure you maintain your usual daily routines of personal care, eating, and social contact by phone or online.

2. Adjusting to new assessment protocols
Until recently, students’ study has been focussed on preparing for closed book exams. The new assessment protocols mean that these exams are now open book. This will mean some changes to your study requirements, but most importantly some additional focus on how you answer questions asked:
The guiding principle for closed-book exams on “making sure you answer the question asked” applies even more in open-book assessments. Students should read questions carefully and make sure that they answer what is being asked. Assessors will be looking to see whether students can concisely answer the question asked. This is the main challenge in open-book assessments.
When studying and reviewing past papers, check that you can identify exactly what is being asked. Remember, exam questions are written so that they can typically be answered in timed conditions, so assessors will look to see quality of your answers addressing the specific chemistry asked, rather than seeing all of their notes reproduced verbatim.
Continue to practice drawing chemical structures, diagrams, and any figures or graphs as you normally would. All of the work you produce for open-book assessments must be in your own hand.

Experience tells us that students often write lots of material, but don’t necessarily answer what is being asked. Make sure your study includes a focus on identifying what is being asked.

3. Connecting with others for study
Much of learning and revision is based around discussions, both among students and between students and staff. Make sure you keep connected as follows:
You should aim to contact lecturers with specific questions arising out of your study, or arising out of revision sessions and discussions with your peers. It is easier for lecturers to address specific questions. Lecturers may also be able to direct you to online reading in the face of library closure.
If you don’t understand a topic generally and don’t know where to start, even after reviewing materials, ask your lecturer for suitable reading. Reading about your topic in a wider context can often help with understanding.
Make sure to form discussion groups. If you are not already in a study group and want to be, the best person to start out with is your lab partner. From there, look to grow it to groups of 4 – 6. It is good when there is a mix of abilities—explaining something to others is very beneficial.
Working in study groups means that you can contact staff as a group with any questions emerging from study.

4. Re-invent the social space online!
One of the challenges over the coming weeks up to the assessment period will be staying focussed during periods of isolation. We’re all missing our lovely Museum, and in place of that, the School are scheduling regular online drop in sessions—these are informal and are intended for social contact as much as addressing any questions among students. Join in these sessions, even if it is only to listen.
All students should make an effort to check in on each other regularly. If someone has been absent from your group meetings or chat, check in with them to see that they are okay. If you aren’t getting responses within a day or two, let your Personal Tutor know.

Keep connected with your class, either as study groups or just social connections. Everyone will need a different level of social contact, but make sure there is someone there for you when you want to disconnect from study. Always remember that staff are available if you need someone to talk to.

5. Stay in touch with the School
If anything is troubling you, or you want to talk about how you are getting on generally, let us know. We can arrange Skype or phone calls at any time. The School’s Personal Tutor system is ready and waiting, and if you have any queries, contact your Personal Tutor, the Senior Personal Tutor (Chris Mowat), the Director of Postgraduate Teaching (Nanna Lilienkampf), or the Director of Teaching (Michael Seery).
All of us are ready to answer and address any queries you have. Even if you don’t have a specific query and just want to reach out, contact us:
Your Personal Tutor
Chris Mowat:
Nanna Lilienkampf:
Michael Seery:
General queries:
There are University resources available including student study resources in the Library and IAD and student support resources. For these, and full details of University guidance regarding COVID-19, see the website:
www.ed.ac.uk/news/covid-19/current-students

Managing and recovering from serious interruption to teaching

The internet has been full of amazing advice over the last few weeks on technical issues relating to moving online. I’ve nothing to add to that, but thought I might offer our perspective from managing the whole process overall. Below are some notes I have sketched out on how we have dealt and are continuing to deal with with events of the last few weeks. We are sharing these with other chemistry departments in UK/IE, and I am publishing here for more general interest. As well as sharing, I am keen to learn. What is missing, what isn’t clear… (what’s wrong…) from the compilation below.

Phase 1: Immediate Disruption – managing teaching alternatives

Aim: to continue the curriculum as much as possible, with clear communication to staff and students as to what options are available, and where resources are. This is done by time release (week by week) of prepopulated online class links and other details on the course homepage of the VLE.

  • Clear decision making process with authority delegated to key stakeholders. One person responsible in this phase for all staff and student communications with a back-up nominated and cc-ed from the outset; staff communications crucial at this stage;
  • Lectures: We mapped our existing timetable online and delivered a combination of:
    • Live lecture via webinar system (e.g. Blackboard Collaborate) – good for staff who have a full slide deck and little board work
    • Lecture recordings (full or in parts) released to students with live lecture discussion via webinar (a form of flipped lecturing) – good where existing recordings are available and staff comfortable with live classrooms
    • Lecture given live and live streamed, with questions via email (assuming building is open) – good for staff who are uncomfortable giving webinar type lecture
  • Tutorials were managed in a similar manner, and staff typically:
    • Go through tutorial answers with discussion in online webchat
    • Release tutorial answers and have discussion in online webchat on any difficulties
    • Release tutorial answers and have discussion by email or discussion board
  • Labs: We were at the end of our lab cycle so finished just short of the finish line. Clearly future contingency will need to consider laboratory alternatives (see Phase 4).
  • Coursework: all activities moved to online submission. Some events, such as presentations, changed to students submitting PowerPoint decks with outline of presentation, or were cancelled.

Phase 2: Preparing for Alternative Examinations

Aim: to introduce an alternative to in-person closed book exams, that matches similar assessment of learning outcomes, allows students to handwrite, can be completed anywhere in the world, being as low tech as possible, and allows students to perform in an equitable manner

  • We have opted to issue our exams as open book assessments.
  • Students will be given paper in advance, password protected, which they can download from a special VLE site for each paper. This reduces internet connection concerns on the day.
  • Students will be asked to agree to a revised Code of Conduct.
  • Students use the password released at the start time of the assessment. Password release by email, on VLE, and possibly by text.
  • Students take photos of their work, and generate a PDF, which can be concatenated (using Adobe Scan mobile app, a solution I am currently exploring – thanks to Ross Galloway for this tip) and uploaded via mobile to VLE, on a question by question basis (use grayscale to reduce size of file). We are exploring a parallel e-mail bin for each exam. Option with advance notice of students sending papers back by registered post.
  • Process will be issued using existing published timetable, assuming no clashes within (exactly) 24 hours of start times.
  • Answer files then distributed to markers in a batch zip file for anonymous marking.
  • Note: at Edinburgh, all non-honours exams were cancelled centrally, so course (module) marks for these are generated by known existing marks – lab and other course marks in our case.

 Phase 3: Get students’ focus back on studying chemistry

Aim: to “normalise” the new arrangements, online teaching, preparation for open book exams, and make sure isolated students’ welfare is closely monitored

  • Return to regular communication with central focus on two streams – one on academic matters and one on student support. At Edinburgh we split this activity across two people (Director of Teaching and Senior Personal Tutor).
  • In academic terms, communications around getting on with study, study advice for different assessment formats, making sure students are pushed back to thinking about tutorials, revisions, thesis writing etc. The tone is regular, even mundane (by design!), just to aim to get students to focus in the current chaos. In addition, we will be releasing a mock exam system so that students can interact with it, get a feel for it and check it out in their situation, for reassurance (on both sides) more than anything else. All academic guidance about new arrangements located on a single webpage, with all VLE course (modules) linking to it. This means students have information centrally as well as in email.
  • In student support terms, we are worried about isolation. We pushed strongly in all email correspondence and Twitter unambiguous messages that we want to stay in contact, and prompting students to contact DoT/SPT/PT for anything; the language here was personal and casual. This is then followed by regular webchats – twice weekly online Q+As where students can ask anything. These have been very popular, and have moved from lots (and lots) of queries about exam paper format to students (and staff) sharing pictures of their dogs. We don’t care, the intention is to just keep a conversation going. We’ve also created a “kudoboard” which will act as an online Yearbook (see: https://www.kudoboard.com/) – students can just post anything they want there.
  • These approaches have been very well received by students. The key message is: you cannot communicate enough, but that communication needs to be well planned, and unambiguous. We have a policy of getting every major email second read, at least, with relevant stakeholders.

Phase 4: Mop up and prepare for future?

Aim: once immediate task is completed, what can we learn? And how do we prepare for a more serious (mid-semester) interruption.

  • We have garnered a huge swell of goodwill from staff who have seen many benefits of lecture recordings, online interaction, and dialogue, and the importance of dialogue in general. We plan to consider how to continue this, even in low tech ways, when teaching returns to normal.
  • The move to open book assessments will open a big opportunity for chemistry to really rethink assessment for 21st century. Lots of opportunities to pursue here.
  • The chemistry community (worldwide) is very underprepared for major interruptions to laboratory teaching, and I think there is scope for significant collaboration in generating datasets and banks of materials for alternatives to lab classes in the event of future interruptions.

 

Looking back on ten years of Chemistry Education Research and Practice

Over the last 10 years from 2010 – 2019, Chemistry Education Research and Practice, a free-to-access journal published by the RSC (of which I am currently Editor) has published 631 articles, which have been cited 5246 times (data from Web of Science). So what has been “hot” this last decade? It seems whatever way you cut it, it was flipped learning and organic chemistry… Below I’ve cut the citation statistics a few ways – these comments are based on citations rather than judgement on the work itself.

In terms of number of citations, Keith Taber’s perspective on the chemistry triplet tops the poll, with 115 citations on Web of Science (Clarivate) and 262 on Google Scholar. In fact the top 4 hits are perspectives or reviews.

Title Authors/ Year Source Total Citations (Google Scholar) Average per Year (Google Scholar Average)  Article Type
Revisiting the chemistry triplet: drawing upon the nature of chemical knowledge and the psychology of learning to inform chemistry education Taber, Keith S. / 2013 Link 115 (262) 16.43 (37.4) Perspective
Rethinking chemistry: a learning progression on chemical thinking Sevian, Hannah; Talanquer, Vicente / 2014 Link 77 (136) 12.83 (22.7) Perspective
Education for Sustainable Development (ESD) and chemistry education Burmeister, Mareike; Rauch, Franz; Eilks, Ingo / 2012 Link 75 (210) 9.38 (26.2) Perspective
Flipped learning in higher education chemistry: emerging trends and potential directions Seery, Michael K. / 2015 Link 67 (145) 13.4 (29) Review
The flipped classroom for teaching organic chemistry in small classes: is it effective? Fautch, Jessica M. /2015 Link 66 (147) 13.2 (29.4) Article
Student attitudes toward flipping the general chemistry classroom Smith, J. Dominic / 2013 Link 61 (132) 8.71 (18.9) Article
A comparative study of traditional, inquiry-based, and research-based laboratory curricula: impacts on understanding of the nature of science Russell, Cianan B.; Weaver, Gabriela C. /2011 Link 54 (97) 6 (10.8) Article
Structure and evaluation of flipped chemistry courses: organic & spectroscopy, large and small, first to third year, English and French Flynn, Alison B. / 2015 Link 53 (102) 10.6 (20.4) Article
Development and validation of the implicit information from Lewis structures instrument (IILSI): do students connect structures with properties? Cooper, Melanie M.; Underwood, Sonia M.; Hilley, Caleb Z. / 2012 Link 51 (85) 6.38 (10.6) Article
Let’s teach how we think instead of what we know Talanquer, Vicente; Pollard, John / 2010 Link 49 (101) 4.9 (10.1) Article

A fairer way of looking at citations is the average number of citations per year. This means that older papers which have had a longer time to accumulate citations are averaged out. (However this is still not fair to recent papers, which will not have had a chance to be cited at all, or will not benefit from a cumulative citation effect). However, on this cut, Weaver’s paper on nature of science in the lab (this is an excellent paper which really should be better known), Cooper’s paper on IILSI, and Talanquer’s paper on teaching how to think drop out of the top 10, and the top 10 based on average citations become (new additions with *):

Title Authors Publication Date DOI Average per Year (Google average)  Article type
Revisiting the chemistry triplet: drawing upon the nature of chemical knowledge and the psychology of learning to inform chemistry education Taber, Keith S. 2013 Link 16.43 (37.4) Perspective
Flipped learning in higher education chemistry: emerging trends and potential directions Seery, Michael K. 2015 Link 13.4 (29) Review
The flipped classroom for teaching organic chemistry in small classes: is it effective? Fautch, Jessica M. 2015 Link 13.2 (29.4) Article
Rethinking chemistry: a learning progression on chemical thinking Sevian, Hannah; Talanquer, Vicente 2014 Link 12.83 (22.7) Perspective
Structure and evaluation of flipped chemistry courses: organic & spectroscopy, large and small, first to third year, English and French Flynn, Alison B. 2015 Link 10.6 (20.4) Article
Education for Sustainable Development (ESD) and chemistry education Burmeister, Mareike; Rauch, Franz; Eilks, Ingo 2012 Link 9.38 (26.3) Perspective
*Flipped classroom modules for large enrollment general chemistry courses: a low barrier approach to increase active learning and improve student grades Eichler, Jack F.; Peeples, Junelyn 2016 Link 9 (20) Article
Student attitudes toward flipping the general chemistry classroom Smith, J. Dominic 2013 Link 8.71 (18.9) Article
*How flip teaching supports undergraduate chemistry laboratory learning Teo, Tang Wee; Tan, Kim Chwee Daniel; Yan, Yaw Kai; Teo, Yong Chua; Yeo, Leck Wee 2014 Link 8 (15.8) Article
*What is a hydrogen bond? Resonance covalency in the supramolecular domain Weinhold, Frank; Klein, Roger A. 2014 Link 8 (11) Perspective

Google Scholar shows much higher number of citations, as Google draws citations from a much broader range of sources. In general, while the order of articles may differ slightly, Google and Web of Science match up well, but there are some notable exceptions – James Nyachwaya’s paper Evaluation of chemical representations in physical chemistry textbooks jumps from 152nd in the Web of Science average list to 9th most average citations in Google Scholar and Bette Davidowitz’s paper on student generated micro-diagrams jumps from 72nd to 12th.

Finally, reviews and perspectives are naturally going to attract more citations, so just considering research articles, the top 10 most average citations are below. It seems it was the decade for flipped learning and organic chemistry!

Title Authors Publication Date DOI Total Citations Average per Year
The flipped classroom for teaching organic chemistry in small classes: is it effective? Fautch, Jessica M. 2015 Link 66 13.2
Structure and evaluation of flipped chemistry courses: organic & spectroscopy, large and small, first to third year, English and French Flynn, Alison B. 2015 Link 53 10.6
Flipped classroom modules for large enrollment general chemistry courses: a low barrier approach to increase active learning and improve student grades Eichler, Jack F.; Peeples, Junelyn 2016 Link 36 9
Student attitudes toward flipping the general chemistry classroom Smith, J. Dominic 2013 Link 61 8.71
How flip teaching supports undergraduate chemistry laboratory learning Teo, Tang Wee; Tan, Kim Chwee Daniel; Yan, Yaw Kai; Teo, Yong Chua; Yeo, Leck Wee 2014 Link 48 8
Development and validation of the implicit information from Lewis structures instrument (IILSI): do students connect structures with properties? Cooper, Melanie M.; Underwood, Sonia M.; Hilley, Caleb Z. 2012 Link 51 6.38
A comparative study of traditional, inquiry-based, and research-based laboratory curricula: impacts on understanding of the nature of science Russell, Cianan B.; Weaver, Gabriela C. 2011 Link 54 6
Characterizing illusions of competence in introductory chemistry students Pazicni, Samuel; Bauer, Christopher F. 2014 Link 32 5.33
Students’ interpretations of mechanistic language in organic chemistry before learning reactions Galloway, Kelli R.; Stoyanovich, Carlee; Flynn, Alison B. 2017 Link 16 5.33
Language of mechanisms: exam analysis reveals students’ strengths, strategies, and errors when using the electron-pushing formalism (curved arrows) in new reactions Flynn, Alison B.; Featherstone, Ryan B. 2017 Link 16 5.33