Managing the open-book exam process

At Edinburgh, we are mostly moving to replacing our 3 hour exams with open book exams. We had initially intended these to be within 24 hour timeframes, but the University has mandated 48. Otherwise, things are as described in the previous post.

So students will need to access an exam paper from a  specific “start time” and submit their written answers no later than 48 hours than that start time. Easy!

Exam Process Guide

Having looked through the various options, I am going with the following 5 step plan based on Blackboard Assignments (rather than Turnitin), described below in terms of front of house and behind the scenes:

For students:

  1. Students will be able to download the paper in advance from Blackboard (called “Learn” here), but it is password protected.
  2. At the start time, the password is released on Blackboard and by email. We didn’t want 150 students trying to download a PDF at the same time.
  3. Students complete their answers on paper.
  4. Students scan their answers using Adobe Scan app to create a PDF. For us, one lecturer corrects one exam question, so we want these uploaded on a question by question basis.
  5. Students upload their answers.

I’ve made a draft video (subtitled) outlining this process, which we are testing robustly this week! We are then going to release to students and allow them play with a mock set-up.

Behind the scenes, this means:

  • Setting timings so that Blackboard courses release in line with exam timetable, and components (e.g. password, answer submission areas) release at the right time
  • Enabling anonymous marking so that the student number doesn’t appear in the file name
  • Allowing multiple submissions so that when students upload the wrong file (it will happen) they can submit the correct file – this needs careful management post hoc.
  • After time window has closed, each question will be downloaded and shared with examiners. I am not going to ask my colleagues to annotate files in any way; they will simply keep a mark tally per exam number related to marking scheme so that they can refer back to that if there are queries.
  • Marks can then be returned in an Excel sheet by exam number, and these can go into “the system” for exam boards.

EASY! :) What can go wrong? (no, really…?)

Supporting student study in the “pivot” online

As mentioned in last post, we are focussing our current efforts on two strands – maintaining and promoting academic focus, and being active in student support, the second aspect being led by m’colleague Chris Mowat.

In terms of academic focus, we are moving our closed book exams to open book. Of course this seems “easier”, but I think brings new challenges for students in their study. All that study time spent learning things off doesn’t seem as important now, and we are moving our focus to asking students to think about what questions are asking, showing their understanding concisely in answers, etc. In other words, now that students have a chance to write as much as they want, the challenge instead becomes, what should they do to show they understand? (It’s like, I don’t know, closed-book exams aren’t a good way to assess or something, huh).

Anyway, to help with this I have produced the guide below for students. Hopefully it has some useful prompts in terms of getting focussed and keeping organised, thinking about “pivoting” (ugh) to the new assessment regime, along with Chris’ ongoing guidance re student support. I’ve reproduced the text below, if any of it is of use, please reproduce as you need. CC-by-CoronaVirus 4.0. (With thanks back to the internet for various bits of guidelines that fed into this.)

Supporting Students Study graphic

Text of graphic:

As we have moved quickly to online and remote learning and teaching, you may need to work to establish new study patterns. This will take a little time to get used to, so take your time, and take care of your wellbeing first.
This guide will outline how you can make a plan to adjust your studying and help you regain control of your work.
In this guide we’ll talk about:
» Staying organised
» Adjusting to new assessment protocols
» Connecting with others
» Keeping in touch with the School

You may need to adapt your study habits. Find out what works best for you and establish a regular pattern of work, ensuring you include downtime. This is a marathon, not a sprint!
1. Organise your notes and study
There has been a lot of upheaval in the last weeks of Semester. Your task is to manage your ongoing workload in an organised and coherent way. To do this you can:
make sure you know where all the materials are for each lecture course unit. Live lecture recordings will be in the Lecture Recordings area, with online classrooms in the Course Collaborate channel linked in each course;
make sure you know any revisions to the assessment protocols, so you can plan your study accordingly;
plan and keep track of your study tasks, including scheduling downtime;
note the contacts for each course lecturer;
share your plans with peers in your study group so that you can coordinate your work.
To keep your study focussed and on target, a suggestion is to divide each study day into thirds, with a study session or a scheduled downtime in each third.
In each study session, you can:
focus on reviewing the content of particular topics, drawing on lecture recordings as you need;
work through tutorial materials, discussing with peers in your study group through whatever electronic communication means you have decided on;
test out your understanding with past paper questions;
draw up a list of questions that you wish to discuss with your peers and with your lecturers.
In each session, stay focussed on one topic. Multi-tasking (or micro-tasking) is a very poor learning strategy (only about 2% of the population can multi-task). Work on one topic, wrap it up so that you can return to it easily (clear questions that need follow up), and move on.
In your study sessions, aim to establish a rhythm. Structure the time within each study session, by using, for example, the Pomodoro technique (e.g. a series of three 50 minutes on, 10 minutes break in one study session).

Remember to schedule downtime including full days away from study. Make sure you maintain your usual daily routines of personal care, eating, and social contact by phone or online.

2. Adjusting to new assessment protocols
Until recently, students’ study has been focussed on preparing for closed book exams. The new assessment protocols mean that these exams are now open book. This will mean some changes to your study requirements, but most importantly some additional focus on how you answer questions asked:
The guiding principle for closed-book exams on “making sure you answer the question asked” applies even more in open-book assessments. Students should read questions carefully and make sure that they answer what is being asked. Assessors will be looking to see whether students can concisely answer the question asked. This is the main challenge in open-book assessments.
When studying and reviewing past papers, check that you can identify exactly what is being asked. Remember, exam questions are written so that they can typically be answered in timed conditions, so assessors will look to see quality of your answers addressing the specific chemistry asked, rather than seeing all of their notes reproduced verbatim.
Continue to practice drawing chemical structures, diagrams, and any figures or graphs as you normally would. All of the work you produce for open-book assessments must be in your own hand.

Experience tells us that students often write lots of material, but don’t necessarily answer what is being asked. Make sure your study includes a focus on identifying what is being asked.

3. Connecting with others for study
Much of learning and revision is based around discussions, both among students and between students and staff. Make sure you keep connected as follows:
You should aim to contact lecturers with specific questions arising out of your study, or arising out of revision sessions and discussions with your peers. It is easier for lecturers to address specific questions. Lecturers may also be able to direct you to online reading in the face of library closure.
If you don’t understand a topic generally and don’t know where to start, even after reviewing materials, ask your lecturer for suitable reading. Reading about your topic in a wider context can often help with understanding.
Make sure to form discussion groups. If you are not already in a study group and want to be, the best person to start out with is your lab partner. From there, look to grow it to groups of 4 – 6. It is good when there is a mix of abilities—explaining something to others is very beneficial.
Working in study groups means that you can contact staff as a group with any questions emerging from study.

4. Re-invent the social space online!
One of the challenges over the coming weeks up to the assessment period will be staying focussed during periods of isolation. We’re all missing our lovely Museum, and in place of that, the School are scheduling regular online drop in sessions—these are informal and are intended for social contact as much as addressing any questions among students. Join in these sessions, even if it is only to listen.
All students should make an effort to check in on each other regularly. If someone has been absent from your group meetings or chat, check in with them to see that they are okay. If you aren’t getting responses within a day or two, let your Personal Tutor know.

Keep connected with your class, either as study groups or just social connections. Everyone will need a different level of social contact, but make sure there is someone there for you when you want to disconnect from study. Always remember that staff are available if you need someone to talk to.

5. Stay in touch with the School
If anything is troubling you, or you want to talk about how you are getting on generally, let us know. We can arrange Skype or phone calls at any time. The School’s Personal Tutor system is ready and waiting, and if you have any queries, contact your Personal Tutor, the Senior Personal Tutor (Chris Mowat), the Director of Postgraduate Teaching (Nanna Lilienkampf), or the Director of Teaching (Michael Seery).
All of us are ready to answer and address any queries you have. Even if you don’t have a specific query and just want to reach out, contact us:
Your Personal Tutor
Chris Mowat:
Nanna Lilienkampf:
Michael Seery:
General queries:
There are University resources available including student study resources in the Library and IAD and student support resources. For these, and full details of University guidance regarding COVID-19, see the website:
www.ed.ac.uk/news/covid-19/current-students

Managing and recovering from serious interruption to teaching

The internet has been full of amazing advice over the last few weeks on technical issues relating to moving online. I’ve nothing to add to that, but thought I might offer our perspective from managing the whole process overall. Below are some notes I have sketched out on how we have dealt and are continuing to deal with with events of the last few weeks. We are sharing these with other chemistry departments in UK/IE, and I am publishing here for more general interest. As well as sharing, I am keen to learn. What is missing, what isn’t clear… (what’s wrong…) from the compilation below.

Phase 1: Immediate Disruption – managing teaching alternatives

Aim: to continue the curriculum as much as possible, with clear communication to staff and students as to what options are available, and where resources are. This is done by time release (week by week) of prepopulated online class links and other details on the course homepage of the VLE.

  • Clear decision making process with authority delegated to key stakeholders. One person responsible in this phase for all staff and student communications with a back-up nominated and cc-ed from the outset; staff communications crucial at this stage;
  • Lectures: We mapped our existing timetable online and delivered a combination of:
    • Live lecture via webinar system (e.g. Blackboard Collaborate) – good for staff who have a full slide deck and little board work
    • Lecture recordings (full or in parts) released to students with live lecture discussion via webinar (a form of flipped lecturing) – good where existing recordings are available and staff comfortable with live classrooms
    • Lecture given live and live streamed, with questions via email (assuming building is open) – good for staff who are uncomfortable giving webinar type lecture
  • Tutorials were managed in a similar manner, and staff typically:
    • Go through tutorial answers with discussion in online webchat
    • Release tutorial answers and have discussion in online webchat on any difficulties
    • Release tutorial answers and have discussion by email or discussion board
  • Labs: We were at the end of our lab cycle so finished just short of the finish line. Clearly future contingency will need to consider laboratory alternatives (see Phase 4).
  • Coursework: all activities moved to online submission. Some events, such as presentations, changed to students submitting PowerPoint decks with outline of presentation, or were cancelled.

Phase 2: Preparing for Alternative Examinations

Aim: to introduce an alternative to in-person closed book exams, that matches similar assessment of learning outcomes, allows students to handwrite, can be completed anywhere in the world, being as low tech as possible, and allows students to perform in an equitable manner

  • We have opted to issue our exams as open book assessments.
  • Students will be given paper in advance, password protected, which they can download from a special VLE site for each paper. This reduces internet connection concerns on the day.
  • Students will be asked to agree to a revised Code of Conduct.
  • Students use the password released at the start time of the assessment. Password release by email, on VLE, and possibly by text.
  • Students take photos of their work, and generate a PDF, which can be concatenated (using Adobe Scan mobile app, a solution I am currently exploring – thanks to Ross Galloway for this tip) and uploaded via mobile to VLE, on a question by question basis (use grayscale to reduce size of file). We are exploring a parallel e-mail bin for each exam. Option with advance notice of students sending papers back by registered post.
  • Process will be issued using existing published timetable, assuming no clashes within (exactly) 24 hours of start times.
  • Answer files then distributed to markers in a batch zip file for anonymous marking.
  • Note: at Edinburgh, all non-honours exams were cancelled centrally, so course (module) marks for these are generated by known existing marks – lab and other course marks in our case.

 Phase 3: Get students’ focus back on studying chemistry

Aim: to “normalise” the new arrangements, online teaching, preparation for open book exams, and make sure isolated students’ welfare is closely monitored

  • Return to regular communication with central focus on two streams – one on academic matters and one on student support. At Edinburgh we split this activity across two people (Director of Teaching and Senior Personal Tutor).
  • In academic terms, communications around getting on with study, study advice for different assessment formats, making sure students are pushed back to thinking about tutorials, revisions, thesis writing etc. The tone is regular, even mundane (by design!), just to aim to get students to focus in the current chaos. In addition, we will be releasing a mock exam system so that students can interact with it, get a feel for it and check it out in their situation, for reassurance (on both sides) more than anything else. All academic guidance about new arrangements located on a single webpage, with all VLE course (modules) linking to it. This means students have information centrally as well as in email.
  • In student support terms, we are worried about isolation. We pushed strongly in all email correspondence and Twitter unambiguous messages that we want to stay in contact, and prompting students to contact DoT/SPT/PT for anything; the language here was personal and casual. This is then followed by regular webchats – twice weekly online Q+As where students can ask anything. These have been very popular, and have moved from lots (and lots) of queries about exam paper format to students (and staff) sharing pictures of their dogs. We don’t care, the intention is to just keep a conversation going. We’ve also created a “kudoboard” which will act as an online Yearbook (see: https://www.kudoboard.com/) – students can just post anything they want there.
  • These approaches have been very well received by students. The key message is: you cannot communicate enough, but that communication needs to be well planned, and unambiguous. We have a policy of getting every major email second read, at least, with relevant stakeholders.

Phase 4: Mop up and prepare for future?

Aim: once immediate task is completed, what can we learn? And how do we prepare for a more serious (mid-semester) interruption.

  • We have garnered a huge swell of goodwill from staff who have seen many benefits of lecture recordings, online interaction, and dialogue, and the importance of dialogue in general. We plan to consider how to continue this, even in low tech ways, when teaching returns to normal.
  • The move to open book assessments will open a big opportunity for chemistry to really rethink assessment for 21st century. Lots of opportunities to pursue here.
  • The chemistry community (worldwide) is very underprepared for major interruptions to laboratory teaching, and I think there is scope for significant collaboration in generating datasets and banks of materials for alternatives to lab classes in the event of future interruptions.

 

Looking back on ten years of Chemistry Education Research and Practice

Over the last 10 years from 2010 – 2019, Chemistry Education Research and Practice, a free-to-access journal published by the RSC (of which I am currently Editor) has published 631 articles, which have been cited 5246 times (data from Web of Science). So what has been “hot” this last decade? It seems whatever way you cut it, it was flipped learning and organic chemistry… Below I’ve cut the citation statistics a few ways – these comments are based on citations rather than judgement on the work itself.

In terms of number of citations, Keith Taber’s perspective on the chemistry triplet tops the poll, with 115 citations on Web of Science (Clarivate) and 262 on Google Scholar. In fact the top 4 hits are perspectives or reviews.

Title Authors/ Year Source Total Citations (Google Scholar) Average per Year (Google Scholar Average)  Article Type
Revisiting the chemistry triplet: drawing upon the nature of chemical knowledge and the psychology of learning to inform chemistry education Taber, Keith S. / 2013 Link 115 (262) 16.43 (37.4) Perspective
Rethinking chemistry: a learning progression on chemical thinking Sevian, Hannah; Talanquer, Vicente / 2014 Link 77 (136) 12.83 (22.7) Perspective
Education for Sustainable Development (ESD) and chemistry education Burmeister, Mareike; Rauch, Franz; Eilks, Ingo / 2012 Link 75 (210) 9.38 (26.2) Perspective
Flipped learning in higher education chemistry: emerging trends and potential directions Seery, Michael K. / 2015 Link 67 (145) 13.4 (29) Review
The flipped classroom for teaching organic chemistry in small classes: is it effective? Fautch, Jessica M. /2015 Link 66 (147) 13.2 (29.4) Article
Student attitudes toward flipping the general chemistry classroom Smith, J. Dominic / 2013 Link 61 (132) 8.71 (18.9) Article
A comparative study of traditional, inquiry-based, and research-based laboratory curricula: impacts on understanding of the nature of science Russell, Cianan B.; Weaver, Gabriela C. /2011 Link 54 (97) 6 (10.8) Article
Structure and evaluation of flipped chemistry courses: organic & spectroscopy, large and small, first to third year, English and French Flynn, Alison B. / 2015 Link 53 (102) 10.6 (20.4) Article
Development and validation of the implicit information from Lewis structures instrument (IILSI): do students connect structures with properties? Cooper, Melanie M.; Underwood, Sonia M.; Hilley, Caleb Z. / 2012 Link 51 (85) 6.38 (10.6) Article
Let’s teach how we think instead of what we know Talanquer, Vicente; Pollard, John / 2010 Link 49 (101) 4.9 (10.1) Article

A fairer way of looking at citations is the average number of citations per year. This means that older papers which have had a longer time to accumulate citations are averaged out. (However this is still not fair to recent papers, which will not have had a chance to be cited at all, or will not benefit from a cumulative citation effect). However, on this cut, Weaver’s paper on nature of science in the lab (this is an excellent paper which really should be better known), Cooper’s paper on IILSI, and Talanquer’s paper on teaching how to think drop out of the top 10, and the top 10 based on average citations become (new additions with *):

Title Authors Publication Date DOI Average per Year (Google average)  Article type
Revisiting the chemistry triplet: drawing upon the nature of chemical knowledge and the psychology of learning to inform chemistry education Taber, Keith S. 2013 Link 16.43 (37.4) Perspective
Flipped learning in higher education chemistry: emerging trends and potential directions Seery, Michael K. 2015 Link 13.4 (29) Review
The flipped classroom for teaching organic chemistry in small classes: is it effective? Fautch, Jessica M. 2015 Link 13.2 (29.4) Article
Rethinking chemistry: a learning progression on chemical thinking Sevian, Hannah; Talanquer, Vicente 2014 Link 12.83 (22.7) Perspective
Structure and evaluation of flipped chemistry courses: organic & spectroscopy, large and small, first to third year, English and French Flynn, Alison B. 2015 Link 10.6 (20.4) Article
Education for Sustainable Development (ESD) and chemistry education Burmeister, Mareike; Rauch, Franz; Eilks, Ingo 2012 Link 9.38 (26.3) Perspective
*Flipped classroom modules for large enrollment general chemistry courses: a low barrier approach to increase active learning and improve student grades Eichler, Jack F.; Peeples, Junelyn 2016 Link 9 (20) Article
Student attitudes toward flipping the general chemistry classroom Smith, J. Dominic 2013 Link 8.71 (18.9) Article
*How flip teaching supports undergraduate chemistry laboratory learning Teo, Tang Wee; Tan, Kim Chwee Daniel; Yan, Yaw Kai; Teo, Yong Chua; Yeo, Leck Wee 2014 Link 8 (15.8) Article
*What is a hydrogen bond? Resonance covalency in the supramolecular domain Weinhold, Frank; Klein, Roger A. 2014 Link 8 (11) Perspective

Google Scholar shows much higher number of citations, as Google draws citations from a much broader range of sources. In general, while the order of articles may differ slightly, Google and Web of Science match up well, but there are some notable exceptions – James Nyachwaya’s paper Evaluation of chemical representations in physical chemistry textbooks jumps from 152nd in the Web of Science average list to 9th most average citations in Google Scholar and Bette Davidowitz’s paper on student generated micro-diagrams jumps from 72nd to 12th.

Finally, reviews and perspectives are naturally going to attract more citations, so just considering research articles, the top 10 most average citations are below. It seems it was the decade for flipped learning and organic chemistry!

Title Authors Publication Date DOI Total Citations Average per Year
The flipped classroom for teaching organic chemistry in small classes: is it effective? Fautch, Jessica M. 2015 Link 66 13.2
Structure and evaluation of flipped chemistry courses: organic & spectroscopy, large and small, first to third year, English and French Flynn, Alison B. 2015 Link 53 10.6
Flipped classroom modules for large enrollment general chemistry courses: a low barrier approach to increase active learning and improve student grades Eichler, Jack F.; Peeples, Junelyn 2016 Link 36 9
Student attitudes toward flipping the general chemistry classroom Smith, J. Dominic 2013 Link 61 8.71
How flip teaching supports undergraduate chemistry laboratory learning Teo, Tang Wee; Tan, Kim Chwee Daniel; Yan, Yaw Kai; Teo, Yong Chua; Yeo, Leck Wee 2014 Link 48 8
Development and validation of the implicit information from Lewis structures instrument (IILSI): do students connect structures with properties? Cooper, Melanie M.; Underwood, Sonia M.; Hilley, Caleb Z. 2012 Link 51 6.38
A comparative study of traditional, inquiry-based, and research-based laboratory curricula: impacts on understanding of the nature of science Russell, Cianan B.; Weaver, Gabriela C. 2011 Link 54 6
Characterizing illusions of competence in introductory chemistry students Pazicni, Samuel; Bauer, Christopher F. 2014 Link 32 5.33
Students’ interpretations of mechanistic language in organic chemistry before learning reactions Galloway, Kelli R.; Stoyanovich, Carlee; Flynn, Alison B. 2017 Link 16 5.33
Language of mechanisms: exam analysis reveals students’ strengths, strategies, and errors when using the electron-pushing formalism (curved arrows) in new reactions Flynn, Alison B.; Featherstone, Ryan B. 2017 Link 16 5.33

 

A new book on teaching chemistry in higher education

Cover webThis summer I published a very special book on teaching chemistry in higher education. Each chapter in the book contains some approach on teaching chemistry, written by someone who has implemented that approach more than once in their own setting. Chapters explain how the approaches are grounded in the literature, explain the rationale for the approach, and then go on to give some detail on the implementation and outcomes of the approach. Thus the book intends to be useful to those new or reconsidering approaches to teaching chemistry in higher education, as well as those involved in education development. While the approaches are situated in chemistry, most chapters will be relevant to many other disciplines. The book contains 30 chapters, with 452 pages. There is something for everyone!

Festschrift tweet compilationThe book is dedicated to Professor Tina Overton, and carries the subtitle of a Festschrift in her honour. Festschrift is a German word for a writing celebration in honour of a scholar, and those invited to contribute a chapter wished to celebrate her influence on their career and/or pedagogical approaches they were describing. The book idea came about when I was in Australia visiting Tina and attending an Australian chemistry conference. It was clear from the education strand of the conference that Tina’s influence in Australia was as strong as it had been in the UK and Ireland – no mean feat given she was only there three years at the time. I decided that we needed to celebrate Tina’s contribution to chemistry education in some way, and following her general lead regarding pragmatism, decided that a book describing useful approaches to teaching chemistry would be the best way to do it. Together with my co-editor Claire Mc Donnell, we invited a range of educators from Ireland, UK, and Australia to contribute chapters. The book is available on Amazon by searching for its title (UK and Ireland Amazon here).

Chapter Details

Foreword: Overton, T. L. (2019), “Foreword from Prof Tina Overton, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 1-4.

  1. Seery, M. K. and Mc Donnell, C. (2019), “Introduction to the Festschrift, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 5-8.
  2. Turner, K. L. (2019), “A framework to evaluate the transition to undergraduate studies in chemistry”, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 9-22.
  3. Read, D., Barnes, S. M., Hyde, J., and Wright, J. S. (2019), “Nurturing reflection in science foundation year undergraduate students, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 23-38.
  4. Ryan, B. J. (2019), “Integration of technology in the chemistry classroom and laboratory, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 39-54.
  5. Yuriev, E., Basal, S. and Vo, K. (2019), “Developing problem-solving skills in physical chemistry, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 55-76.
  6. Shallcross, D. E. (2019), “A pre-arrival summer school to solve the maths problem in chemistry, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 77-88.
  7. Lancaster, S. J., Cook, D. F. and Massingberd-Mundy, W. J. (2019), “Peer instruction as a flexible, scalable, active learning approach in higher education, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 89-104.
  8. Lawrie, G., Matthews, K. E. and Gahan, L. (2019), “Collaborative, scenario-based, open-ended, problem-solving tasks in chemistry, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 105-122.
  9. Williams, D. P. (2019), “Context- and problem-based learning in chemistry in higher education, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 123-136.
  10. O’Connor, C. M. (2019), “Approaches to context-based learning in higher education chemistry, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 137-150.
  11. Rowley, N. M. (2019), “Developing inquiring minds through learning chemistry”, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 151-164.
  12. Mistry, N. (2019), “Diagnosing and addressing the issues faced when students learn stereochemistry”, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 165-180.
  13. Fergus, S. (2019), “Using PeerWise to support the transition to higher education, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 181-194.
  14. Gaynor, J. W. (2019), “Student-led interviews to develop employability skills, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 195-208.
  15. Mc Donnell, C. and Murphy, V. L. (2019), “Implementing community engaged learning with chemistry undergraduates, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 209-224.
  16. Essex, J. (2019), “Implementing inquiry-based learning activities in chemistry education, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 225-236.
  17. Sedghi, G. (2019), “A sustainable peer assisted learning model for chemistry undergraduates, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 237-248.
  18. Pask, C. M. and Pugh, S. L. (2019), “Developing business and employability skills for undergraduate chemists, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 249-264.
  19. Haxton, K. J. (2019), “Undergraduate screencast presentations with self-, peer-, and tutor-assessment, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 265-282.
  20. Southam, D. C. and Rohl, B. M. (2019), “Computational thinking in the chemical sciences curriculum, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 283-300.
  21. Slaughter, J. L. and Bianchi, L. (2019), “Student-led research groups for supporting education research projects, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 301-314.
  22. Spagnoli, D., Rummey, C., Man, N. Y. T., Wills, S. S. and Clemons, T. D. (2019), “Designing online pre-laboratory activities for chemistry undergraduate laboratories, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 315-332.
  23. Capel, N. J., Hancock, L. M., Haxton, K. J., Hollamby, M. J., Jones, R. H., Plana, D. and McGarvey, D. J. (2019), “Developing scientific reporting skills of early undergraduate chemistry students, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 333-348.
  24. Seery, M. K., Agustian, H. Y. and Lambert, T. O. (2019), “Teaching and assessing technical competency in the chemistry laboratory, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 349-362.
  25. Ziebell, A., George-Williams, S. R., Danczak, S. M., Ogunde, J. C., Hill, M. A., Fernandez, K., Sarkar, M., Thompson, C. D. and Overton, T. L. (2019), “Overturning a laboratory course to develop 21st century skills, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 363-376.
  26. Thomson, P. I. T., McShannon, L. and Owens, S. (2019), “Introducing elements of inquiry in to undergraduate chemistry laboratories, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 377-390.
  27. Burnham, J. A. J. (2019), “Developing student expertise in scientific inquiry, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 391-404.
  28. Hyde, J. (2019), “Design of a three year laboratory programme for international delivery, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 405-420.
  29. Flaherty, A, Overton, T. L., O’Dwyer, A, Mannix-McNamara, P. and Leahy, J. J. (2019), “Working with chemistry graduate teaching assistants to enhance how they teach in the chemistry laboratory”, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 421-434.
  30. Randles, C. A. (2019), “Developing reflective practice with graduate teaching assistants”, in Seery, M. K. and Mc Donnell, C. (Eds.), Teaching Chemistry in Higher Education: A Festschrift in Honour of Professor Tina Overton, Creathach Press, Dublin, pp. 435-451.

Five suggestions for future VICEPHEC meetings

It is great news that there is going to be a Committee of Elders looking after future VICEPHEC meetings. Here are five suggestions on the structure of this conference:

  1. Structure discussion time – discussion time needs to be structured into the programme. A good rule of thumb is to have at least half of the time allocated to presentations as discussion, so a 10 minute talk should have 5 minutes discussion, a 30 minute keynote should have 15 minutes discussion. Discussions should be structured by Chairs, perhaps with prompting questions (in the case of keynotes) and questions on a theme (in the case of oral presentations). Where possible, allow for some think-pair-share talk before the Q&A and discussion with the group. If the conference is going to be streamed, discussions will be one of the main reasons to come along. Also, there is a pretty healthy backchannel at these conferences, with a bit of thought could be brought into discussions (e.g. up-voting question apps, etc).
  2. Simplify the presentation formats – at the moment we have keynotes, oral presentations, oral bites, and posters. People submit for one (e.g. a presentation) but get allocated another (e.g. a bite). The original idea of bites was that they would be a quick and easy talk about a “good idea” or something that people wanted to quickly share. But bites now are mostly mini talks, where presenters squeeze their 15 minute research talk into 5 minutes. Now that we have posters, let’s get rid of bites. Make all presentations 10 minutes long (+ 5 separate minutes of discussion). Keynotes for a one-and-a-half day conference should be 30 minutes (+ 15 discussion). Chairing needs to be ruthless. The poster session on Friday morning worked great, and perhaps giving poster presenters 60 seconds at the end of Thursday to pitch their poster would help attendees navigate to posters of interest the following morning.
  3. Have a single stream – perhaps most controversial (ooooh); I think the conference would benefit from a single stream. It keeps everybody together; keeps that sense of discussion going, and means the conference as a whole can start to generate some overall headline outcomes in advancing the disciplines, rather than just a series of lots of little things. I think the keynotes and oral bites sessions (given to one audience) show that single streams are beneficial. Parallel workshops do make sense, but given the shortness of the conference, one hour workshops are more appropriate.
  4. Guidelines for presenters – there is a variety in style and quality of presentations and for people new to the discipline, it would be useful to set out what different types of presentations are valuable, along with guidance on structuring these. This is especially important for people transmuting from bench chemistry type conferences into education conferences. This guidance should be available at abstract submission stage.
  5. Bring in external voices – I think it was useful having an external voice this year (someone not directly from chemistry or physics education) and perhaps that should be a continuing feature. What can we learn from leaders in other disciplines, learning and cognitive scientists, career guidance people, industrialists, primary and secondary educators, students themselves…? I’m guessing quite a lot.

A Year of Ups but also Downs

It has been an interesting (academic) year at Seery Towers and our fridge has had plenty of bubbly alcohol flowing through it. Some of the many highs of the year include becoming Director of Teaching, becoming Editor of Chemistry Education Research and Practice, running another MICER, publishing the Overton Festschrift, being elected Chair of the 2023 CERP Gordon Research Conference (wut?), and of course finding out in February (May, of course I mean May!) that I was promoted to Professor. Yay!

Schadenfreude

Academia is generally a place where we are used to talking about success. I think this is a good thing – the work involved in new publications and prizes and promotions (P3), sometimes against the odds of university systems they were achieved in, should be celebrated and lauded.

There is much less talk of associated “downs” – I am reluctant to call them failures as I believe they are part of the successes that we are more public about. Certainly we see more openness about grant/paper rejections on social media, and while that is ephemeral, it does help contribute to a more realistic perspective of the full breadth of academic life. In many ways I wonder if we should have a “Not Successful” part of our official websites – wouldn’t that make the successes all the more special? Beckett said to “fail again, fail better!” – isn’t that academia?

Further to that, even failing is bloody hard work, and maybe we should be honest about times when we are just floating a bit. I’m coming around to an idea that academic life isn’t a marathon, but a mountain hike. There are times when you can give your all, creative juices flowing, outputs both good and bad are in overdrive. But also times when you just want to sit and watch the clouds go by while you eat your cheese sandwich. Coming to terms with the fact that there might be times when easing off the accelerator and just being has helped me deal with the near constant concern of burning out, the stress of feeling that I should be writing those papers in the queue or replying to those emails or doing something – anything – above the normal baseline. But perhaps for periods of time, like a kind of academic circadian rhythm, the baseline is just ok.

Recently I had email correspondence with someone who was looking for help with something. It was something I would have loved to have done, but knew that I wouldn’t have time to do properly. I (eventually) replied, full of apology. The response was that the reader had gained solace in reading that I was too busy, and was happy that I was able to say no because it resonated with their own sense of feeling too busy while wanting to do things. If we only project all of our successful things, will others who just see our shiny websites only see that side of us? The knowledge of our “failures” can put the extent of our success into context, including what those successes truly mean to us.

I’ve been battling with my own failures this year. I’ll mention two. I haven’t had any serious writing time and therefore my output has reduced. This is a “not success” for me because the reason I am an academic is that I love reading and writing (and teaching, naturellement!). Not being able to write means I am not doing the part of the job I enjoy most. I don’t intend this to be a statement reflecting how super busy I am – I could find the time but I have not been successful in protecting it, nor using the time I have properly. Knowing that failure helps me redouble efforts to protect time to write, or just to enjoy a cheese sandwich, and therefore being allowed to indulge just a little more when that final “accept” email does come.

To offset the glorious opening, I will conclude with sharing that I did not receive a National Teaching Fellowship this year. I thought I met the standard, but somehow did not convey that in my application. Colleagues have kindly played the Victor Meldrew role for me, but I am ok about it (hearing in Tuscany en route to a glass of prosecco meant I was quickly consoled). But if I ever were to receive it in future, wouldn’t everyone know that it meant more because I didn’t get it first time around? And I guess that is the point.

Building evidence for teaching-focussed promotions

Almost all academic promotion criteria will list teaching activities as one of the core areas where candidates will need to demonstrate evidence, but there is a growing population of teaching-focussed/teaching-only/teaching and scholarship/teaching fellow staff where this criterion will obviously be much more important. Having been on a working group on this matter in my own university, been asked to write quite a lot of references for people in this population, and thinking about my own career, this topic has been close to my heart for the last few years, and I plan here to share some observations that might help others focus their work on generating different types of evidence for promotion.

One major caveat to all of this is that the whole area of teaching focussed promotions is disruptive to the academic norm, and therefore impossible to generalise. Therefore what I say below in no way means to get away from the fact the topic of teaching focussed promotions will likely depend on institutional policy, on particular local circumstances, and on a general unspoken matter of who the local gatekeeper is. A head of school who has notions about what is or isn’t The Path to Promotion will likely be undeterred from The Path by any policy or guidance. All I can say is that I think and hope things are improving generally.

Terminology

When talking about promotions, the terminology becomes complicated, so I have set out the kinds of terms used that I am familiar with and show how these align with standard grades we use here at Edinburgh, which because is out of 10 means it is easy. This alignment isn’t exact, but I can explain my reasoning if anyone wants.

Grade /10 Teaching Fellow Categories Usual UK Categories US-influenced UK Categories
7 Teaching Fellow
8 Senior Teaching Fellow Lecturer Assistant Professor
9 (Senior Teaching Fellow -> Senior Lecturer) Senior Lecturer Associate Professor
Platform 9 ¾ Reader  

Professor

10 Professor

Headlines

Some key things to remember about teaching focussed promotions are:

  1. As you move up the scale, the evidence necessary moves further from the chalkboard. This is a sad reality, but ultimately just being a really great teacher, year in and year out, is unlikely to be enough to make a case beyond grades 7 & 8.
  2. What changes is increasing leadership in teaching, certainly at grades 9 & 10, so that beyond the baseline of doing a really good job in your own teaching, you influence the teaching of others. This aligns well generally with the notion that academic promotions are associated with increasing leadership in the field.
  3. Being a busy bee is less valuable than being a strategically busy bee. Aligning teaching and leadership activities to your school and university strategies, or problems that emerge on an ongoing basis, or demonstrating something new that offers an advantage for your School in your university or compared to other departments in your discipline, means it will more likely get noticed. Doing things because you think they should be done but don’t resonate with anyone else (read: anyone involved in making decisions) will not carry much weight. I’ve a whole cupboard full of stuff that I have done but no one cares about. It’s personally satisfying, but won’t add to a promotion case.
  4. As well as recognising the emerging role of leadership, the second main thing is recognising the importance of evidence of your activity and its impact. Ultimately a case is going to be: (1) here is the sad state of affairs; (2) here is what I did; (3) this is how it is better and this is how I know. I’m going to dwell on the last point below. If you do something, and no-one knows what happened as a result of it, it might as well not have happened for the purposes of promotion.
  5. Finally, get used to saying “I”. It is your promotion case, so the evaluators will look to see what you did beyond anyone else. So make it clear what you did. This means not being humble. “We redeveloped the lab course” – did “we”? Or was it you who stayed on working through everything to get it finished. Similarly with committee work; are you on a university or Society committee? If so what did you do that made a difference on that committee? Many people are on committees so they can say they are on committees, and any decent evaluation would look to see what the individual’s actions were. While things are naturally collaborative, make it clear what it was that you did that made a difference.

Some suggestions follow…

A: Activity within your school

More than just a safe pair of hands

Teaching focussed people are generally busy with teaching, and it can be hard to move beyond just dealing with the freight train of workload each year. One approach is to identify what the hot topics in your School are. What is coming up in NSS? Are students moaning about labs (yes, they always are)? What kinds of issues emerge at Staff Student meetings? What emerged as things that needed some firefighting? These kinds of things tend to get on the radar of heads of school and similar, and they can be useful places to start building a narrative: “Here is something that was not working well” (make sure that is logged in the minutes of any meeting if you intend to approach it). It may be that the teaching work you are doing could, with some tweaks, help to address the issue you identify. If students are moaning about labs, and you are involved in labs, making some change that will likely have some impact. The narrative extends to: “this is what I did”. Then you need some way of capturing impact – and this can be tricky, as students are less vociferous when things are going super. I’ve found that students talking across different year groups helped surface changes (“that was awful”… “no it’s actually ok now” – GET THAT LITTLE BEAUTY MINUTED!). Sometimes if you’re lucky it might emerge in course questionnaires. Or sometimes someone senior in the department might say something about how things have really improved (MINUTE IT!). Keep emails with anything indicating improvement in a special folder. And so the narrative concludes with: “it’s better now, and here is how I know”. The point is that if you do something but don’t think of the narrative, it is hard to retrospectively include that narrative in promotion paperwork.

Introducing change

As well as improving things to address known issues, improving things just to make things better is probably a better form of leadership. If you see that staff are killing themselves all working on some piece of busywork assessment, and you can think of a better way and introduce it, this is likely to make those who are concerned about workload happy. This can be tough, and usually involves a bit of ongoing communication to assuage concerns, but if you can make a case of over-assessment or considering learning outcomes, it helps with your narrative. Or perhaps you developed a new course in response to a particular need identified by external examiners or course review. Again, documenting impact or outcomes is important. Usually this kind of activity can be incorporated with being on the School T&L Committee, as that is where activities beyond your immediate control of your own courses can be surfaced. I think it is worth noting here that this likely extends to changes not directly relating to teaching and learning, but the broader student environment. Have you worked on student connections, departmental culture, career developments, widening participation, and so on…

For higher grades of 9&10, it is likely that this change is going to need to be more widespread. What can you say you have done that impacted teaching approaches in the whole school, or university, or your discipline more broadly.

Teaching innovation

While I do worry about students being experimental laboratories for people thinking about promotions so that they can show they are “innovative”, there is a general sense that someone looking for a teaching focussed promotion should as a baseline be a good teacher themselves. Course evaluation questionnaires can be useful evidence but are known to be flawed; so look to comments from senior colleagues, or classroom observations, or emails/comments from particular students. If you are looking for some innovative ideas, buy this book

There are some other forms of evidence. One way to highlight teaching innovation and excellence is to win an award for it, and so teaching awards have become important career milestones in recent years, which I personally think is a real pity. Another way is to document any invited talks given on teaching approaches, with invitation being more important as you move up the grades. And if you are invited to give one talk, contact some other universities nearby and let them know you are in town. Invited talks are something senior academics understand. Finally, perhaps you can be part of (grades 7 and 8) or lead (grades 9 and 10) applications to teaching grants in your university for teaching developments. Working with or supervising interns or colleagues on these can make for useful narratives.

Reporting what you did

As well as introducing change or doing things well, reporting what you did in an education publication can be a useful piece of evidence. It has the advantage of looking and feeling like something people unfamiliar with this world know about (academic publication), is a formal piece of evidence that you did some good (it is peer reviewed), and is a contribution to the wider community. There is a saying that an education publication is an education publication, and lord knows there are some shockers out there. I think this is probably still true, but I think that evaluation panels will become increasingly a little more savvy about the type of publication it is, and where it is published.

A really sad thing about not publishing is that everyone benefits from your activity apart from you – the students and your school benefit from your work; the publication would be an important piece of evidence for you and your promotion. Reporting what you did can extend to beyond formal publishing – blogging is a way to get your name out there as Someone Who Does Things. I still remember the extreme joy I felt when a former line manager referred to my blog as a potential output. Remembering the word “blog” emit from a chemical physicist still makes me happy.

B: Wider influence

As you move up the grades, it is likely that wider influence will become more important to demonstrate, so that it moves roughly from Grade 8 being influential at school/university, Grade 9 at university/outside university, and Grade 10 at outside university/internationally. Usual caveats apply. Things become much more nebulous here, because it will depend on particular activities, and the same principles apply – look for narratives that include evidence addressing things of strategic importance, or things that senior academics will understand and can relate to.

Accreditation

Accreditation with professional bodies is a way to demonstrate professional recognition in a field. While the RSC’s CChem is not quite understood in academia, Fellowship is much more so. Being an FRSC gives some reassurance that you are “in the club”. I have to say publicly that I really battled to get my own FRSC. I was turned down twice; once many years ago, perhaps unsurprisingly, and once about two years ago, which was a genuine shock. After adding a supplement to my application to show I really was, like super amazing, the imperial thumb turned upwards. I mention this only to highlight that accreditation, much like promotion, is still stifled by the “be in my mould” mindset.

As well as professional society accreditation, there is the Advance HE accreditation. This is becoming more the norm, because many professional development courses in the UK align with it, so you just “get it”, but going for Senior or Principal Fellow would be a good way to demonstrate that there is more widespread recognition of you as an educator. In honesty I am not sure senior people really know what P/S/FHEA really means, but they likely know it is “a thing”.

Professional activities

Just as committee work in university is a way to document influence, committee work on external bodies is also a way to document wider influence. The important thing will be to demonstrate what it is you did, and how that went down in the community of the profession. You need to distinguish yourself from The Committee Careerist, who is just on the committee to say they are on it. This can include organising events, or contributing to community in some way, been seen as a leader or champion for particular activities. Be clear with your narrative – what was the demand/reason for some activity, what did you do, and how did it go. Number of attendees/feedback are useful pieces of evidence. Sometimes very nice people write to you with testimonials. (Be nice: write to people when they have done something that has been valuable to you. It is important evidence and shows impact beyond university.)

I think that’s enough for now. What have I left out? What could be clearer or what is plain wrong…

Board-Game-Snake-and-Ladders-Periodic-Table

A New Role

Such is the pace of life at the moment that major life events (well, major in my life anyway) pass by undocumented. In January I became Editor of the journal Chemistry Education Research and Practice, or as I like to call it “Chemistry Education Research and Practice – Free to Access“. CERP is the Royal Society of Chemistry’s education journal.

I love CERP. I’m not just saying this now – here I am writing in 2011 about it. The fact that a learned Society such as the RSC gives its support to the journal speaks volumes about the high value the Society places on education, much more than strategy documents or long monologic committee meetings might do. The RSC through its Education Division supports CERP so that it is free to access. CERP and its predecessor U. Chem. Ed. has given voice to generations of chemists who want to say: education is important – here is why. Many of those chemists are the leading lights of chemistry education past and present.

The first editorial

One of the first tasks in the new role was to consider where we take the journal. I’m very firmly in it for this sense of voice. Work with MICER and elsewhere has demonstrated a swell of interest in CER, and CERP is a natural home for such work. In our first editorial we did a few things. The first is that it is a joint editorial, written by me and the three Associate Editors; Ajda Kahveci, Scott Lewis, and our newest AE, Gwen Lawrie. As CERP grows, so do the layers and range of expertise needed to edit it. I felt a joint editorial would be a useful way to express this, and set out the interests of all editors.

Secondly, we talked through the typical considerations necessary for a journal article in CERP. Many of these are obvious to experts in the area, but if this is unfamiliar, we intended to demystify what actually makes for an education paper, especially to those coming from a scientific background. We included a generic article structure, along with the kinds of things that should be included along the way. A lot of this was drawn from our experience as editors, and we hope that this headline guidance will be useful to authors. The table from the editorial is shown below.

Typical structure for research article in CERP and guidance for what should be included in each section
Typical structure for research article in CERP and guidance for what should be included in each section

Finally, we talked about what actually happens when a manuscript is submitted. Who looks at it; how do we select reviewers; how do we evaluate their commentary? The purpose again here is to make it clearer about the journey a manuscript goes on, and where our processes might differ from other journals. A particular value of the culture at CERP is the role the editors play in offering commentary on the manuscript that draws together main reviewing comments. Authors regularly comment on the value of reviewer and editor commentary and how it can be helpful in improving their work.

CERP will always be evolving and I look forward to conversations and debates about its continuing journey. The editorial team interests covers the full range from research to practice, from university to school. We are of the field, and in the field, and want to hear from contributors and readers about how we can continue to develop and grow this fantastic journal.

 

What can we advise chemistry students about studying?

I’ve started a new blog over on University of Edinburgh’s new blog website – these are posts specifically focussed on chemistry education, looking at important and I hope interesting things from staff/student perspectives. The first one is: What can we advise chemistry students about studying? I’ve pasted the opening paragraph below.

The importance of actively considering study

A quick glance at the specifications for any university lecture course will show that while lectures, labs, and tutorials will make up the formal part of how we interact with students, usually more than half of the time allocated to courses is given over to independent study. This time is crucial for students to be able to work with the materials of the course, both for the purpose of understanding the material as part of their overall journey to becoming a professional chemist, as well as needing to demonstrate their knowledge and understanding in an assessment. Indeed, while study often has negative connotations – a chore, or something negative because of its association with grading – it is ultimately a pleasant experience. We choose to study because we enjoy indulging in a topic and finding out more about it.

How do students study? How should students study? While there is universal agreement on the need for study, we are perhaps less clear in higher education about what students do, and indeed what we should recommend to them. In this post I will review some of the literature on studying, and draw together some recommendations based on theories of learning, and research done in chemistry education and elsewhere… [continue reading on the ChemDoT blog]

A Framework for Learning in the Chemistry Laboratory

What is the key literature on chemistry laboratory education? What kinds of factors should be considered when designing laboratory curricula? An invite for a journal special issue gave me the final push to write something I’ve wanted to write for a long time addressing these questions. When writing it, I have in mind “typical academics”, who may be doing learning and teaching courses or people interested in broadening their reading about chemistry education. This special issue was a good place for it because it is a special issue in a “normal” chemistry journal, with the theme of chemistry education. Therefore the expected audience of the issue is the general body chemistry faculty. It was too good a chance to miss!

The article is now published. We set out our stall early with some guiding principles that we adhere to:

  1. The overarching purpose of laboratory learning is to teach learners how to ‘do’ science.
  2. Preparing students for learning in the laboratory is beneficial.
  3. Explicit consideration needs to be given to teaching experimental techniques.
  4. Consideration of learners’ emotions, motivations, and expectations is imperative in laboratory settings.

While I think this is helpful (obvs), it does expose the difficulty with considering changes to our laboratory curricula – there is a lot to think about! (We give a lot of things to think about in Table 1 of the paper). And after thinking, it takes a lot of work to implement change. I really think this is why laboratory curricula in chemistry are so resistant to change.

After explaining the four principles listed above with key references, we propose a framework, shown in the figure, which in our case at Edinburgh is oriented towards developing independence, skills, and capability to experiment (as in really experiment). Our way of doing this is to consider the progressive development of skills and competencies over the curriculum, and how each stage builds on the previous one.

framework for learning in the chemistry laboratory

We talk about each stage, with some suggestions. Initially I was reluctant to do this and indeed our original submission did not have these examples. There is so much in the literature that can be described as “good ideas in the lab” and I wanted to focus minds on prompting people to think about a model for their curriculum and not individual practicals they could add to their course without thinking about an overall framework. The reviewers and special issue editors came back with requests to add in some examples. I confess I first resisted – I really wanted to emphasise a curricular perspective. But the editors (patiently) argued that by showing some examples, we could illustrate how people might take existing published experiments and consider how to fit them into their pre-determined framework. I’m glad I was convinced that I was wrong and I think the paper is stronger because of it.

If you would like to read the paper in its full glory, head on over to:

Seery, M.K., Agustian, H.Y. and Zhang, X., 2018. A Framework for Learning in the Chemistry Laboratory. https://onlinelibrary.wiley.com/doi/abs/10.1002/ijch.201800093

Of course I am happy to provide pre-prints to anyone who does not have access.

Timeline entry page on VLE

VLEs are choccaful of information and we are doing a lot of thinking about how to make it as easy as possible to get to the relevant information. Previously our model has mimicked essentially what a computer folder on your desktop might look like – logically arranged but you sort of need to know the structure first before you can find anything. A nice idea I picked up while externing at MMU was a kind of “what’s new this week” timeline that is the entry page for a course. The model we’ve gone with is below.

timeline VLE

As each new week approaches, a new tab is added (scheduled by date release). These link to lecture information, labs and tutorial work etc and relevant text book reading that is the focus of that particular week. The aim is that rather than having to navigate through, the things most necessary for that week are in one place.  Linking to things is a bit clunky because, …Blackboard, but it seems to work ok. The reading links to reading list application which connects to library and has gotten good feedback from elsewhere.

This is fine for week-by-week info, but of course when students come to study, they are likely to want all topic information in one place, so the left hand menu links to topics. A small problem is that this left hand menu doesn’t auto-show when you are on mobile, you need to click it out, but hopefully this isn’t too terrible as the links to course topics shown in the weekly tabs will still bring people to the topic folder.

While scouting around how best to do this, I noticed very little online in terms of structuring VLEs, so welcome any thoughts/links to how other people do it.

Promoting teaching focussed academics

I read with interest this series of blog posts on promotion in academia, discussing external promotions (having to move beyond your institution to get promoted), internal promotions, and using the former to achieve the latter.

image

There is an additional layer of considerations for promotion of teaching focussed academics working in disciplinary departments (as opposed to education departments). The first is whether teaching focussed academics should be promoted on the basis of their work in teaching. The argument against is that if research is the traditional metric, then one who does not do research should not be promoted, and certainly not to professorial rank. This is still a prevalent view, although one that is changing somewhat. In the UK, there are not many Professors of Chemistry Education for example; I can think of a handful. In the last year there has been quite a few examples of people being promoted to Associate Professor (Reader) level. But it is still early days.

One of the difficulties is even if an institution wishes to promote someone to senior level, what criteria do they use? Typical criteria of funding and to a certain extent publication record are more difficult to apply. A general finger in the wind idea is whether the candidate has an international reputation, which seems sensible enough, but then I might say that. In a world of professional social media, a reputation and a reputation online are becoming confused. So instead, institutions might look for candidates to be Senior or Principal Fellow of the HEA, as an externally judged metric of reputation. This requires quite a commitment on the part of the applicant, and reflects back on what is the promotable action – teaching quality itself, or impact of that on others within and beyond the institution. For professorship, my own institution seeks things like external awards, student nominated awards, contribution to university policy, development of a MOOC (what?!), authorship of influential textbook, author of publications, invites to major conferences, or PFHEA. Perhaps I am blinkered, but one feels that if this level of criteria was equivalent across the board for all promotions, we would have very few professors generally.

One of the ironies of teaching focussed promotions is that it somewhat focusses on shiny new things that appear above the baseline of just doing the regular teaching grind. Things have to be excellent and innovative and while in a way that is good to encourage creativity, I do fear a little for students who are exposed to some crazier ideas so that the (ir)responsible academic can write about their snap-bang-whizz in appropriate promotion documentation.

A second concern is that publish-and-be-damned is even more enticing for education focussed academics looking for evidence. Promotion panels are unlikely to know the difference between reputable education publications with some merit in the field and The Secret Diary of Chemistry Education, promising all the latest news in the field. An education publication is an education publication, and to a passing eye, might appear so. One would hope that at least in this instance, an external reviewer might comment on that.

Finally, there is the issue of external reviewers. If someone is going for promotion on the basis of their teaching, and internally the university finds it difficult to judge quality, how can an external person do so? It will come back to perception, and perceptions are going to be influenced by bias. Over the last few years, I have been asked to write review letters for a whole variety of teaching focussed academic promotions from junior to professorial levels. One thing is very common – the difficulty in moving beyond perception and basing references on something tangible – there is often a lack of tangible evidence, often for the reasons above. So while the horizon isn’t clear and the pathway isn’t quite mapped out yet, recording and documenting evidence will be useful to support applications when the clouds clear. I’m open to suggestions as to what that might entail…

Reflecting on #MICER18

This week I ran the third Methods in Chemistry Education Research meeting (MICER18). It was a really interesting and useful day – we had a good range of speakers and lots of discussion; certainly the scope of the meeting this year was the most ambitious so far. As the meeting is beginning to settle into a space on the calendar, I’ve been thinking a lot about how to take it forward.

Micer Timeline

For me MICER operates on three levels. At one level this meeting has a very simple purpose – to share approaches that can be used when doing educational research, and especially applied to chemistry education research. This is achieved by asking speakers to give talks on how do undertake particular approaches, accompanied by activities and discussion – effectively a series of workshops. Over the series (2016, 2017, 2018) we have covered things like doing interviews, thematic analysis, using Likert scales, designing questionnaires, statistical approaches, thinking about theoretical frameworks and ethics, and the holy grail of writing a decent research question. The purpose is to give insight into the language and processes around educational research for the audience of people with a scientific background, who are moving towards the light. The emphasis on sharing methods of how things were done, rather than what happens as a result of doing things means that this meeting can eek out a little bit of light beyond the shadow of the annual Variety in Chemistry Education meeting. 

A second level is about identity. Most people working as a discipline based researcher in the U.K. and Ireland will likely be doing this as a kind of part-time hobby, in the few gaps available when doing a full time teaching position. Lack of funding means that the discipline is amateur; people are doing things with not much time and less money. If we are to professionalise, people need to feel confident in saying that they are a “chemistry education researcher”, loud and proud. This is very difficult to do if you don’t feel professionally grounded in that discipline. To gain that confidence, there needs to be a community into which they can situate themselves, and a sense of personal expertise to allow them to make the claim. By sharing methods and approaches, and demonstrating that there is a community, the meeting aims to help raise this confidence. This year we included a “reports from practice” section; hearing from people who are just like us doing CER in their own situation. It was wonderful, and a real highlight of the day, a kind of showcasing of what real live chemistry education researchers look like. I was also struck this year by the number of people in the room who spoke about projects they are working on, or where they were situating themselves on the spectrum of evaluation, reflection, and research. This highlights to me that the landscape is shifting slowly. But there needs to be considerable support; financial yes, but also in terms of identity. This is something the RSC needs to grapple with firmly. 

Finally, the entire education landscape is shifting. The growing emphasis on teaching at third level means that those situated in a teaching and scholarship roles are thrust into a political ping pong. At school level there is an expectation that teachers will engage with education research with little support or guidance, save for some grassroots heroes. We aimed to address this head on this year with a keynote talk about this very landscape; one which I think well characterised it and also offered clues as to how we might navigate it. We also included a talk on managing student projects; standing firm in a world where in some institutions, the presence of such projects is contentious. 

MICER19

In setting out on the MICER journey, I was only ever really concerned with (and indeed thought about) the first level. I knew from conversations that people wanted to know about the how of doing education research. The additional levels have grown, partly thrust upon us as a nascent community, partly necessary for us to be a community. But there is a danger that in aiming to do everything, the meeting tries to do too much, and as a result, does not achieve the sum of its parts. It is only a one-day meeting after all, and I am left wondering whether we should refocus our thoughts on the first part, and the others will work themselves out elsewhere (come on RSC!). 

I’m planning to send out feedback survey and hope to use that to guide the focus of future meetings. There are other less lofty considerations; the meeting was full by February and likely needs a bigger venue. The characteristics of attendees is broadening. While the registration cost is cheap, getting to London is expensive. Getting funding to support the meeting is getting more difficult; there is a limit to the hit that the supporting interest group budgets can manage.  

But the interest in, and outcomes of, the meeting mean that I think the effort will be worth the while. 

 

A memorable teaching scenario for #Chemedcarnival

Katherine Haxton has challenged us to write a blog post on a memorable teaching situation.

When I was 19 I attended a Scout training weekend as part of a course to become a canoe instructor. I had been canoeing for several years under the patient instruction of canoe instructor, Keith, who was also a former scout leader. Keith is tall and patient and a scientist and has a clipped English accent with excellent projection. “Lean downstream, Mick” he’d boom up and down rivers, while I’d lean upstream, and capsize.

All this made Keith very exotic. Coming from a small country village where everyone mumbled and there were no scientists, much less well-spoken English ones, he was very different. We adored him. He had rare qualities of being The Adult but never condescending, paternal but never patronising. We mimicked him constantly (out of earshot) but woe betide anyone who even hinted a bad word about him.

So the country fellows went to training course to learn how to be leaders. The teachers on the course seemed tough and scary and, well, from Dublin, which is to say they were under the influence most of the time. When we arrived we found out that Keith was going to give one of the sessions on this weekend of the course. This is more than half a life ago, but I recall the excitement that news brought. We were used to him in the context of our own canoeing, going down (and upside-down) the rivers of Wicklow, but now we would see him somewhere different. We knew him and these chaps from Dublin didn’t. He was ours and we were loaning him to the mob and he would be amazing.

He was amazing. The session was about safety, and the kinds of decisions that need to be made quickly when on rivers. He was clear and authoritative and we sat and listened in total silence. Bursting with pride. Everything he said made sense.

But then; one of the Dublin fellows shouted out in the silence: “But Keith – that’s WRONG!” You can guess our horror. Keith listened, and responded, and moved on. And then; another interruption!

A game was afoot. Even for simple country chaps, it was clear that the whole lesson had been structured, with planned interruptions prepared well in advance. Tension eased, we all played along, throwing out ideas and suggestions and discussing various scenarios and decisions.

It is a “teaching moment” that has always stuck with me. There aren’t many lessons from half a life ago that I remember so well. A few months ago I was in the newly refurbished National Gallery of Ireland, and wandered into a wing featuring some new artists. I came across this portrait of our hero. The link explains more.

ROBINSON, N-770

Lessons from running webinars

We are now coming up to half way for the webinar series I launched this year. Webinars run monthly, thereabouts, and are on the theme of chemistry education research. I’ve never hosted webinars before so it has been interesting, and when the technology decides not to work, heart-stopping. Useful responses to a post (plea) requesting ideas/guidance are listed here. I think I have incorporated most of the suggestions.

CERGinar 2017 - 2018 Series

Some thoughts on format

What’s been a real pleasure has been the opportunity to hear speakers I love give a talk. This year, because I was testing the water, I chose speakers who I have heard and who I know will do a good job, and somewhat selfishly that I want to hear again. This led to a list of 42 names scrawled on my office noticeboard, and picking just a few of these was really tough.

Alison Flynn set us off to a stellar talk with a talk that ran the spectrum from methods of doing the research right through to implementation in teaching. This was really popular and meant that it addressed the difficulty of the breadth of audience types. Keith Taber made us think more about methodologies… are experimental approaches appropriate, and what are their limitations? Nathaniel Grove picked up on the format set by Alison, again looking at methods and then looking at implications, and this seems to be a formula that works. In both cases, this meant that a natural break in proceedings was a chance to have a mid-presentation set of questions. And that echoes something I have learned from MICER: people love to discuss. Opportunities for discussion compete with wanting to squeeze as much out of the speakers as possible, and the balance is fine tuned. For an hour slot, thought, 45/15 seems to work out. Nathan’s talk included the guest chair Niki Kaiser; this was really useful as it meant I could focus on technical matters, Niki asked questions, and it also means the whole thing is less “my” webinar series, but one of the community.

How to choose speakers?

As well as the criterion (this time around) of having seen all the speakers present, there was the difficulty of choosing just a few from my list of favourites. Donald Wink is the next speaker in the series. He gave a talk at Gordon CERP last year, which was stellar, probably the best talk I heard in a year of many conferences. It was one of those talks where you stop taking notes and just listen to try to absorb as much as possible. His clarity on discussing case studies is one that I think deserves a very wide audience. Then, we have Nicole Graulich, who won best poster at Gordon CERP, meaning she got to give a short talk at the end of the conference. I was left wanting to hear much more. Ginger is doing some amazing work around students writing, and Vicente… well we all want to hear Vicente. Both of these are again Gordon speakers. I thought that this range of speakers represented some well established figures, some newer to a wider audience, different aspects of chemistry, and a balance of gender. But I’m sure I can choose another set that will fulfill those criteria.

On and on?

Chemistry education research, as a young discipline in the UK, has two difficulties as I see it. One: there is no money. And two: as there is no money, people do a lot of this work in their spare time or squeezed into a very busy day job. That means that things like this tend to get squeezed, and it becomes difficult for people to attend. The purpose of these webinars was to act as a proxy for the academic seminars our colleagues will be used to in chemistry departments, except focussed on education.  I have to say I thought that attendance (because of point 2) would be very low, but it has been way above expectations, with lots of discussion in the chat area.

I’d be interested in hearing from people as to whether we should continue with a new series in the Autumn, and proposed ideas for format/speakers. In the mean time, do register for Prof Donald Wink’s seminar, 21st Feb. You won’t be disappointed.

 

The Likert Debate

David Read blew my cover on Sunday night with a tweet mentioning my averaging of Likert data in our recent work on badges. If there is ever a way to get educational researchers to look up from their sherry on a Sunday evening, this is it.

Averaging Likert scales is fraught with problems. The main issue is that Likert response is ordinal, meaning that when you reply to a rating by selecting 1, 2, 3, 4, 5 – these are labels. Their appearance as numbers doesn’t make them numbers, and Likevangels note correctly that unlike the numbers 1, 2, 3… the spaces between the labels 1, 2, 3… do not have to be the same. In other words, if I ask you to rate how much you enjoyed the new season of Endeavour and gave you options 1, 2, 3, 4, 5 where 1 is not at all and 5 is very much so, you might choose 4 but that might be just because it was while it was near perfect TV, it wasn’t quite, there were a few things that bothered you (including that new Fancy chap), so you are holding back from a 5. If you could, you might say 4.9…

But someone else might say well it was just better than average, but only just mind. That new fella wasn’t a great actor but, hell, it is Endeavour, so you put 4 but really if you could you would say 3.5.

So both respondents are choosing 4, but the range of thought represented by that 4 is quite broad.

I can’t dispute this argument, but my own feeling on the matter is that this is a problem with Likert scales rather than a problem with their subsequent analysis. Totting up all of the responses in each category, we would still get two responses in the ‘4’ column, and those two responses would still represent quite a broad range of sentiments. Also, while I understand the ordinal argument, I do feel, that on balance, when respondents are asked to select between 1 and 5, there is an implied scale incorporated. One could of course emphasise the scale by adding in more points, but how many would be needed before the ordinal issue dissipates? A scale of 1 to 10? 1 – 100? Of course you could be very clever by doing what Bretz does with the Meaningful Learning in the Lab questionnaire and ask students to use a sliding scale which returns a number (Qualtrics allows for this more unusual question type). Regardless, it is still a rating influenced by the perception of the respondent.

Our badges paper tried to avoid being led by data by first exploring how the responses shifted in a pre-post questionnaire, so as to get some “sense” of the changes qualitatively. We saw a large decrease in 1s and 2s, and a large increase in 4s and 5s. Perhaps it is enough to say that; we followed the lead of Towns, whose methodology we based our own on,  in performing a pre-post comparison with a t-test. But like any statistic, the devil is in the detail, the statistic is just the summary. Conan Doyle wrote that “You can, never foretell what any one man will do, but you can say with precision what an average number will be up to. Individuals vary, but percentages remain constant. So says the statistician.

There is a bigger problem with Likert scales. They are just so darned easy to ask. It’s easy to dream up lots of things we want to know and stick it in a Likert question. Did you enjoy Endeavour? Average response: 4.2. It’s easy. But what does it tell us? It depends on what the respondent’s perception of enjoy is. Recently I’ve been testing out word associations rather than Likert. I want to know how student feel at a particular moment in laboratory work. Rather than asking them how nervous they feel or how confident they feel, I ask them to choose a word from a selection of a range of possible feelings. It’s not ideal, but it’s a move away from a survey of a long list of Likert questions.

When is a conference real?

Respected Dr Dr Mrs Seery, we hope that you can come to our conference in somewhere you’ve never heard of and tell us about your interesting and exciting work in Pre-Lecture Resources for Reducing Cognitive Load at our Conference on Chemistry and Chemical Engineering in Sub-Oceanic Conditions. Please reply.

Most of us now receive daily invites to conferences around the world – oh the travel we could do! – and the usual fare is a greeting like that above; a dodgy mail merge of incorrect title, a paper title you have published and a conference that has a theme that bears no resemblance to the topic that you have been on. But the targeting is getting cleverer, and there are now quite a few Chemistry Education “conferences” doing the rounds.

These conferences are organised to make money. The model is that academics are invited to speak at conferences, and they, like all attendees will pay to attend.  The organisers know nothing about the topic, and the conference will not have any coherent theme, but the organisers will have delivered on their promise to host a conference, and gather all the money raised in the process as profit. Academics will provide free labour by presenting at the conference, perhaps peer-reviewing, being members of the “Advisory Board”… It all mirrors an actual conference very closely, but of course the problem is that the themes of these “conferences” are so broad that little meaningful discussion could take place. So how do you know what is real or not?

Three key places to look are: (1) who is on the advisory board (2) is there a professional body underpinning the conference, and (3) what are the conference themes.

ICCE2018Organisers and Advisory Board

If you are going to a conference on chemistry education, and the advisory board is populated by Professor of Forestry from Transylvania, then an alarm bell should ring. Are the names familiar? If you were to Google some of them, would you come up with some CER publications? Pictured are the Local Organising Committee and International Advisory Board of the very real 25th ICCE 2018 conference happening in Sydney in 2018. A cursory glance at this list for anyone involved in chemistry education would show that these are people with a genuine investment in the discipline.

Sadly, this check on authenticity is becoming more difficult, as academics are bounded by a singular characteristic: we love doing things for free. So when you get an email that asks you if you want to be part of an organising committee for a conference whose title interests you, well why not? If you don’t look into it too much and you’ve always wanted to go to the Mongolian mountains for a hike, this fits the bill. Before you know it you are profiled on the conference website and credence is added to the meeting because of your affiliation.

Professional Body

A second thing to check is if there is a a professional body underpinning the conference. The very real 25th ICCE 2018 conference happening in Sydney in 2018 is being organised under the auspices of IUPAC, as have all of the conferences in the ICCE series, and the national chemistry body, RACI. This lends an air of authority to the meeting – these are professional bodies who are interested in promotion of chemistry education, rather than just out to make money.

ICCE2018logo

Conference Themes

But what if there is a conference that is out to make profit but means well and wants to host a good conference on a particular theme, where it has identified a gap. This isn’t illegal or morally wrong. We can use the conference themes to get a sense of how invested the organisers are in organising a conference about a topic that will bring a lot of like minded individuals together. I’ve pasted below an image from a tweet from the organisers of the “8th Edition of International Conference on Chemistry Education and Research” (note the ‘and’).

DPzbt2WVwAYUwyO

Exercising Judgement

It is in the interest of organisers of conferences such as these to spread the net widely; the more themes they cover, the more people will likely match. But of course, the broader the net, the more useless the meeting will be. It is worth exercising some judgement by considering the three points above. Even the conference title needs consideration: including an “and” is very popular as it allows a much broader range of topics while sounding like another very well established conference. Compare:

  • 25th IUPAC International Conference on Chemistry Education (ICCE2018)

with

  • 8th Edition of International Conference on Chemistry Education and Research

At a passing glance in a busy email reading session, both look similar.

Many readers of this will likely have received an invitation from the “Journal of Chemistry: Education, Research and Practice” and perhaps confuse it with the journal Chemistry Education Research and Practice. In this case, punctuation reveals very different intentions.

Take care, and if you do go to one of these “conferences”, I hope the scenery is nice!

Harmony in the Chemistry Lab

One of the difficulties students often raise is that the lab report they are required to produce is different for one section (not looking at anyone…) than it is for others. I think it is a fair comment. In Scotland and Ireland, students complete four year undergraduate Bachelor courses, and the first year in these courses is usually a highly organised, well-oiled machine which is consistent in format across the year (it would be similar in nature to the “Gen Chem” courses internationally). So when a student enters Year 2, I think it must be quite a shock to find out about different sections, and that different sections have their own cultures.

Chemistry 2 Laboratory Web

One thing we have done this year is to agree on a common report template for reports. Yes, I know Physical like to do it this way and Organic like it that way (Inorganic chemists don’t seem too fussy).  Our agreed template tries to accommodate these differences by mentioning particular emphases in each component of the report, although not without compromise. The intention is that as students move from one section to another through their year, feedback they get on a particular component of the report in one section in November is useful to them when they are doing a report for another section in March. Or rather, the clarity about the value of that feedback is better.

Once we had this in the bag, other things fall into place. The poster shows what is now in every Chemistry 2 laboratory manual and outside the laboratory. As well as the report assessment, we’ve harmonised how we treat pre-labs, what the expectations are in the lab. But we’ve also made clear (I hope!) how the current programme builds on Year 1 work, as well as outlining what is next. A key point is that each section (Inorganic, Physical, Organic) in the year is described in terms of the main focus (outcomes), showing students what the similarities and differences are. I think that this kind of information, which is often implicit, is useful to extend to students. More importantly, it keeps staff focussed on considering the practical course as one course rather than three courses.

As I begin to think about next year’s manuals, I’ll happily hear any comments or suggestions!