How to do a literature review when studying chemistry education

It’s the time of the year to crank up the new projects. One challenge when aiming to do education research is finding some relevant literature. Often we become familiar with something of interest because we heard someone talk about it or we read about it somewhere. But this may mean that we don’t have many references or further reading that we can use to continue to explore the topic in more detail.

So I am going to show how I generally do literature searches. I hope that my approach will show you how you can source a range of interesting and useful papers relevant to the topic you are studying, as well as identify some of the key papers that have been written about this topic. What I tend to find is that there is never any shortage of literature, regardless of the topic you are interested in, and finding the key papers is a great way to get overviews of that topic.

Where to search?

For literature in education, there are three general areas to search. Each have advantages and disadvantages.

For those with access (in university) Web of Science will search databases which, despite the name, include Social Sciences and Arts and Humanities Indexes. Its advantage is that it is easy to narrow down searches to very specific terms, times, and research topics, meaning you can quickly source a list of relevant literature. Its disadvantage is that it doesn’t search a lot of material that may be relevant but that doesn’t pass the criteria for inclusion in the database (peer review, particular process regarding review, etc). So for example, Education in Chemistry articles do not appear here (As they are not peer reviewed as EiC is a periodical), and CERP articles only appeared about 10 years ago, thanks to the efforts of the immediate past editors. CERP is there now, but the point is there are a lot of discipline based journals (e.g. Australian Journal of Education in Chemistry) that publish good stuff but that isn’t in this database.

The second place to look is ERIC (Education Resources Information Center) – a US database that is very comprehensive. It includes a much wider range of materials such as conference abstracts, although you can limit to peer review. I find ERIC very good, although it can link to more obscure material that can be hard to access.

Finally, there is Google Scholar. This is great as everyone knows how to use it, it links to PDFs of documents are shown if they are available, and it is very fast. The downside is that it is really hard to narrow your search and you get an awful lot of irrelevant hits. But it can be useful if you have very specific search terms. Google Scholar also shows you who cited the work, which is useful, and more extensive than Web of Science’s equivalent feature, as Google, like ERIC, looks at everything, not just what is listed in the database. Google is also good at getting into books which you may be able to view.

A practice search

I am going to do a literature search for something I am currently interested in: how chemistry students approach studying. I’m interested in devising ways to improve how we assist students with study tasks, and so I want to look to the literature to find out how other people have done this. For the purpose of this exercise, we will see that “study” is a really hard thing to look for because of the many meanings of the word. I intend it to mean how students interact with their academic work, but of course “study” is very widely used in scientific discourse and beyond.

It’s important to write down as precisely as you can what it is you are interested in, because the first challenge when you open up the search database is to choose your search terms.

Let’s start with Web of Science. So I’ve said I’m interested in studying chemistry. So what if I put in

study AND chem*

where chem* is my default term for the various derivatives that can be used – e.g. chemical.

study and chemstar

Well, we can see that’s not much use, we get over 1 million hits! By the time I go through those my students will have graduated. The problem of course is that ‘study’ has a general meaning of investigation as well as a specific one that we mean here.

Let’s go back. What am I interested in. I am interested in how students approach study. So how might authors phrase this? Well they might talk about “study approaches”, or “study methods” or “study strategies” or “study habits”, or “study skills”, or… well there’s probably a few more, but that will be enough to get on with.

(“study approach*” or “study method*” or “study strateg*” or “study habit*” or “study skill*”) AND Chem*

So I will enter the search term as shown. Note that I use quotations; this is to filter results to those which mention these two words in sequence. Any pair that match, AND a mention of chem* will return in my results. Of course this rules out “approaches to study” but we have to start somewhere.

Search2

How does this look? Over 500. OK, better than a million+, but we can see that some of the hits are not relevant at all.

search2 results

In Web of Science, we can filter by category – a very useful feature. So I will refine my results to only those in the education category.

refine search2

This returns about 80 hits. Much better. Before we continue, I am going to exclude conference proceedings. The reason for this is that very often you can’t access the text of these and they clutter up the results. So I will exclude these in the same way as I refined for education papers above, except in this case selecting ‘exclude’. We’re now down to 60 hits, which is a nice enough number for an afternoon’s work.

Thinning

Let’s move on to the next phase – an initial survey of your findings. For this phase, you need to operate some form of meticulous record keeping, or you will end up repeating your efforts at some future date. It’s also worth remembering what we are looking for: approaches people have used to develop chemistry students’ study skills. In my case I am interested in chemistry and in higher education. It is VERY easy to get distracted here and move from this phase to the next without completing this phase; trust me this will just mean you have to do the initial trawl all over again at some stage.

This trawl involves scanning titles and then abstracts to see if the articles are of interest. The first one in the list looks irrelevant, but clicking on it suggests that it is indeed for chemistry. It’s worth logging for now. I use the marked list feature, but you might choose to use a spreadsheet or a notebook. Just make sure it is consistent! Scrolling through the hits, we can very quickly see the hits that aren’t relevant. You can see here that including “study approach” in our search terms is going to generate quite a few false hits because it was picked up by articles mentioning the term “case study approach”.

I’ve shown some of the hits I marked of interest below. I have reduced my number down to 21. This really involved scanning quickly through abstract, seeing if it mentioned anything meaningful about study skills (the measurement or promotion of study skills) and if it did, it went in.

marked list 1

Snowballing

You’ll see in the marked list that some papers have been cited by other papers. It’s likely (though not absolutely so) that if someone else found this paper interesting, then you might too. Therefore clicking on the citing articles will bring up other more recent articles, and you can thin those out in the same way. Another way to generate more sources is to scan through the papers (especially the introductions) to see which papers influenced the authors of the papers you are interested in. You’ll often find there are a common few. Both these processes can “snowball” so that you generate quite a healthy number of papers to read. Reading will be covered another time… You can see now why about 50 initial hits is optimum. This step is a bit slow. But being methodical is the key!

A point to note: it may be useful to read fully one or two papers – especially those which appear to be cited a lot – before going into the thinning/snowballing phases as this can help give an overall clarity and context to the area, and might mean you are more informed about thinning/snowballing.

A practice search – Google Scholar

What about Google Scholar? For those outside universities that don’t have access to Web of Science, this is a good alternative. I enter my search terms using the Advanced search term accessed by the drop down arrow in the search box: you’ll see I am still using quotation marks to return exact matches but again there are limitations for this – for example strategy won’t return strategies, and Google isn’t as clever. So depending on the hit rate, you may wish to be more comprehensive.

Google

With Google, over 300 hits are returned, but there isn’t a simple way to filter them. You can sort by relevance, according to how Google scores that, or by date, and you can filter by date. The first one in the list by Prosser and Trigwell is quite a famous one on university teacher’s conceptions of teaching, and not directly of interest here – although of course one could argue that we should define what our own conception of teaching is before we think about how we are going to promote particular study approaches to students. But I’m looking for more direct hits here. With so many hits, this is going to involve a pretty quick trawl through the responses. Opening hits in new tabs means I can keep the original list open. Another hit links to a book – one advantage of Google search, although getting a sense of what might be covered usually means going to the table of contents. A problem with books though is that only a portion may be accessible. But the trawl again involves thinning and snowballing, the latter is I think much more important in Google, and as mentioned scopes a much broader citing set.

Searching with ERIC

Finally, let’s repeat with ERIC. Putting in the improved Web of Science term returns 363 hits (or 114 if I select peer-reviewed only).

Eric search

ERIC allows you to filter by journal, and you can see here that it is picking up journals that wouldn’t be shown in Web of Science, and would be lost or unlikely in Google. You can also filter by date, by author, and by level (although the latter should be treated with some caution). Proquest is a thesis database, so would link to postgraduate theses (subscription required, but you could contact the supervisor).

eric filters

The same process of thinning and snowballing can be applied. ERIC is a little frustrating as you have to click into the link to find out anything about it, whereas the others mentioned show you, for example, the number of citations. Also, for snowballing, ERIC does not show you the link to citing articles, instead linking to the journal webpage, which means a few clicks. But for a free database, it is really good.

Which is the best?

It’s interesting to note that in the full search I did using these three platforms, each one threw up some results that the others didn’t. I like Web of Science but that’s what I am used to. ERIC is impressive in its scope – you can get information on a lot of education related publications, although getting access to some of the more obscure ones might be difficult. Google is very easy and quick, but for a comprehensive search I think it is a bit of a blunt instrument.  Happy searching!

What does active learning look like in college science classrooms?

A new review addressing this topic was recently published. I love reviews (someone else does all the hard work and you just have to read their summary!) and this one does a good job of categorising many of the approaches under the “active” umbrella. There are some limitations (for me) in their analysis, but the categorisation is useful nonetheless.

Most interestingly, the authors present a framework to consider active learning. There are two components to this. One is perhaps obvious: considering active learning means that you must first have an overall approach (i.e. are you teacher/student-centred, constructivist, etc); a strategy – a basis for why you will design particular activities in the classroom; and finally what these activities are. That seems pretty obvious.

The authors then draw on social interdependence theory (no, I hadn’t either) which identifies whether there is positive, negative, or no benefit from cooperating with others in attaining a goal. This is interesting. They then place on a grid the various activities depending on whether there is benefit from interdependence or not (positive or none) and what kind of peer interactions an active learning strategy might employ. The grid they come up with is shown, and they highlight:
– the difference between one and two-stage polling (clicker questions vs more formal peer-instruction)
– there are a lot of activities that depend on only ‘loose’ peer interactions; interactions which are short lived and do not involve the same peers.

active learning social interdependence

Types of active learning

The review is useful as it offers a smorgasbord of the kinds of activities people undertake. These are categorised into four main headings:
1. Individual non-polling activities such as the minute-paper, writing exercises, individualy solving a problem, concept maps, building models…
2. In-class polling activities formalised in terms of peer-instruction (question, vote, discussion with peer, revote) and sequence of questions, and non-formalised such as one-off voting, poll followed by written answer…
3. Whole class discussions involving an activity, a facilitated discussion, and questions/answers…
And
4. In-class activities such as POGIL, lecture-tutorials, PBL activities, and jigsaws (different groups do different parts of a problem and then it is all brought together).

Defining active learning

The authors define active learning as: “active learning engages students in the process of learning through activities and/or discussion in class, as opposed to passively listening to an expert. It emphasises higher order thinking and often involves group work.

I think it is useful piece of work, and certainly the social interdependence piece is interesting. But I do wonder how they searched for information. They give details on this, but they seem to be missing a whole tranche of work around the flipped lecture movement, for example, and it would have been interesting to read about how active learning scenarios were facilitated in terms of curriculum design and what kinds of things were done in class once that information was available (or indeed relied on that process). Also they describe how they got papers but have a very worrying (in my view) statement about not looking at discipline specific journals, such as CERP. But a CERP paper is referenced. In fairness they do not declare to be comprehensive, but this kind of thing makes me wonder about the extent of the literature surveyed. But that is a minor gripe, and I think it is well worth a read.

Reference

Arthurs, L. A., & Kreager, B. Z. (2017). An integrative review of in-class activities that enable active learning in college science classroom settings. International Journal of Science Education, 1-19.

A new review on pre-labs in chemistry

Much of my work over the last year has focussed on pre-labs. In our research, we are busy exploring the role of pre-labs and their impact on learning in the laboratory. In practice, I am very busy making a seemingly endless amount of pre-lab videos for my own teaching.

These research and practice worlds collided when I wanted to answer the question: what makes for a good pre-lab? It’s taken a year of reading and writing and re-reading and re-writing to come up with some sensible answer, which is now published as a review.

There are dozens of articles about pre-labs and the first task was to categorise these – what are others doing and why they are doing it. We came up with a few themes, including the most common pair: to introduce the theory behind a lab and to introduce experimental techniques. So far so obvious. Time and again these reports – we gathered over 60 but there are likely more our search didn’t capture – highlighted that pre-labs had benefit, including unintended benefits (such as a clear increase in confidence about doing labs).

Why were pre-labs showing this benefit? This was rarer in reports. Some work, including a nice recent CERP paper, described the use of a underpinning framework to base the design of pre-labs upon and meant that the outcomes could be considered in that framework (in that case: self-regulation theory). But we were looking for something more… over-arching; a framework to consider the design considerations of pre-labs that took account of the unique environment of learning in the laboratory.

We have opted to use the complex learning framework as a framework for learning in laboratories, for various reasons. It is consistent with cognitive load theory, which is an obvious basis for preparative work. It describes the learning scenario as one where several strands of activity are drawn together, and is ‘complex’ because this act of drawing together requires significant effort (and support). And it offers a clear basis on the nature of information that should be provided in advance of the learning scenario. Overall, it seemed a sensible ‘fit’ for thinking about laboratory learning, and especially for preparing for this learning.

What makes for a good pre-lab?

We drew together the learning from the many reports on pre-lab literature with the tenets from complex learning framework to derive some guidelines to those thinking about developing pre-laboratory activities. These are shown in the figure. A particular advantage of the complex learning framework is the distinction between supportive and procedural information, which aims to get to the nitty-gritty of the kind of content that should be incorporated into a pre-lab activity. Casual readers should note that the “procedural” used here is a little more nuanced than just “procedure” that we think about in chemistry. We’ve elaborated a lot on this.

I hope that this review is useful – it has certainly been a learning experience writing it. The pre-print of the review is now available at http://dx.doi.org/10.1039/C7RP00140A and the final formatted version should follow shortly.

5 guielines for developing prelabs

A talk on integrating technology into teaching, learning, and assessment

While in Australia, I was invited to present a talk to the Monash Education Academy on using technology in education. They recorded it and the video is below. The talk had a preamble about a theme of “personalisation” that I am increasingly interested in (thanks especially to some work done by the Physics Education Research Group here at Edinburgh), and then discussed:

  1. Preparing for lectures and flipping
  2. Discussion Boards
  3. Using video for assessment

A view from Down Under

Melbourne Seventh City of Empire, part of the Australia 1930s Exhibition at National Gallery of Victoria
Melbourne Seventh City of Empire, part of the “Brave New World: Australia 1930s” Exhibition at National Gallery of Victoria

I’ve spent the last two week in Australia thanks to a trip to the Royal Australian Chemical Institute 100th Annual Congress in Melbourne. I attended the Chemistry Education symposium.

So what is keeping chemistry educators busy around this part of the world? There are a lot of similarities, but some differences. While we wrestle with the ripples of TEF and the totalitarian threat of learning gains, around here the acronym of fear is TLO: threshold learning outcomes.  As I understand it, these are legally binding statements stating that university courses will ensure students will graduate with the stated outcomes. Institutions are required to demonstrate that these learning outcomes are part of their programmes and identify the level to which they are assessed. This all sounds very good, except individuals on the ground are now focussing on identifying where these outcomes are being addressed. Given that they are quite granular, this appears to be a huge undertaking and is raising questions like: where and to what extent is teamwork assessed in a programme?

Melbourne from the Shrine
Melbourne from the Shrine

This process does appear to have promoted a big interest in broader learning outcomes, with lots of talks on how to incorporate transferable skills into the curriculum, and some very nice research into students’ awareness of their skills. Badges are of interest here and may be a useful way to document these learning outcomes in a way that doesn’t need a specific mark. Labs were often promoted as a way of addressing these learning outcomes, but I do wonder how much we can use labs for learning beyond their surely core purpose of teaching practical chemistry.

Speaking of labs, there was some nice work on preparing for laboratory work and on incorporating context into laboratory work. There was (to me) a contentious proposal that there be a certain number of laboratory activities (such as titrations) that are considered core to a chemist’s repertoire, and that graduation should not be allowed until competence in those core activities be demonstrated. Personally I think chemistry is a broader church than that, and it will be interesting to watch that one progress. A round-table discussion spent a good bit of time talking about labs in light of future pressures of funding and space; and it does seem that we are still not quite clear about what the purpose of labs are. Distance education – which Australia has a well-established head start in – was also discussed, and I was really glad to hear someone with a lot of experience in this say that it is possible to generate a community with online learners, but that it takes a substantial personal effort. The lab discussion continued to the end, with a nice talk on incorporating computational thinking into chemistry education, with suggestions on how already reported lab activities might be used to achieve this.

Gwen Lawrie delivers her Award Address
Gwen Lawrie delivers her Award Address

Of course it is the personal dimension that is the real benefit of these meetings, and it was great to meet some faces old and new. Gwen Lawrie wasn’t on the program as the announcement of her award of Education Division Medal was kept secret for as long as possible. I could listen to Gwen all day, and her talk had the theme “Chasing Rainbows”, which captured so eloquently what it means to be a teacher-researcher in chemistry education, and in a landscape that continues to change. [Gwen’s publications are worth trawling] Gwen’s collaborator Madeline Schultz (a Division Citation Winner) spoke about both TLOs and on reflections on respected practitioners on their approaches to teaching chemistry – an interesting study using a lens of pedagogical content knowledge. From Curtin, I (re-)met Mauro Mocerino (who I heard speak in Europe an age ago on clickers) who spoke here of his long standing work on training demonstrators. Also from that parish, it was a pleasure to finally meet Dan Southam. I knew Dan only through others; a man “who gets things done” so it was lovely to meet him in his capacity as Chair of the Division and this symposium, and to see that his appellation rang true. And it was nice to meet Elizabeth Yuriev, who does lovely work exploring how students approach physical chemistry problem and on helping students with problem solving strategies.

Dinner Date
Dinner Date

There were lots of other good conversations and friendly meetings, demonstrating that chemistry educators are a nice bunch regardless of location. I wasn’t the only international interloper; Aishling Flaherty from University of Limerick was there to spread her good work on demonstrator training – an impressive programme she has developed and is now trialling in a different university and a different country. And George Bodner spoke of much of his work in studying how students learn organic chemistry, and in particular the case of “What to do about Parker”. The memory of Prof Bodner sitting at the back of my talk looking at my slides through a telescopic eye piece is a happy one that will stay with me for a long time. Talk of organic chemistry reminds me of a presentation about the app Chirality – 2 which was described – it covers lots of aspects about revising organic chemistry, and looked really great.

The Pioneer, National Gallery of Victoria
The Pioneer, National Gallery of Victoria

My slightly extended trip was because I had the good fortune to visit the research group of Prof Tina Overton, who moved to Melbourne a few years ago, joining native Chris Thompson in growing the chemistry education group at Monash. It was an amazing experience immersing in a vibrant and active research group, who are working on things ranging from student critical thinking, chemists’ career aspirations, awareness of transferable skills, and the process and effect of transforming an entire laboratory curriculum. I learned a lot as I always do from Tina and am extremely grateful for her very generous hosting. I leave Australia now, wondering if I can plan a journey in 2018 for ICCE in Sydney.

From Hokusai exhibition, NGV
From Hokusai exhibition, NGV. My interpretation of students managing in a complex learning environment

On the need for funding in UK Chemistry Higher Education

In 2014/2015, 18,495 people opted to study undergraduate chemistry at higher education in the UK. What do we know about their experience of learning chemistry?

A search of Web of Science for those based in the UK publishing about chemistry education in the period 2014, 2015, 2016 was conducted. This returned 88 hits. An initial screening reduced this number to 71. Those removed included things like returns for a “Wales” hit that was New South Wales, or book chapters that weren’t about chemistry teaching in classrooms, or authors based in the UK but writing about non-UK classrooms.

71

Of these 71, the results were categorised as follows. 7 papers referred to chemical engineering, or something specific about chemistry for engineers. While these may have value to chemists, they are not about teaching chemistry to chemists. Similarly, 1 paper was specifically discussing some detail of chemistry relevant to a pharmacy syllabus.

63

This left 63. A further 10 articles were about school chemistry. 6 were editorials, and 2 were reviews about some aspect of chemistry, which, while written in the UK, obviously extended their reach into international curricula. A further 3 were about informal chemistry; public or outreach or the presentation of chemistry in popular books.

42

So now we are left with 42 articles about chemistry education in higher education written by someone based in the UK. The majority of these – 22 – were about something to do with laboratory work; typically some new laboratory experiment, but occasionally some approach to doing something in the lab (e.g. alternative lab reports). A further 16 were categorised innovative ideas; unusual or novel approaches that are being reported because of novelty. The evaluation on whether such innovations are effective or not in this category tend to be based on student evaluations or questionnaires, and typically lack a formal research basis.

4

This leaves 4 articles published in 2014, 2015, 2016 that described something about the curriculum or the learners in higher education chemistry. 2 of these articles were written by Overton and Randles at Hull, both of whom are now abroad. The remaining 2 were on how well maths prepares students for studying chemistry – part of a large project looking at the relevance of maths for a variety of subjects; and on the design and evaluation of a polymer course. That’s it.

Chem Ed Publications in UK

We know nothin’

I argue then that we have no idea how students experience chemistry in higher education in the UK. We have no idea how well school chemistry prepares them, what their difficulties are, how they study, or what particular aspects of studying the UK chemistry degree are challenging. We’ve no idea how students experience lectures, what they learn in laboratories, nor how tutorials run. We don’t know how students balance workload, how they study, what affect part-time work has, nor whether students are able to discuss the relevance of chemistry to everyday life. We get occasional glimpses about issues around students’ employability, thanks to the work such as that done at Nottingham (CERP, 2017, outside the time boundary of this search) but we have never revisited the glory of Hanson and Overton, 2010. We will of course have copious amounts of data on students’ entry performance. We can guess that we will have normal distributions in grade data in our annual assessments, and that external examiners will generally be satisfied. Entry and output detail are all we can cling on to.

Around 19,000 chemistry students will be starting in September. We desperately need some funding to begin to find out in a serious way what experience awaits them.

200th Blog Post

This is my 200th blog post. Now I should say that, while I am impressed with that number, given that it is over seven years since Róisín Donnelly and Muireann O’Keeffe gently broached the idea of starting a blog, it is not a fantastic output rate: to borrow Kevin Bridges joke about losing 4 stone over 10 years, I don’t think I’ll be writing a book on how to blog.

I’m going to avoid the kind of post where I reflect on my blogging, think about what I’ve learned, and look with renewed wistful enthusiasm to the future: fail better! I’m also going to avoid my usual call to encourage others to blog, as my conversion rate is low. (Briefly: yes you have something to say, yes you can write, yes it is worth the time, and you’ll only be threatened with litigation once).

So instead I will celebrate by highlighting a few blogs I like to go to for my chemistry/education fix. I should say this isn’t meant to be encyclopedic, but rather the blogs that, when I see a new post, I will make time to read it. I have previously done more formal summaries for Nature Chemistry and Education in Chemistry.  (I see you have to pay $18 to read the Nature Chemistry one! Holy Mother of Jerusalem how did we get to this state?)

Read these blogs:

Katherine Haxton: Possibilities Endless – I love Katherine’s blog; it’s always a dose of pragmatism and reality, mixed in with something useful. She’s honest and funny and it is all very refreshing;  This is also one of the very few blogs left in the world where people seem to leave comments.

Blogs about chemistry teaching and evidence based practice

  • David Paterson: Thoughts on chemistry and education – think this is a newish blog. Given its title, its appeal is apparent. And it doesn’t disappoint. It’s clear that the author is someone who thinks very seriously about teaching and his blogs are wonderful summaries and conversations about those thoughts.
  • Kristy Turner: Adventures in chemistry education on both sides of the transition between school and HE – just like the title, Kristy is someone with a lot to say and this blog is one of the outlets she uses. Always some good insight and highlighting things you mightn’t have thought of.
  • Niki Kaiser: NDHS Blogspot – This is an amazing website and growing resource. I can’t actually keep up with it but Niki and others post stuff of on lots of aspects of teaching chemistry; cognitive science being the strand I follow. Very useful.

Blogs about chemistry education research and ongoing projects

This category is sadly not well occupied. How wonderful would it be to get updates and insights into the work people are doing. We tried with our badging lab skills site, and it got lots of interest, so despite promising not to, I do really encourage people to do it. Two nascent blogs in this category offering real hope are:

  • Stephen George-Williams Investigating the effects of Transforming Laboratory Learning – Stephen is updating about his PhD project which is centred around lab education. I think this is a great idea and it will be interesting to follow to see the kind of data gathered and the kinds of processes done with it.
  • Nimesh Mistry Mistry Research Group – Nimesh is also blogging about his observations as part of research in his group. His latest one documents questions students ask in the lab, and plans to think about how he will use those observations in planning lab design. More please!

I’m sure I have forgotten some and hope that if I have, I am reminded, so that I can apologise most profusely.

 

 

Mayer’s Principles: Using multimedia for e-learning (updated 2017)

Anyone involved in e-learning will know of the cognitive theory of multimedia learning, which draws together information processing model (dual coding), cognitive load theory (working memory), and the notion of active processing. You can read a little more of this in this (old) post.

Anyway, for most of us who don’t do full on e-learning, Mayer’s principles have value when we make things like videos or multimedia that we wish the students to interact with outside of their time with us. As such, Mayer’s principles, as reported in The Cambridge handbook of multimedia learning are well cited. Mayer has just published an update (HT to the wonderful new Twitter feed: https://twitter.com/CogSciLearning), and because I have nothing better to do than twiddle my thumbs for the summer (thank you Adonis), I made a graphic summarising the 12 principles he describes. Many seem obvious but that is probably no bad thing; as well as thinking about videos, there might be some lessons about PowerPointing here too. Click on the image to embiggen.

Mayer’s Principles: Using multimedia for e-learning (from Mayer, R. E. (2017) Using multimedia for e-learning. Journal of Computer Assisted Learning, doi: 10.1111/jcal.12197)
Mayer’s Principles: Using multimedia for e-learning (from Mayer, R. E. (2017) Using multimedia for e-learning. Journal of Computer Assisted Learning, doi: 10.1111/jcal.12197)

Lessons from a decade of ‘doing’ chemistry education

It’s been 10 years since “Developing practical chemistry skills by means of student-driven problem based learning mini-projects” was published in Chemistry Education Research and Practice, and it marked the kick-starting of an accidental career invested in chemistry education. This paper was published with two colleagues and friends, Claire Mc Donnell and Christine O’Connor, who inducted me into the ways of all things chem-ed. We would continue to work together; Claire and I guest-edited a special issue of CERP on technology in chemistry education in 2013, writing an editorial that is surprisingly cited quite often (for an editorial – I think it is because we say… something). And Christine and I wrote a book-chapter for the 2015 Wiley book on Chemistry Education, which in full respect to the editors, has quite a list of names assembled as authors. But wait: this is not an article about how great Seery is (that’s the next one, and the one before).

Capture

Can I say anything sensible about making this a profession? I’m often asked by people who are interested in teaching and learning and education-focussed careers what kinds of things they should think about to achieve this. I don’t know if I am qualified to answer that as much of what has happened was unplanned, fostered by a benevolent, and often indifferent department, working in a REFless, TEFless culture. Moving to Edinburgh has meant that this is now my ‘proper job’ rather than a hobby, but I wouldn’t call the transition or the journey a career path. So much of what follows is what I would advise, considering the REFful, TEFful culture we now live in. With that caveat, here are the top tips…

1. Separate the inner scholar from the teacher

One of the biggest difficulties for someone invested in teaching and learning chemistry is that all aspects of teaching and learning chemistry are probably of some interest. I remember going to conferences and wanting to see everything and #ohmygodthatssocoolwemusttrythat and being overwhelmed very quickly, because of course you don’t have time to try everything. You will not have time to think about everything at once, so my headline piece of advice is to identify what it is you will focus on; what will become your niche. You as a teacher will need to think about labs and lectures and tutorials and online marking and placement and professional development and…

But what will you as a scholar focus on? What is going to be the topic you will be able to have an intellectual basis in? Name it and begin to be strict with yourself about focussing on it. I see a lot of people who don’t produce any outputs even though they are doing good work because they are trying to do too many things.

Imagine an organic chemist. They of course know about most aspects of ongoing organic chemistry generally – they could teach any 2nd year course, but they specialise in their research on one or two particular aspects.

2. Read

If you are going to be scholarly about something, then you must read. I find it very surprising how little people read, or worse, how people cherry-pick some literature. Reading is important if we are going to move on from “gut feeling” or “in my experience” that drags down our academic standing. It is impossible to read everything, but that doesn’t mean you don’t read anything, and certainly doesn’t mean you rely on 140 character summaries as your academic insight. Twitter is amazing for pointing out unusual highlights and what other people who you respect consider important; and I have discovered countless gems that way. But you must be more systematic. This involves identifying a series of journals that you think are of interest and keeping up to date with what is published. If getting into a new area, it involves surveying the literature (hopefully finding a review!), finding out who the key players are. Reading also helps develop a kind of cultural capital – how do people go about things in this field; what are the acceptable norms? What the hell does being ethical mean?

How do you read? The challenge of reading 1000 papers might be a bit daunting. So of course you don’t need to read every line (except mine, for those: read every, single, line), but rather you are reading with a purpose. Perhaps you are making notes on how people implemented online quizzes in their courses. It doesn’t really matter if someone in University of West Nowhere scores went from 45.6% to 52.1%; what matters in this initial survey is what was their rationale and context, how did they go about it, how did they measure, what limitations did they state, and who did they cite to be of influence. You can very quickly build up a map of studies so that you now have a basis for designing your study on exploring how online quizzes; you can state what other people have done, give a rationale for your approach, and compare your results to others. Too often, this analysis is done post hoc. This is not a scholarly approach.

Reading also involves becoming familiar with learning theories. Again, just reading lots of learning theories is a passive way to approach this. Everybody is a constructivist because everybody is a constructivist. But what does that even mean? I thought you liked cognitivism too? How do you marry those thoughts?

3. Generate outputs

Many people do identify an area and do read, but never “get around” to publishing. This is tragic, because it means everyone has benefited from their scholarship except them. Developing outputs; at the very least conference presentations; is the only way the world (and also promotion and interview panels) know something exists. Everyone is a great teacher, everyone can quote some line of good feedback; don’t get that confused with the work of a scholar – producing some output to share with the world that is the result of academic work. The obvious output is a journal publication, but what if you made artefacts as part of some study – can you publish them online – maybe even have a link on the department website. Especially for those new to the field, this will be a useful indicator to show that you have demonstrated interest and will be a useful talking point at interviews.

One of the difficulties people find with writing outputs is that they don’t know how to write. They had a great idea, they got some nice results, and now they’ve got to make it look academic, which involves finding some references that look appropriate (See reading, above). Of course this is not the way to go about things. Our organic chemist does not just go into the lab one day and mix some things, happen across an interesting result, and then think about finding some sensible rationale as to why those chemicals were mixed. And it is unlikely that our educator was similarly flippant – there is likely some rationale in there but it makes life so much easier if the reading was done in advance to give that rationale some basis.

In my own experience, I cannot understate the value of keeping a blog has been to develop writing (yes I know this one is a long waffle). When you write something, you learn to think of how to present arguments, write a narrative, and in cases of academic blogging, have to have read something before writing about it. They say you don’t understand something until you teach it; trust me: you don’t understand something until you blog about it and expose your thoughts to the world. The world, in return, is usually grateful for you sharing those thoughts. And it all ends up being an accidental output.

4. Make friends

It’s nice to discuss your work and have support. One of the best things I had was the support of my original two co-authors. Claire and I went on to formalise this in a study we subsequently did and developed a critical friendship – one that was grounded in the knowledge that we both wanted what was best for each other, but not afraid to call the other up when something was awry about some aspect of the work or a conclusion. It was fantastic, not only for the actual conversations, but for the imagined ones too; I would wonder what Claire would think about something even before I would talk to her. This isn’t easy to find but worth seeking out. At the very least, connecting with others at conferences – yes we would all rather stand facing a corner and check our phone occasionally but come on now everyone, turn around. Talk. Introduce yourself. I was taken aback recently when someone I had always been afraid to talk to came up and introduced. We had a great chat and ended up with a group hug (it’s the way I roll). The point is, people go to these things because we have a common interest. Very often, there is someone else in the department who is interested in teaching. Talk with them (not at them).

One key aspect of making friends is the ability to listen. One thing that is slightly grating (to me) is that people will listen to you long enough to find out about how they can link onto something you are saying to something that they have done that is much better. This is not a way to learn about people and how they do things. Use your blog to show off. A better approach might be to think about how what the thing the person is talking about and what your own experiences are might marry into a useful collaboration? You’re a constructivist aren’t you?

In terms of making friends in the UK, the annual VICE conference is a good place to start. Registration deadline for this year is imminent (www.vicephec2017.com).

 

 

Student study approaches

Many thanks to Scott Lewis/USF who put an interesting paper on students’ approaches to study in introductory chemistry my way. The paper describes the development of a framework for learning approaches in chemistry, and they come up with four levels: (1) gathering facts; (2) learning procedures; (3) confirming understanding; and (4) applying ideas.

Do students know how to study? In an exam dominated system, one might reasonably expect students to focus on the second approach – if they learn the procedures and have the facts to hand, then they will be able to use these in an exam. Of course, as teachers we hope students will aspire to developing (and confirming) understanding, and begin to see how this understanding is useful in a wider sense.

So my initial reaction was to make a poster on these learning approaches. It seems that both levels 2 and 3 have similar outcomes; students are able to use procedures in assessments; but the motivations are very different for both. My first attempt at a poster is below, but having spent an afternoon making it and refining it, I am wondering if it now immediately needs a friend in the form of a poster detailing specific strategies. The intention is to (a) make it clear to students that different approaches exist, and then (b) give them some strategies (beyond rewriting notes) that they can put into place.  (Note I left level 4 off this poster for reasons I think I can defend).

More thought needed…

Study Approaches

Reflections on #MICER17

Two related themes emerged for me from the Methods in Chemistry Education Research meeting last week: confidence and iteration.

Let’s start where we finished: Georgios Tsaparlis’ presentation gave an overview of his career studying problem solving. This work emerged out of Johnstone’s remarkable findings around working memory and mental demand (M-demand).1,2 Johnstone devised a simple formula – if the requirements of a task were within the capability of working memory, students would be able to process the task; if not, students would find it difficult. This proposal was borne out of the plots of performance against complexity (demand) which showed a substantial drop at the point where M-demand exceeded working memory, and these findings seeded a remarkable amount of subsequent research.

However, things are seldom as simple as they seem and Tsaparlis’ work involved studying this relationship in different areas of chemistry – looking, for example, at how students solve problems in organic chemistry compared to physical chemistry, and the effect of the type of question. Each study was an iteration, an investigation of another aspect of this multi-dimensional jigsaw, aiming to make a little bit more sense each time. Sometimes the results led to an ill-fitting piece, with data being consigned to a desk drawer for a few years until further study allowed it to be explored in a new light. Towards the end of this arc of work, he began to move away from linear modelling, where we look at the strength of individual aspects on an overall hypothesis, to more complex models such as the “random walk”. It is another iteration.

The point to make here is there was no study that said: this is how students solve equilibrium questions. Rather, each study added a little more to understanding of a particular model framed around this understanding. Indeed Keith Taber outlined in his Ethics workshop the importance of context and situation in publishing results. Things are rarely definitive and usually context dependent.

For me this is reassuring. Just like Johnstone’s “ON-OFF” findings for working memory, there is a fear that one is either able to complete education research or one isn’t; a few participants indicated that “confidence” was one of the barriers in getting involved in education research in responding to Suzanne Fergus’ pre-meeting prompts, which guided her talk on writing research questions. I remember talking to an eminent chemistry professor who said something along the lines of “never look back!” – to just publish what you know to be your best understanding (and instrumentation) at a given time, accepting that more studies and analysis might lead to more understanding.

While this probably wasn’t intended to be as carefree as I leverage it here, there will always be one more publication, one better approach, one alternative understanding. The task then is to continually inform and develop our understanding of what it is we wish to understand. The action research cycles outlined more formally by Orla Kelly in her presentation facilitate this, although of course one might complete several cycles before considering publication. But I think iterations happen naturally as any research study progresses. Graham Scott illustrated this nicely in his presentation; later publications adding further depth to earlier ones. Stewart Kirton discussed building this iteration onto the design of research instruments.

Our task as education researchers then is to ensure that we are publishing to the best of or understanding and publishing with good intent – that we believe what we are saying at a particular time is an honest overview of our understanding of our study at that time in a given context.

Our task as practitioners is to move on from the duality of things that “work” and things that “don’t work”. The education literature isn’t unique in that it tends to publish more positive results than not, so when looking for the “best” way to do something, a keen reader may soon become overwhelmed, and even frustrated with the education literature for its lack of clarity. A task of those attending MICER then is not necessarily in translating research into practice; a common call, but rather communicating a greater awareness of the process of education research, along with how to meaningfully interpret outcomes so that they may be used – or not – in our teaching of chemistry.

 

We are grateful to Chemistry Education Research and Practice for their support of this event, along with the RSC interest groups: Tertiary Education Group and Chemistry Education Research Group.

Methods in Chemistry Education Research 2017 Welcome Slide

[1] J. Chem. Educ., 1984, 61, p 847

[2] Education in Chemistry, 1986, 23, 80-84

Wikipedia and writing

Academics have a complicated relationship with Wikipedia. There’s a somewhat reluctant acknowledgement that Wikipedia is an enormously used resource, but as the graphical abstract accompanying this recent J Chem Ed article1 shows, WE ARE NOT TOO HAPPY ABOUT IT. Others have embraced the fact that Wikipedia is a well-used resource, and used this to frame writing assignments as part of chemistry coursework.2-4  There is also some very elegant work on teasing out understanding of students’ perceptions of Wikipedia for organic chemistry coursework.5

Graphical abstract of M. D. Mandler, Journal of Chemical Education, 2017, 94, 271-272.
Graphical abstract of M. D. Mandler, Journal of Chemical Education, 2017, 94, 271-272.

Inspired by a meeting with our University’s Wikimedian in Residence I decided to try my hand at creating a Wikipedia article. The topic of the article was about a little-known chemist who hadn’t been written about before, and I’d say is unknown generally. I found her name listed on the Women in Red page, which is outside the scope of this post, save to say: go look at that page.

Writing the article was interesting, and some implications from a teaching perspective are listed:

  1. If there isn’t a Wikipedia article, writing a summary overview is quite a lot of work.

One of the great things about Wikipedia is of course that it offers a nice summary of the thing you are interested in, which then prompts you to go and look up other stuff which you can then pretend to have found originally. But what if there isn’t a Wikipedia article? Where do you start? Of course Googling and getting some information is part of this, but there is a step before, or at least coincident with this, which involves scoping out the content of what you want to summarise. This will involve reading enough so that you can begin this overview plan, and then searching to find information about the plan. In chemistry, the order of searching will likely go Google > Google Scholar > Databases like Web of Science etc > Google Books… Because of my context, I also got stuck into the RSC’s Historical Collection (a terribly under-promoted amazing resource). In any case, there is some good work to do here on developing information literacy (which in a formal education setting would probably need to be structured).

  1. #citethescheise

I was encouraged in writing to cite my work well, linking to original and verifiable sources. I am long enough in the game to know this, and may be known to advise novice academic writers to “referencify” their work for journals; the academic genre is one where we expect lots of superscript numbers to make a text look like it is well informed. Wikipedia has a very particular style where essentially every fact needs a citation. This is something I did reasonably well, but was very pleasantly surprised to see that someone else looked quite closely at these (new articles are reviewed by people who make amendments/changes). I know this because in my case I cited a modern J Mat Chem paper which offered an example of where the original contribution of my chemist had been cited about century later in 2016 (notability is a requirement in Wikipedia so I had this in mind). This reference had been checked, with the relevant line from it added to the citation. It was reassuring to know that someone took the time to consider the references in this amount of detail.

From a teaching point of view, we try in lab report and theses to encourage students to verify claims or opinion with data or literature. This seems like very good training for that. The point was also made to me that it teaches students to explore the veracity of what they read on Wikipedia, by considering the sources quoted.

  1. Learning to write

Wikipedia is an encyclopaedia (duh) and as such it has a particular style. I actually found it very difficult to write initially and went through quite a few drafts on Word with a view to keeping my piece pretty clinical and free of personal opinion.  Asking students to write Wikipedia articles will undoubtedly improve their writing of that style; I’m not too sure yet how beneficial that is; I feel the greater benefits are in information searching and citing, and in scoping out a narrative. But that is probably a personal bias. Edit: fair point made in this tweet: https://twitter.com/lirazelf/status/865124724166320128

  1. Writing to learn

Whatever about developing writing skills, I certainly learned a lot about my subject as well as much more context about the particular topic. Quite a lot of what I read didn’t make it into the final article (as it might have, for example if I were writing an essay). But as we know from preparing lecture notes, preparing a succinct summary of something means that you have to know a lot more than the summary you are presenting.

Why Wikipedia?

In challenging the arguments about Wikipedia such as those indicated in the graphical abstract above, I do like the idea of students getting to know and understand how the site works by interacting with it. Wikipedia usage is here to stay and I do think there is a strong argument around using it in academic writing and information literacy assignments. One very nice outcome is that something real and tangibly useful is being created, and there is a sense of contributing. Writing for something that is going to go live to the world means that it isn’t “just another exercise”. And Wikipedia articles always come to the top of Google searches (mine was there less than an hour after publishing).

Search view at 16:12 (left) after publishing, and at 16:43 (right)
Search view at 16:12 (left) after publishing, and at 16:43 (right).

I’m interested now in looking at Wikipedia writing, certainly in informal learning scenarios. A particular interest is going to be exploring how it develops information literacy skills and how we structure this with students.

My page, I’m sure you are dying to know is: https://en.wikipedia.org/wiki/Mildred_May_Gostling.

Lots of useful points about Wikipedia here – see Did You Know)

With thanks to Ewan McAndrew and Anne-Marie Scott.

Referencifying 

  1. M. D. Mandler, Journal of Chemical Education, 2017, 94, 271-272.
  2. C. L. Moy, J. R. Locke, B. P. Coppola and A. J. McNeil, Journal of Chemical Education, 2010, 87, 1159-1162.
  3. E. Martineau and L. Boisvert, Journal of Chemical Education, 2011, 88, 769-771.
  4. M. A. Walker and Y. Li, Journal of Chemical Education, 2016, 93, 509-515.
  5. G. V. Shultz and Y. Li, Journal of Chemical Education, 2016, 93, 413-422.

 

On the sublime

“Sublimity,” Hauptmann says, panting, “you know what that is, Pfennig?” He is tipsy, animated, almost prattling. Never has Werner seen him like this. “It’s the instant when one thing is about to become something else. Day to night, caterpillar to butterfly. Fawn to Doe. Experiment to result. Boy to man.”

All the Light We Cannot See, Anthony Doerr

Moonlight Scene, James Arthur O'Connor
Moonlight Scene, James Arthur O’Connor

Links to Back Issues of University Chemistry Education

I don’t know if I am missing something, but I have found it hard to locate past issues of University Chemistry Education, the predecessor to CERP.  They are not linked on the RSC journal page. CERP arose out of a merger between U Chem Ed and CERAPIE, and it is the CERAPIE articles that are hosted in the CERP back issues. Confused? Yes. (More on all of this here)

Anyway in searching and hunting old U Chem Ed articles, I have cracked the code of links and compiled links to back issues below. They are full of goodness. (The very last article published in UCE was the very first chemistry education paper I read – David McGarvey’s “Experimenting with Undergraduate Practicals“.)

Links to Back Issues

Contents of all issues: http://www.rsc.org/images/date_index_tcm18-7050.pdf 

1997 – Volume 1:

1 – remains elusive… It contains Johnstone’s “And some fell on good ground” so I know it is out there… Edit: cracked it – they are available by article:

1998 – Volume 2:

1 – http://www.rsc.org/images/Vol_2_No1_tcm18-7034.pdf

2 – http://www.rsc.org/images/Vol_2_No2_tcm18-7035.pdf

1999 – Volume 3:

1 – http://www.rsc.org/images/Vol_3_No1_tcm18-7036.pdf

2 – http://www.rsc.org/images/Vol_3_No2_tcm18-7037.pdf

2000 – Volume 4:

1 – http://www.rsc.org/images/Vol_4_No1_tcm18-7038.pdf

2 – http://www.rsc.org/images/Vol_4_No2_tcm18-7039.pdf

2001 – Volume 5:

1 – http://www.rsc.org/images/Vol_5_No1_tcm18-7040.pdf

2 – http://www.rsc.org/images/Vol_5_No2_tcm18-7041.pdf

2002 – Volume 6:

1 – http://www.rsc.org/images/Vol_6_No1_tcm18-7042.pdf

2 – http://www.rsc.org/images/Vol_6_No2_tcm18-7043.pdf

2003 – Volume 7:

1 – http://www.rsc.org/images/Vol_7_No1_tcm18-7044.pdf

2 – http://www.rsc.org/images/Vol_7_No2_tcm18-7045.pdf

2004 – Volume 8:

1 – http://www.rsc.org/images/Vol_8_No1_tcm18-7046.pdf

2 – http://www.rsc.org/images/Vol_8_No2_tcm18-7047.pdf

I’ve downloaded these all now in case of future URL changes. Yes I was a librarian in another life.

UCE logo

Dialogue in lectures

This is not a post on whether the lecture is A Good Thing or not. Lectures happen. PERIOD!

A paper by Anna Wood and colleagues at the Edinburgh PER group, along with a subsequent talk by Anna at Moray House has gotten me thinking a lot over the last year about dialogue and its place in all of our interactions with students. The literature on feedback is replete with discussion on dialogue, sensibly so. The feedback cycle could be considered (simplistically) as a conversation: the student says something to the teacher in their work; the teacher says something back to the student in their feedback. There’s some conversation going on. Feedback literature talks about how this conversation continues, but what about the bit before this conversation begins?

Not a monologue

The spark from Wood’s work for me was that lectures are not a monologue. She is considering active lectures in particular, but cites Bamford* who gives a lovely overview of the nature of conversation in lectures in general. Bamford presents the case that lectures are not a monologue, but are a conversation. Just as in the feedback example above, two people are conversing with each other, although not verbally. In a lecture, the lecturer might ask: “Is that OK?”. An individual student might respond inwardly “Yes, I am getting this” or “No, I haven’t a freaking clue what is going on and when is coffee”. A dialogue happened. Wood’s paper discusses these vicarious interactions – a delicious phrase describing the process of having both sides of the conversation; an internal dialogue of sorts. She describes how this dialogue continues in active lectures, but sadly there is only one Ross Galloway, so let’s think about how this conversation might continue in lectures given by us mere mortals. How can we help and inform these vicarious interactions?

Developing a conversation

A problem you will by now have identified is that the conversation: “Is that OK?” and retort isn’t much of a conversation. So how can we continue this conversation? My intention is to consider conversation starters in lectures that foster a sense with each individual student that they are having a personal conversation with the lecturer at points during the lecture. And incorporates guides for the student to continue this conversation after the lecture, up to the point that they submit their work, prompting the conversation we started with above.

In Woods talk, she mentioned specific examples. The lecturer would ask something like: “Is 7 J a reasonable answer?” A problem with “Is that OK?” is that it is too broad. It’s difficult to follow up the conversation specifically as it likely ends with yes or no.

How about a lecturer asks: “Why is this smaller than…?” You’re a student, and you’re listening. Why is it smaller? Do you know? Yes? No? Is it because…? Regardless of your answer, you are waiting for the response. You think you know the answer, or you know you don’t.

If we are to take dialogue seriously, then the crucial bit is what happens next. Eric Mazur will rightly tell us that we should have allow discussion with peers about this, but we are mortals, and want to get on with the lecture. So how about the conversation goes something like this:

“Why is this smaller than…?”

[pause]

You are a student: you will have an answer: You know, you think you know, you don’t know, you don’t know what’s going on. You will have some response.

The lecturer continues:

“For those of you who think…”

The lecturer responds with a couple of scenarios. The conversation continues beyond a couplet.

Did you think of one of these scenarios? If so the lecturer is talking to you. Yes I did think that and I have it confirmed now I am right. Or: yes I did think that, why is that wrong?

The lecturer can continue:

“While it makes sense to think that, have a look in reference XYZ for a bit more detail”.

The lecturer thus concludes this part of the conversation. A dialogue has happened and each student knows that that they have a good idea what is going on, they don’t but know where to follow up this issue, or that they haven’t a clue what is going on. Whichever case, there is some outcome, and some action prompted. Indeed one could argue that this prompted action (refer to reference) is a bridge between the lecture and tutorial – I checked this reference but don’t understand – and so the conversation continues there.

This all seems very obvious, and maybe everyone else does this and didn’t tell me. My lectures tend to have lots of “Is that OK?” type questions, but I do like this idea of a much more purposeful design to structuring the conversation with a large class. I should say that this is entirely without research base beyond what I have read, but I think it would be very empowering for students to think that a lecturer is aiming to have a conversation with them.

rf-freq-radio-wave

*Bamford is cited in Wood’s paper and I got most of it on Google Books.

Revising functional groups with lightbulb feedback

I’m always a little envious when people tell me they were students of chemistry at Glasgow during Alex Johnstone’s time there. A recent read from the Education in Chemistry back-catalogue has turned me a shade greener. Let me tell you about something wonderful.

The concept of working memory is based on the notion that we can process a finite number of new bits in one instance, originally thought to be about 7, now about 4.  What these ‘bits’ are depend on what we know. So a person who only knows a little chemistry will look at a complex organic molecule and see lots of carbons, hydrogens, etc joined together. Remembering it (or even discussing its structure/reactivity) would be very difficult – there are too many bits. A more advanced learner may be able to identify functional groups, where a group is an assembly or atoms in a particular pattern; ketones for example being an assembly of three carbons and an oxygen, with particular bonding arrangements. This reduces the number of bits.

Functional groups are important for organic chemists as they will determine the reactivity of the molecule, and a challenge for novices to be able to do this is to first be able to identify the functional groups. In order to help students practise this, Johnstone developed an innovative approach (this was 1982): an electronic circuit board.

Functional Group Board: Black dots represent points were students needed to wire from name to example of functional group
Functional Group Board: Black dots represent points were students needed to wire from name to example of functional group

The board was designed so that it was covered with a piece of paper listing all functional groups of interest on either side, and then an array of molecules in the middle, with functional groups circled. Students were asked to connect a lead from the functional group name to a matching functional group, and if they were correct, a lightbulb would flash.

A lightbulb would flash. Can you imagine the joy?!

Amide backup card
Amide backup card

If not, “back-up cards” were available so that students could review any that they connected incorrectly, and were then directed back to the board.

The board was made available to students in laboratory sessions, and they were just directed to play with it in groups to stimulate discussion (and so as “not to frighten them away with yet another test”). Thus students were able to test out their knowledge, and if incorrect they had resources to review and re-test. Needless to say the board was very popular with students, such that more complex sheets were developed for medical students.

Because this is 1982 and pre-… well, everything, Johnstone offers instructions for building the board, developed with the departmental electrician. Circuit instructions for 50 x 60 cm board were given, along with details of mounting various plans of functional groups onto the pegboard for assembly. I want one!

 

Reference

A. H. Johnstone, K. M. Letton, J. C. Speakman, Recognising functional groups, Education in Chemistry, 1982, 19, 16-19. RSC members can view archives of Education in Chemistry via the Historical Collection.

What is the purpose of practical work?

I have been reading quite a lot about why we do practical work. Laboratory work is a core component of the chemistry (science) curriculum but its ubiquity means that we rarely stop to consider its purpose explicitly. This leads to many problems. An interesting quote summarises one:

One of the interesting things about laboratories is that there has never been definite consensus about those serious purposes. Perhaps that is why they have remained popular: they can be thought to support almost any aim of teaching.1

Even within institutions, where their might be some prescription of what the purpose is in broad terms, different faculty involved in the laboratory may have different emphases, and subsequently the message about what the purpose of practical work is differs depending on who is running the lab on a given day.2

This matters for various reasons. The first is that if there is confusion about the purpose of practical work, then everyone involved will place their attention onto the part that they think is most important. Academics will likely consider overarching goals, with students developing scientific skills and nature of science aspects.3 Demonstrators will think about teaching how to use instruments or complete techniques. Students will follow the money, and focus on the assessment, usually the lab report, which means their time is best utilised by getting the results as quickly as possible and getting out of the lab.4 Everybody’s priority is different because the purposes were never made clear. As in crystalline.

The second reason that thinking about purposes is that without an explicit consideration of what the purposes of practical work are, it is difficult to challenge these purposes, and consider their value. How many lab manuals open up with a line similar to: “The purpose of these practicals is to reaffirm theory taught in lectures…”? The notion that the purpose of practicals is in somehow supplementing taught material in lectures has long come in for criticism, and has little basis in evidence. Laboratories are generally quite inefficient places to “teach” theory. Woolnough and Allsop argued vehemently for cutting the “Gordian Knot” between theory and practical, arguing that practical settings offered their own unique purpose that, rather than being subservient to theory work, complemented it.5 Kirchner picks this argument up, describing science education in terms of substantive structure and syntactical structure. The former deals with the knowledge base of science, the latter with the acts of how we do science.6 Anderson had earlier distinguished between “science” and “sciencing”.

Discussion therefore needs to focus on what this syntactical structure is – what is “sciencing”? Here, the literature is vast, and often contradictory. To make a start, we look to Johnstone who, with his usual pragmatism, distinguished between aims of practical work (what we set out to do) and objectives of practical work (what the students achieve).8 With this in mind, we can begin to have some serious discussion about what we want practical work to achieve in our curricula.

Links to source
Links to source

References

  1.  White, R. T., The link between the laboratory and learning. International Journal of Science Education 1996, 18 (7), 761-774.
  2. Boud, D.; Dunn, J.; Hegarty-Hazel, E., Teaching in laboratories. Society for Research into Higher Education & NFER-Nelson Guildford, Surrey, UK: 1986.
  3. Bretz, S. L.; Fay, M.; Bruck, L. B.; Towns, M. H., What faculty interviews reveal about meaningful learning in the undergraduate chemistry laboratory. Journal of Chemical Education 2013, 90 (3), 281-288.
  4. (a) DeKorver, B. K.; Towns, M. H., General Chemistry Students’ Goals for Chemistry Laboratory Coursework. Journal of Chemical Education 2015, 92 (12), 2031-2037; (b) DeKorver, B. K.; Towns, M. H., Upper-level undergraduate chemistry students’ goals for their laboratory coursework. Journal of Research in Science Teaching 2016, 53 (8), 1198-1215.
  5. Woolnough, B. E.; Allsop, T., Practical work in science. Cambridge University Press: 1985.
  6. Kirschner, P. A., Epistemology, practical work and academic skills in science education. Science & Education 1992, 1 (3), 273-299.
  7. Anderson, R. O., The experience of science: A new perspective for laboratory teaching. Teachers College Press, Columbia University: New York, 1976.
  8. Johnstone, A. H.; Al-Shuaili, A., Learning in the laboratory; some thoughts from the literature. University Chemistry Education 2001, 5 (2), 42-51.

 

Why do academics use technology in teaching?

This week is All Aboard week in Ireland, essayed at “Building Confidence in Digital Skills for Learning”. I am speaking today in the gorgeous city of Galway on this topic, and came across this paper in a recent BJET which gives some useful context. It summarises interviews with 33 Australian academics from various disciplines, on the topic of why they used technology in assessment. While the particular lens is on assessment, I think there are some useful things to note for those espousing the incorporation of technology generally.

Four themes emerge from the interviews

The first is that there is a perceived cost-benefit analysis at play; the cost of establishing an assessment process (e.g. quizzes) was perceived to be offset by the benefit that it would offer, such as reducing workload in the long-run. However, some responses suggest that this economic bet didn’t pay off, and that lack of time meant that academics often took quick solutions or those they knew about, such as multiple choice quizzes.

The second theme is that technology was adopted because it is considered contemporary and innovative; this suggests a sense of inevitability of using tools as they are there. A (mildly upsetting) quote from an interview is given:

“It would have been nice if we could have brainstormed what we wanted students to achieve, rather than just saying “well how can ICT be integrated within a subject?”

The third theme was one around the intention to shape students’ behaviour – providing activities to guide them through learning. There was a sense that this was expected and welcomed by students.

Finally, at the point of implementation, significant support was required, which often wasn’t forthcoming, and because of this, and other factors, intentions had to be compromised.

The authors use these themes to make some points about the process of advocating and supporting those integrating technology. I like their point about “formative development” – rolling out things over multiple iterations and thus lowering the stakes. Certainly my own experience (in hindsight!) reflects the benefit of this.

One other aspect of advocacy that isn’t mentioned but I think could be is to provide a framework upon which you hang your approaches. Giving students quizzes “coz it helps them revise” probably isn’t a sufficient framework, and nor is “lecture capture coz we can”. I try to use the framework of cognitive load theory as a basis for a lot of what I do, so that I have some justification for when things are supported or not, depending on where I expect students to be at in their progression. It’s a tricky balance, but I think such a framework at least prompts consideration of an overall approach rather than a piecemeal one.

There’s a lovely graphic from All Aboard showing lots of technologies, and as an awareness tool it is great. But there is probably a huge amount to be done in terms of digital literacy, regarding both the how, but also the why, of integrating technology into our teaching approaches.

map2
Click link to go to All Aboard webpage

 

On learning “gains”

I have heard the term learning gain on and off but considered that it was just an awkward phrase to describe learning, a pleonasm that indicated some generally positive direction for those needing reassurance. And I am sure I have read about the concept of learning gain in the past but never took too much notice, likely too busy checking Twitter to focus on the subject at hand. But it was through Twitter this week that I was released from my gains-free bubble, and enough neurons aligned for me to grasp that learning gains is actually A Thing.

And what a horrendous Thing it is. HEFCE have a website dedicated to it and define it as “an attempt to measure the improvement in knowledge, skills, work-readiness and personal development made by students during their time spent in higher education”. If that isn’t clear, then how about this: a 125 page document published by RAND for HEFCE define learning gain (LG) as

LG = B – A’

They put it like this too – an equation on a separate line so that the scientists can nod along but the humanities people can just skip to the next piece of text. Good scientists should note that the variables should be italicised. Neither route offers any solace. This seemingly placid text masks one of the most remarkably awful ideas – gosh darn it THE most remarkable fuckwittery – I have come across in a long time.

Devoid of being able to monitor very much about teaching and learning in higher education, some genius has dreamt up this clanger. Rather than wondering why it is difficult (impossible) to achieve this task, the approach instead is to quantify with a number how much individual students have learned each year. A student will be measured at the start of the year, and again at the end of the year, and the difference is… well it’s obvious from the formula – there’ll be gainz. Each student’s own gain will be individual, so these poor lab rats exposed to this drudge are going to have to sit in an interview in a few years’ time and explain to someone what this garbage is about. Not only that, but this insidious beast is going to creep into every aspect of students’ learning. Degree transcripts will list a learning gain of 3.4 in career awareness and 2.9 in self-sufficiency. You were on a student society? Add 0.5 to your learning gain. Is this who we are now? How many centuries have universities existed, and now, at our supposed pinnacle, we have been reduced to this tripe. Don’t trust experts, Michael Gove said. Lots of very clever people are involved in this, and while I hope their intentions are good, I don’t trust the motives behind it. Writing an introduction to a special issue of HERD on measurement in universities, the editors cited Blaise:

A measurement culture takes root as if it is education because we are no longer sufficiently engaged with what a good education is. (HT)

HEFCE have funded several universities to wheel out creaking wooden horses, under the guise of pilot studies. It’s a cruel move, throwing some breadcrumbs at an education sector starved of research money to do anything. A horrible irony, it is, that in the name of ensuring value for money for taxpayers, that money is being spent on this. Don’t those involved see the next act in this tragedy? Institutional comparisons of average learning gains, far removed from the individual it was supposed to be personalised to, are easily computed. And numbers, whether they have meaning or not, are very powerful.

So when your student is sitting with you at the end of the year, and you have to measure their career awareness improvement, and your institution is under pressure as its gains are not as good as their protein-fuelled competitor, most likely a private operator in the dystopian near future the Tories are setting up, you might think again about their ability to discuss career pathways. Worse still is the difficulty in measuring a subject gain, or the even greater idiocy of using a proxy to measure something that can’t be meaningfully expressed as a number in the first place.

Another problem with measuring anything half as complicated as this, as a century of predictive studies has found, is that it is very easy to measure the large changes and the little changes, but the mass of students in the middle is mostly a muddle. And with LG = B – A’, there will be lots of muddle in the middle. What about the very bright student who actually knew most of the content at the start of the year because of prior schooling, but was shy and wanted to settle in for the first year of university. What about the one who enjoys just getting by and will focus in Year 3, but in the meantime has to be subject to self-efficacy tests to demonstrate improvement. What about the geniuses? What about first time in families? What about the notion that university is more than a number and the very darn fact that these are not things that we should try to measure.

I want my students to graduate mature and happy and adult and functioning members of society. If they learn some thermodynamics along the way, then that’s great. I can measure one of these things. It will be a sorry state of affairs if we have to measure others, and our future graduates, burdened with this heap of gunge won’t thank us for it.

Rounding up the peer review and digital badge project

Marcy Towns’ lovely paper from 2015 described the use of digital badges in higher education chemistry, specifically for assessment of laboratory skills. This work was important.  The concept of badges had been around for a while. When I first came across them while doing an MSc in E-Learning back in 2010, laboratory work seemed an obvious place to use them. But while the technology promised a lot, there wasn’t feasible systems in place to do it. And what exactly would you badge, anyway? And would undergraduate students really take a digital badge seriously?

Towns’ work was important for several reasons. On a systematic level, it demonstrated that issuing badges to a very large cohort of students in undergraduate laboratories was feasible. At Purdue, they used an in-house system called Passport to manage the badging process from submission of videos to viewing online for assessment, and subsequent issuing of the badges. But more importantly for me, Towns’ work reasserted the notion of laboratory skills and competencies as something worth assessing in their own right. Skills weren’t being implicitly assessed via quality of results or yield in a reaction. This approach – videoing a student while they demonstrate a technique was directly assessed. This is not something you come across very often in the education literature (some notable exceptions are pointed out in our paper).

This work answered two of the three questions I had about badges – there are systems in place (although as I discovered, Blackboard just about manages to do this, with a lot of creaking).  And there is scope for badging laboratory skills – the concept of badging is built on demonstrable evidence, and videoing techniques is part of this. Whether students take badging seriously I think still needs to be answered. My own sense is that there will need to be a critical mass of badges – an obvious ecosystem where it is clear to students how they can progress, and our own work in this regard is extending into more advanced techniques.

Incorporating peer review

One of the great insights Towns and her students shared at a workshop at BCCE last summer was the notion of narration in demonstrating techniques. Early videos in their initial pilot studies were eerily silent, and it was difficult for them to know what the students’ understanding was as they completed a technique – why they were doing things in a particular way? So they built in narration into the requirements of the demonstration. I think this is one of those things in hindsight that is obvious, but for us to know up front in our own implementation was invaluable.

We opted to have a system where students would video each other rather than be videoed by a demonstrator, and that the narration would be in effect one peer telling the other how they were doing the technique. To facilitate a review of this at the end, the demonstrators in the project came up with the idea of a peer observation sheet (they designed them too – bonus points!). The whole set up was to encourage dialogue – genuine interactions discussing the experimental technique, and allowing for feedback on this based on the guidelines presented in the peer observation sheets. These acted as a framework on which the lab was run. Lord knows chemists like instructions to follow.

Feedback then is given in-situ, and indeed if the student demonstrating feels after discussion that they would like to video it again, they can. This notion of quality, or exemplary work, is underscored by the exemplars provided to students in advance; pre-laboratory videos dedicated to correct display of technique. This whole framework is based around Sadler’s design for feedback, discussed… in the paper!

We’ve documented our on-going work on the project blog and the paper summarising the design and analysis of evaluation is now available in CERP.  It is part of the special issue on transferable skills in the curriculum which will be published in the Autumn, primarily as we felt it developed digital literacy skills in addition to the laboratory work; students were required to submit a link to the video they hosted online, rather than the video itself. This is giving them control over their digital footprint.

Resources

Resources – digital badges, peer observation sheets, links to exemplar videos – are all freely available on the project website.  I really think there is great scope for badges, and look forward to where this project will go next!

Three badges