Reflecting on #MICER18

This week I ran the third Methods in Chemistry Education Research meeting (MICER18). It was a really interesting and useful day – we had a good range of speakers and lots of discussion; certainly the scope of the meeting this year was the most ambitious so far. As the meeting is beginning to settle into a space on the calendar, I’ve been thinking a lot about how to take it forward.

Micer Timeline

For me MICER operates on three levels. At one level this meeting has a very simple purpose – to share approaches that can be used when doing educational research, and especially applied to chemistry education research. This is achieved by asking speakers to give talks on how do undertake particular approaches, accompanied by activities and discussion – effectively a series of workshops. Over the series (2016, 2017, 2018) we have covered things like doing interviews, thematic analysis, using Likert scales, designing questionnaires, statistical approaches, thinking about theoretical frameworks and ethics, and the holy grail of writing a decent research question. The purpose is to give insight into the language and processes around educational research for the audience of people with a scientific background, who are moving towards the light. The emphasis on sharing methods of how things were done, rather than what happens as a result of doing things means that this meeting can eek out a little bit of light beyond the shadow of the annual Variety in Chemistry Education meeting. 

A second level is about identity. Most people working as a discipline based researcher in the U.K. and Ireland will likely be doing this as a kind of part-time hobby, in the few gaps available when doing a full time teaching position. Lack of funding means that the discipline is amateur; people are doing things with not much time and less money. If we are to professionalise, people need to feel confident in saying that they are a “chemistry education researcher”, loud and proud. This is very difficult to do if you don’t feel professionally grounded in that discipline. To gain that confidence, there needs to be a community into which they can situate themselves, and a sense of personal expertise to allow them to make the claim. By sharing methods and approaches, and demonstrating that there is a community, the meeting aims to help raise this confidence. This year we included a “reports from practice” section; hearing from people who are just like us doing CER in their own situation. It was wonderful, and a real highlight of the day, a kind of showcasing of what real live chemistry education researchers look like. I was also struck this year by the number of people in the room who spoke about projects they are working on, or where they were situating themselves on the spectrum of evaluation, reflection, and research. This highlights to me that the landscape is shifting slowly. But there needs to be considerable support; financial yes, but also in terms of identity. This is something the RSC needs to grapple with firmly. 

Finally, the entire education landscape is shifting. The growing emphasis on teaching at third level means that those situated in a teaching and scholarship roles are thrust into a political ping pong. At school level there is an expectation that teachers will engage with education research with little support or guidance, save for some grassroots heroes. We aimed to address this head on this year with a keynote talk about this very landscape; one which I think well characterised it and also offered clues as to how we might navigate it. We also included a talk on managing student projects; standing firm in a world where in some institutions, the presence of such projects is contentious. 

MICER19

In setting out on the MICER journey, I was only ever really concerned with (and indeed thought about) the first level. I knew from conversations that people wanted to know about the how of doing education research. The additional levels have grown, partly thrust upon us as a nascent community, partly necessary for us to be a community. But there is a danger that in aiming to do everything, the meeting tries to do too much, and as a result, does not achieve the sum of its parts. It is only a one-day meeting after all, and I am left wondering whether we should refocus our thoughts on the first part, and the others will work themselves out elsewhere (come on RSC!). 

I’m planning to send out feedback survey and hope to use that to guide the focus of future meetings. There are other less lofty considerations; the meeting was full by February and likely needs a bigger venue. The characteristics of attendees is broadening. While the registration cost is cheap, getting to London is expensive. Getting funding to support the meeting is getting more difficult; there is a limit to the hit that the supporting interest group budgets can manage.  

But the interest in, and outcomes of, the meeting mean that I think the effort will be worth the while. 

 

A memorable teaching scenario for #Chemedcarnival

Katherine Haxton has challenged us to write a blog post on a memorable teaching situation.

When I was 19 I attended a Scout training weekend as part of a course to become a canoe instructor. I had been canoeing for several years under the patient instruction of canoe instructor, Keith, who was also a former scout leader. Keith is tall and patient and a scientist and has a clipped English accent with excellent projection. “Lean downstream, Mick” he’d boom up and down rivers, while I’d lean upstream, and capsize.

All this made Keith very exotic. Coming from a small country village where everyone mumbled and there were no scientists, much less well-spoken English ones, he was very different. We adored him. He had rare qualities of being The Adult but never condescending, paternal but never patronising. We mimicked him constantly (out of earshot) but woe betide anyone who even hinted a bad word about him.

So the country fellows went to training course to learn how to be leaders. The teachers on the course seemed tough and scary and, well, from Dublin, which is to say they were under the influence most of the time. When we arrived we found out that Keith was going to give one of the sessions on this weekend of the course. This is more than half a life ago, but I recall the excitement that news brought. We were used to him in the context of our own canoeing, going down (and upside-down) the rivers of Wicklow, but now we would see him somewhere different. We knew him and these chaps from Dublin didn’t. He was ours and we were loaning him to the mob and he would be amazing.

He was amazing. The session was about safety, and the kinds of decisions that need to be made quickly when on rivers. He was clear and authoritative and we sat and listened in total silence. Bursting with pride. Everything he said made sense.

But then; one of the Dublin fellows shouted out in the silence: “But Keith – that’s WRONG!” You can guess our horror. Keith listened, and responded, and moved on. And then; another interruption!

A game was afoot. Even for simple country chaps, it was clear that the whole lesson had been structured, with planned interruptions prepared well in advance. Tension eased, we all played along, throwing out ideas and suggestions and discussing various scenarios and decisions.

It is a “teaching moment” that has always stuck with me. There aren’t many lessons from half a life ago that I remember so well. A few months ago I was in the newly refurbished National Gallery of Ireland, and wandered into a wing featuring some new artists. I came across this portrait of our hero. The link explains more.

ROBINSON, N-770

The Likert Debate

David Read blew my cover on Sunday night with a tweet mentioning my averaging of Likert data in our recent work on badges. If there is ever a way to get educational researchers to look up from their sherry on a Sunday evening, this is it.

Averaging Likert scales is fraught with problems. The main issue is that Likert response is ordinal, meaning that when you reply to a rating by selecting 1, 2, 3, 4, 5 – these are labels. Their appearance as numbers doesn’t make them numbers, and Likevangels note correctly that unlike the numbers 1, 2, 3… the spaces between the labels 1, 2, 3… do not have to be the same. In other words, if I ask you to rate how much you enjoyed the new season of Endeavour and gave you options 1, 2, 3, 4, 5 where 1 is not at all and 5 is very much so, you might choose 4 but that might be just because it was while it was near perfect TV, it wasn’t quite, there were a few things that bothered you (including that new Fancy chap), so you are holding back from a 5. If you could, you might say 4.9…

But someone else might say well it was just better than average, but only just mind. That new fella wasn’t a great actor but, hell, it is Endeavour, so you put 4 but really if you could you would say 3.5.

So both respondents are choosing 4, but the range of thought represented by that 4 is quite broad.

I can’t dispute this argument, but my own feeling on the matter is that this is a problem with Likert scales rather than a problem with their subsequent analysis. Totting up all of the responses in each category, we would still get two responses in the ‘4’ column, and those two responses would still represent quite a broad range of sentiments. Also, while I understand the ordinal argument, I do feel, that on balance, when respondents are asked to select between 1 and 5, there is an implied scale incorporated. One could of course emphasise the scale by adding in more points, but how many would be needed before the ordinal issue dissipates? A scale of 1 to 10? 1 – 100? Of course you could be very clever by doing what Bretz does with the Meaningful Learning in the Lab questionnaire and ask students to use a sliding scale which returns a number (Qualtrics allows for this more unusual question type). Regardless, it is still a rating influenced by the perception of the respondent.

Our badges paper tried to avoid being led by data by first exploring how the responses shifted in a pre-post questionnaire, so as to get some “sense” of the changes qualitatively. We saw a large decrease in 1s and 2s, and a large increase in 4s and 5s. Perhaps it is enough to say that; we followed the lead of Towns, whose methodology we based our own on,  in performing a pre-post comparison with a t-test. But like any statistic, the devil is in the detail, the statistic is just the summary. Conan Doyle wrote that “You can, never foretell what any one man will do, but you can say with precision what an average number will be up to. Individuals vary, but percentages remain constant. So says the statistician.

There is a bigger problem with Likert scales. They are just so darned easy to ask. It’s easy to dream up lots of things we want to know and stick it in a Likert question. Did you enjoy Endeavour? Average response: 4.2. It’s easy. But what does it tell us? It depends on what the respondent’s perception of enjoy is. Recently I’ve been testing out word associations rather than Likert. I want to know how student feel at a particular moment in laboratory work. Rather than asking them how nervous they feel or how confident they feel, I ask them to choose a word from a selection of a range of possible feelings. It’s not ideal, but it’s a move away from a survey of a long list of Likert questions.

When is a conference real?

Respected Dr Dr Mrs Seery, we hope that you can come to our conference in somewhere you’ve never heard of and tell us about your interesting and exciting work in Pre-Lecture Resources for Reducing Cognitive Load at our Conference on Chemistry and Chemical Engineering in Sub-Oceanic Conditions. Please reply.

Most of us now receive daily invites to conferences around the world – oh the travel we could do! – and the usual fare is a greeting like that above; a dodgy mail merge of incorrect title, a paper title you have published and a conference that has a theme that bears no resemblance to the topic that you have been on. But the targeting is getting cleverer, and there are now quite a few Chemistry Education “conferences” doing the rounds.

These conferences are organised to make money. The model is that academics are invited to speak at conferences, and they, like all attendees will pay to attend.  The organisers know nothing about the topic, and the conference will not have any coherent theme, but the organisers will have delivered on their promise to host a conference, and gather all the money raised in the process as profit. Academics will provide free labour by presenting at the conference, perhaps peer-reviewing, being members of the “Advisory Board”… It all mirrors an actual conference very closely, but of course the problem is that the themes of these “conferences” are so broad that little meaningful discussion could take place. So how do you know what is real or not?

Three key places to look are: (1) who is on the advisory board (2) is there a professional body underpinning the conference, and (3) what are the conference themes.

ICCE2018Organisers and Advisory Board

If you are going to a conference on chemistry education, and the advisory board is populated by Professor of Forestry from Transylvania, then an alarm bell should ring. Are the names familiar? If you were to Google some of them, would you come up with some CER publications? Pictured are the Local Organising Committee and International Advisory Board of the very real 25th ICCE 2018 conference happening in Sydney in 2018. A cursory glance at this list for anyone involved in chemistry education would show that these are people with a genuine investment in the discipline.

Sadly, this check on authenticity is becoming more difficult, as academics are bounded by a singular characteristic: we love doing things for free. So when you get an email that asks you if you want to be part of an organising committee for a conference whose title interests you, well why not? If you don’t look into it too much and you’ve always wanted to go to the Mongolian mountains for a hike, this fits the bill. Before you know it you are profiled on the conference website and credence is added to the meeting because of your affiliation.

Professional Body

A second thing to check is if there is a a professional body underpinning the conference. The very real 25th ICCE 2018 conference happening in Sydney in 2018 is being organised under the auspices of IUPAC, as have all of the conferences in the ICCE series, and the national chemistry body, RACI. This lends an air of authority to the meeting – these are professional bodies who are interested in promotion of chemistry education, rather than just out to make money.

ICCE2018logo

Conference Themes

But what if there is a conference that is out to make profit but means well and wants to host a good conference on a particular theme, where it has identified a gap. This isn’t illegal or morally wrong. We can use the conference themes to get a sense of how invested the organisers are in organising a conference about a topic that will bring a lot of like minded individuals together. I’ve pasted below an image from a tweet from the organisers of the “8th Edition of International Conference on Chemistry Education and Research” (note the ‘and’).

DPzbt2WVwAYUwyO

Exercising Judgement

It is in the interest of organisers of conferences such as these to spread the net widely; the more themes they cover, the more people will likely match. But of course, the broader the net, the more useless the meeting will be. It is worth exercising some judgement by considering the three points above. Even the conference title needs consideration: including an “and” is very popular as it allows a much broader range of topics while sounding like another very well established conference. Compare:

  • 25th IUPAC International Conference on Chemistry Education (ICCE2018)

with

  • 8th Edition of International Conference on Chemistry Education and Research

At a passing glance in a busy email reading session, both look similar.

Many readers of this will likely have received an invitation from the “Journal of Chemistry: Education, Research and Practice” and perhaps confuse it with the journal Chemistry Education Research and Practice. In this case, punctuation reveals very different intentions.

Take care, and if you do go to one of these “conferences”, I hope the scenery is nice!