At last – a way to quantify if you are asking a nasty question or not!
Problem solving imposes a cognitive load on novice learners. Even if the problem is simple (often called exercises, if they involve routine algorithmic tasks), the learner will need to recall how to approach each stage of the exercise in order to solve the entire problem. Thus the question arises: if a problem involves several tasks, does each one add to the cognitive load? Which ones do learners find difficult.
This question was addressed for Gas Laws in an interesting paper in Journal of Chemical Education. The authors took typical gas law questions, and determined what processes were required to solve them. Each of these processes had a level of difficulty. For example, sometimes the number might be presented in scientific notation, or sometimes a unit change was required. The authors listed five variables that could be distinguished:
- The gas identity: whether it was “an ideal gas” or a “mixture of gases” or an “unknown gas”.
- The number format: whether it was general (1.23) , decimal (0.0123) or scientific (1.23E-2).
- The unit change required in volume: no change (L to L or mL to mL), or conversion mL to L, L to mL.
- The unit change required in temperature.
- The units of pressure.
From this, questions of different complexity could be derived using all of these variables – in total a possible 432 combinations. These range from easy questions where no conversion was required, to difficult questions where conversions were required. The authors then analysed several thousand answers from chemistry and non-chemistry major students in their first year. Based on what was involved in each question, they could determine what was causing least and most difficulty. Read paper for lots of statistics—I’m going to highlight the results.
The results are interesting for two reasons: they identify for this particular set of questions what variables caused most difficulty, and more so, the authors generate a cognitive load increment for all items ranging from those which don’t cause a significant cognitive load to those that do. I think it is an interesting way to present the data. The load was given a rating of 0: no additional load increment; 0.25: small effect; 0.5: medium effect; 1: large effect. The total cognitive load increment for a question is determined by adding up the individual components.
Of the five variables listed above, only two showed significance in the analysis.
- The number format: If the number was in scientific notation, this caused difficulty (a cognitive load increment of 0.5). Decimal format had a smaller effect (0.25).
- The volume conversion: interestingly, if students were given L(itres) and need to convert to mL, this had a significant load associated with it—the largest observed. The conversion in the other direction also had a significant load, although the former was perceived to be more difficult, as it involved dividing. Both were assigned a load increment of 1.
- Additionally, there was marginal significance for the temperature value – providing and requiring °C (i.e. have to convert through K). this had an increment of 0.5.
Thus a very easy question would be that shown below (given in the paper – complexity factors in bold). No conversions are required and the number format is general. This has a cognitive load increment of zero according to the authors’ scheme.
An ideal gas occupies an initial volume of 6.22 L at a temp of 262 L. What is the final volume in units of L if the temperature is changed to 289.6 K while the pressure remains constant.
On the other hand, this whopper is a hard question. Scientific notation and unit changes abound, and this has a cognitive load increment of 2.25:
A mixture of ideal gases (…) occupies an initial volume of 3.21 x 106 mL at a temperature of 62.8 °C. What is the final volume in L(itres) if the temperature is changed to 89.6 °C while the pressure remains at a constant value of 1.2 atm.
Now its time to analyse our gas law questions – are we being too easy or too hard?!
J. D. Schuttlefield , J. Kirk , N. J. Pienta , and H. Tang, Investigating the Effect of Complexity Factors in Gas Law Problems, Journal of Chemical Education, 2012, 89, 586-591. [Link]