This post follows up from Keywords and fluency of understanding, a post written before the Autumn Term. It seems apt that I reflect back on the vocab books and weekly testing I have introduced with my Upper Sixth class.
How it looked in the classroom
Every student in my class was issued with a vocabulary book in September. This was specifically for the first topic, Plant Physiology and Biochemistry. The students themselves numbered it from 1 to 35 with two numbers per page. Examples from three different students should make this format clear, see below. I had decided the key words in advance (see appendix for the full list) and – as we gradually covered the topic – gave them words to put in the correct place. E.g. number 13 was the Calvin cycle. For homework students found a definition – handily enough there was a full glossary at the back of their textbook – copied it down and learned it for a vocab test at the start of the next lesson.
Student A – key words 23 to 26
Student B – key words 27 to 30
Student C – key words 31 to 34
Fortuitously the arrangement of other Upper Sixth Biology classes gave this intervention a quasi-experimental design that I could compare my class to the other three classes. The fact that the cohort numbers 40 and I have ten, exactly a quarter, in my class was further good luck. This would allow me to evaluate the vocab books and weekly testing for my class which I will now call the experimental group. All 30 other Biology A level students in the Upper Sixth did not use the vocab books or have regular key word testing, therefore these students will be called the control group.
What happened?
Good question (and for those who like a bit of value-added data (VAD) tomfoolery this will be a real treat!). Firstly, as a basic comparison the table below compares the mean average score for the Plant Physiology and Biochemistry end-of-topic test for the experimental and control groups:
| Experimental Group | 66% |
| Control Group | 64% |
Two percent! I have written before about gains as small as 1%, but is 2% a good result? Ultimately this does not tell us too much. The classes are all mixed-ability by the standards of the school, so it might seem that my intervention has had a positive effect. But students are organised into classes by option blocking of their other subjects, therefore I might have randomly been allocated students whose attainment is higher by chance.
This is where VAD might help. Our students are given ALIS scores which project future A level performance from GCSE results; see the clever people at CEM for more details. So I can now use the projected ALIS grade to compare to student performance in the end-of-topic test to calculate a residual. E.g. if a student is projected an A grade but scores a B their VAD residual is -1, if the same student gains an A* their residual is +1 and if they get their projected grade the residual is 0. The table below shows the average VAD residual for both groups:
| Experimental Group | -0.8 |
| Control Group | -1.2 |
This means the experimental group, on average, scored 0.8 of a grade lower than their projected grade. However, this was better than the control group where students scored, on average, 1.2 grades lower.
Does this mean anything? Well, let us muddy the waters a little by considering that the experimental group has a second teacher (an excellent person and very knowledgeable biologist) who takes them for half of their allocated teaching time. How do the experimental group’s scores compare in the topic they did with me, using vocab books and regular testing, to the topic they did with Teacher 2? The information is tabulated below:
|
Topic |
Mean average score | Average VAD residual |
|
Plant Phys. and Biochem. |
66% |
-0.8 |
|
Genetics |
65% |
-1.1 |
We are still looking at very fine margins but it certainly seems that they have attained better grades, in terms of VAD, for the topic they used vocab books and regular key word testing. However, (and as all good biologists know) correlation does not mean causation. There are a huge number of reasons for this, really very small, discrepancy. For example, is the Plant Physiology and Biochemistry topic the same difficulty as the Genetics topic? Questions like this will continue to be considered as the year goes on.
What did the students think?
They have humoured me and done as asked, producing the vocab books and gamely accepting the vocab tests. They seem to not object massively to the intervention and, as the topic came to an end, were quite pleased with their final vocab books (I think). In fact, they have even suggested making the vocab test harder by mixing up previous key words.
What next?
My class / the experimental group have almost finished their second topic, Energy and Respiration, so I will have further data to consider in due course. However, their points of view are equally important to me so I have arranged a group interview with a few of the students and another separate group interview with a few students in the control group. It will be interesting to consider their responses amongst the data!
Appendix
Key words for the Plant Physiology and Biochemsitry topic:
- Abscisic acid (ABA)
- Absorption spectrum
- Accessory pigment
- Action spectrum
- ADP
- Aerenchyma
- ATP
- ATP Synthase
- Auxin
- Bundle sheath cells
- C3 plant
- C4 plant
- Calvin cycle
- Carotenoid
- Chlorophyll
- Chloroplast
- Dormancy
- Endosperm
- Giberellin
- Granum
- IAA
- Light dependent reactions
- Light independent reactions
- Limiting factor
- Mesophyll
- Palisade mesophyll
- PEP
- PEP carboxylase
- Phosphorylation
- Photoautotroph
- Photolysis
- Photophosphorylation
- Photorespiration
- Photosystem
- Reaction centre
tlamjs 
Good work Mikey. Somewhat inspired by your idea, but also by all the stuff on drill, I’ve been doing something similar with my Y11s. Each Wednesday they have a learning homework and each Friday a test, based on that week’s additions to our master list of key points (which you can see it here: https://padlet.com/sam_pullan/ahd5ealac08b). Most questions are on the new entries, but by no means all. I haven’t done any kind of analysis, but my general feeling is that this year’s set is better able to recall the precise date of Bloody Sunday (the Russian one) than last year’s. Sorry, last year’s.
Also, I agree that a fine margin is worth pursuing, provided (obviously) that the time spent achieving it wouldn’t be better used on something else. I think that’s probably not the case here, in both our subjects: the recall of facts is so critical that anything that makes it easier must be a good thing.