Creating the learning design for a first year, compulsory unit can be a challenge, particularly when the subject matter is difficult. In 2008 Jackie Reid received academic renewal funding to take on such a challenge for STAT100, a unit she had actually created herself almost 10 years prior. Although she had taught the unit for many years, Jackie took a break between 2003 and 2007, and found this allowed her to look at the unit with fresh eyes in the review process.
The Challenges for STAT100
A major challenge when teaching STAT100 is motivation – students don’t understand why they have to do it. Most have very little exposure to statistics in high school, so when they come into university they think “I want to be a zoologist or an environmentalist or whatever, so why do I have to do statistics?” This is likely to apply to every new group of enrolling students, so addressing motivational issues needs to be built into the unit.
The second challenge is that fundamental concepts in statistics are challenging for students. There is new terminology, and new concepts such as variability, sampling distributions , and inferential statistics. Statistics is about making decisions in the face of uncertainty. When students are used to calculating exact answers in mathematics, the uncertainty of a probability can be hard to understand. Not only do students need to understand it themselves, but they need to be able to communicate their understanding to others.
Finally there is the challenge of learning to use specialist statistical software. The software used most recently was powerful and freely available on the web, but proved challenging for students to learn. This meant students often focused on the process of trying to analyse the data, rather than the meaning behind the problem and the interpretation of the results.
Building the New Learning Design
The unit redesign was guided by a working group consisting of people from the various disciplines that had STAT100 as a core unit, a TLC representative, and, of course, Jackie herself. Student input came from student ratings and comments from end of semester surveys over several years.
STAT100 was thoroughly revised, with improved and increased learning resources, and learning activities that scaffolded student learning more effectively. Improvements to the learning resources included: addition of a new textbook, a revised study guide that complemented the textbook, a greater range of practical examples, demo videos for the software, and student workbooks for the practical sessions. Improvements to the learning activities included: online quizzes to reinforce the fundamentals of terminology and notation, restructuring of practical exercises to ensure basic skills were developed before progressing to more complex skills, and use of the workbooks to help build skills in interpreting results. To help provide off campus students with the on campus equivalent of a tutor led practical session, an online workbook was developed, which at the end of each section, would display model answers, thereby providing instantaneous feedback.
Is it Working?
The changes have been introduced gradually into the unit – last year saw the introduction of the new textbook, study guide and online quizzes, with the online practicals and software demo videos introduced only this semester. However with the basics in place last year, three of the four cohorts showed improved student feedback ratings, and attrition both last year and this year have reduced. Jackie expects that with the new resources introduced this year student feedback ratings will improve again.
To get more detailed information, extra questions about the quizzes were added to the unit survey last year . Students were very positive about how these quizzes helped their learning. At the end of 2008, one of the online pracs was trialled on a voluntary basis by a handful of students. Feedback was very positive and so Jackie completed all those required for the first half of this semester. Once again students responded favourably along the lines of “I find they have been a great confidence builder and being able to check answers straight away is excellent. I hope you can do them for the rest of the chapters.” Jackie said “It would have been impossible to do this level of work without the Academic Renewal funding, and it was a huge amount of work over and above what the funding covered. So when a previous student came up to me the other day and said ‘Jackie, I don’t hear negative reports about STAT100 anymore. What’s going on?’ it makes it all feel worthwhile.”
Read on for more about how the learning design was modified to address the three key challenges
Challenge 1: Building Motivation
In the on campus STAT100 class, students are asked in the opening lecture “Why do you think you are doing this unit?” At least 90% say it is because to the unit is compulsory, not because they think stats sounds interesting. This can lead into a discussion about statistics: What is statistics? Why is statistics important? Why do students think they are being made to do it in their Bachelor of Natural Resources, or Rural Science or Science? How are they going to use it? This helps students think right from the start about why they might be doing this subject, rather than just seeing it as a series of hoops to jump through.
Examples are chosen to be relevant to the agricultural scientists and the zoologists and so on, but in the introductory unit the examples are still quite basic. Unfortunately it is not until the students start doing their own research that they really want to know more. Jackie said that’s when the third and fourth years come back to her saying “I wish I’d done more stats”, and these are the same students who, when they were in first year, were saying “Why do we have to do this?”
Challenge 2: Developing Understanding
Students need to develop a good understanding of a number of fundamental statistical concepts. Jackie considers the understanding of variability to be a key difficulty, and therefore focuses her own research in statistics education on this area. She says understanding variability is one of the key struggles students have, “because they come from mathematics with a fairly deterministic viewpoint and to understand that statistics is about probability and variability is a big leap conceptually.” Consider medical diagnostic testing for instance, which most of us assume is 100% accurate. It isn’t. Consider swine flu. The test you take to see if you have swine flu isn’t 100% accurate. You may get a false positive result (i.e., the test result says you have swine flu when, in fact, you don’t), which is not so problematic unless you’re a Rugby League player being asked not to play. More concerning is a false negative result (i.e. the test result indicates that you do not have the flu when, in fact, you do): the person tested could come out of isolation and resume their normal activities, unknowingly infecting others. Tests therefore place higher emphasis on minimizing false negative results over false positive results, but it is important to understand the result is still only a probability.
The learning resources were broadened to include an excellent textbook and practical session workbook. The study guide has a summary for each chapter of the text, which identifies relevant sections in the textbook to read, summarises and reinforces the main concepts, provides examples on using the software to analyse example data sets, and identifies other activities such as practical sessions or online quizzes that need to be completed.
The unit now incorporates five online quizzes designed to help students learn fundamental terminology and notation. In order to maximize learning (advertised to the students as maximizing marks) students have two opportunities to complete each quiz. This allows students to revise weak areas and improve on their second attempt. Students appreciate this opportunity for formative learning. The quizzes were interspersed between practical sessions and assignments so application of concepts was immediate.
Students are provided with a workbook to guide their practical sessions. Previously students would come to the session, work through the task, and write down the results produced by the software, without spending time thinking about what those results implied. This meant the focus was on obtaining the result rather than understanding the problem, the meaning of results, and how best to communicate their findings. The questions in the workbook have been designed to not only help them develop skills in using the software but to encourage them to think about the results all the way through the session. There are questions before they analyse the data to get them to think ahead such as: How would you analyse this? What results or outcome might you expect? Why? They are also asked to compare their predictions with their results and resolve any discrepancies. At the end they are asked to write a meaningful conclusion in the context of the problem. By ensuring the questions are answered in the workbook it means students have a learning resource available when they do the assignments, and in particular they are learning to communicate their understanding of statistical results.
In a real project, the ability to communicate in everyday language that relates back to the context of the problem is vital. The marking of assessment tasks communicates this importance. Marks are not only allocated to the reporting of the results but also to a well-written informative conclusion that does not use statistical jargon. Students can use these skills to report a real project with an executive summary up the front that summarises the implications for this experimental study with the details of the results in the body of the report. The implications might be to invest in a particular variety of wheat, provide more education in a particular area, or market a new drug.
Software
Software has always been an issue. It is an extra financial burden if students have to purchase software as well as a text. In 2003 we started using a software product called R which had excellent functionality and was free. On the downside it had a command line interface (like MS DOS for those of us who remember it) meaning it could take students a long time to learn how to use it. In 2006 a menu driven interface (Rcmdr) became available which alleviated a lot of usability problems, but there’s still some difficulty with installation for off campus students (on campus students can use the computer labs). And for many students, learning to use the software can distract their attention from understanding the question and interpretation of results.
With the unit revision Jackie therefore aimed to make installation and learning to use the software easier, and to encourage students to focus on meaning as they were working through a problem. The installation problem has been solved as a completely automated installation wizard was developed. Results this semester suggest that this innovation has decreased the number of queries from off-campus students at the beginning of the semester. To aid learning, a series of video clips about the software were created – from “How to Open R”, “Introduction to Rmdr”, through to using a range of specific functions. The latter are directly referred to in the practical workbook as required.
The new practical workbook, as well as being available in paper form, was also developed into electronic form. This facilitated embedding of video links, but more importantly facilitated immediate feedback after each practical session – for off campus students it is almost as good as having a tutor available. Question forms include multiple choice, direct number entry, or short text answer responses (once again encouraging written communication). The first two can be marked directly, but as that’s not possible for the short text answers, a button was added that says “show me the answer” whereby a model answer is displayed. This is a vast improvement over the previous practice of posting solutions at the end of the week. One student said “direct feedback straight away is useful because [it] makes you realize your mistake before you remember how to do it wrong..”
Finally, the requirement to submit assignments electronically meant we faced the problem of how students submit equations. It can be done in MS Word but it is not intuitive. So another short video was developed which shows students how to do use Microsoft equation editor.
Jackie will be demonstrating some of these innovations at the Faculty of Arts and Sciences Teaching and Learning Showcase onFriday 14 August 2009