Sunday, October 10, 2010

Accuracy versus Precision - Some Teacher Conceptions

When it comes to lab, measurement is the road to quantitative data. And when it comes to measurement, there is a wealth of issues related to the reliability of the measured data that our students collect. The issues associated with the data our students collect include accuracy, precision, reproducibility, uncertainty, margin of error, calibration, percent error and significant digits. In the United States, with its alphabetical approach to high school science instruction (biology first, then chemistry, then physics), these issues are most commonly addressed in the first few weeks of chemistry class - the first quantitative science class that students are exposed to. It seems to me that much less attention is given to the topic in physics - the second quantitative science class.


The Question in Question
Question #21 on our most recent Honors Chemistry test has stolen a good deal of my mind space during this past week. If I had to guess what question on the test would provide fuel for a blog post, I certainly would not have picked this one. In fact, I would have bet that a question on a chemistry test would never prompt a post in a blog about physics lab work. But for certain, question #2 is a lab question, driving to the heart of the concepts associated with reliability of measurement. Here is question #21 from the test:
Three different measuring tools were used to measure the volume of the same sample of water. The resulting measurements are shown:
Tool A: 120 mL
Tool B: 123 mL
Tool C: 123.4 mL
What conclusion can one make about the three measuring tools?
a. The three tools have varying degrees of precision.
b. Tool A is not as accurate as tool B and tool C.
c. Tool C is the most accurate tool of the three.
d. None of the tools are accurate.

Before you read on in search of my answer, give the question some thought and make a commitment to either an answer or to the verdict that the question is a terrible question. As you think through the question, give attention to your thought process. What are your internal conceptions of accuracy and precision? What images do these terms conjure up in your mind?


Teacher Conceptions of Accuracy and Precision
What captivated me about this question was that it seems that so few of us agree about what the answer is or if there is any answer at all. My first recognition that there was a lack of agreement about the terms accuracy and precision was when I checked my test key against a colleague's. We disagreed about the answer to question #21. Being intrigued (and concerned), I asked two of my respected colleagues what their answer would be. Once more, there was disagreement. The count was 2:2 and my intrigue over the question and the conceptions that we hold regarding accuracy and precision grew. And so I began an informal survey of several colleagues (approximately 13 others) in our science department. I presented them with question #21 and asked them how they would answer it. I found that my science teaching colleagues fell into three categories.

The first category is Category C - those who answered C with very little reservation. As they thought out loud about the question, they commented that the answer is definitely not A. Their conception of precision was that an instrument was precise if the measurements that it took were reproducible. Since there was only one measurement made with each tool, there is no way to evaluate the precision of the three instruments. Multiple measurements with the same tool would be required in order to evaluate the precision of the tool. For Category C teachers, precision had to do with reproducibility and without multiple measurements from the same instrument, there is no way to judge an instrument's precision. Yet because Tool C was able to make measurements with a higher number of significant digits, it was the tool that could make the most accurate measurement.

The second category is Category N - those who responded by saying there is no answer to the question. A couple of teachers within this category responded loudly with That's a terrible question! and Where did you get this question? comments. Like Category C teachers, Category N teachers were unable to make a decision regarding the precision of the tools. Once more, their conception of precision had to do with reproducibility and since only one measurement was made with each tool, their precision could not be compared. But Category N teachers had an additional problem with evaluating the accuracy of the tools since "the real volume" or "the true reading" or "the actual amount" was not stated in the question. For Category N teachers, their conception of accuracy had to do with proximity of a measurement to the true, real, or actual value. When I probed a bit about what was meant by true,real, or actual, I eventually was able to elicit the phrase accepted value from these teachers. Because of the lack of information required to evaluate both the precision and the accuracy of the tools, these teachers regarded the question as a bad question that had no answer.

The third category is Category A - those who answered that A was the correct response. Like Category N teachers, Category A teachers believed that there was a need for knowledge of the accepted volume of the sample in order to judge the accuracy of the tool. As such, these teachers ruled out choice B, C and D as possible answers. But these teachers quickly gravitated towards answer A. Their conception of precision had nothing to do with reproducibility. Their conception of precision had to do with exactness of the tool. The number of division present on the tool was the critical feature that marked a tool as being precise. The more divisions that were present on the tool, the more precise the measurement. For Category A teachers, a tool was precise if it was able to make a measurement to a greater number of significant digits.

Three of the Category A teachers had difficulty comparing the precision of tool A and tool B. For these teachers, it seemed that both tool A and tool B could make measurements of the volume to three significant digits. They admitted that their understanding of significant digits was stale and they were unsure of the rule regarding the significance of trailing zeroes in a measurement. If the topic came up and the rule was clarified (trailing zeroes are only considered significant if a decimal place is present in the number), these teachers decisively chose A as the answer. All three teachers were physics teachers and significant digits are given significantly less attention (if any at all) by physics teachers in our department.


The Dart Board Analogy
The most interesting aspect of my dialogue with the 16 science teachers was the frequency with which they referenced a particular analogy as they approached the question. In our post-survey discussions, every one of these teachers discussed the dart board or bull's-eye analogy. The dart board or bull's-eye analogy is commonly found in chemistry textbooks when discussing the distinction between accuracy and precision. It seems to be the common modus operandi of both textbook authors and classroom teachers in explaining how accuracy and precision are different.


Of the seven chemistry teachers that I surveyed, all but one of those teachers explicitly referenced the dart board analogy. And each one that did was either a Category C or a Category N teacher. Ingrained in their mind was the conception of precision as being equivalent with reproducibility. Only one chemistry teacher was a Category A teacher; when reading the question, this teacher did not conjure up a picture of a dart board, but rather conjured up images of volumetric measuring tools with varying amounts of divisions.


Six of the eight physics teachers that I surveyed were Category A teachers. These six physics teachers did not equate precision with reproducibility. In our post-survey discussion, each of these Category A teachers mentioned the dart board analogy but did not feel that the analogy was significant to the question. One of the physics teachers was a Category C teacher; he quickly referenced the dart board analogy as he reasoned through to his answer. In fact, he commented that the thing he remembers most about chemistry class was the dart board analogy. The other non-Category A physics teacher explicitly defined accuracy (proximity to a target value) and precision (reproducibility) in a manner consistent with the dart board analogy. This teacher was a Category N teacher.


Is the Dart Board Analogy On Target?
My intrigue over this question was intensified by the realization of the power of an analogy on teachers' thought processes. The analogy dominated the thinking of nearly every chemistry teacher as they approached this question. As one colleague put it, "Every chemistry textbook defines precision as reproducibility using the dart board analogy."

So what is this dart board analogy? And why is it so popular? Perhaps the most concise and representative presentation of the dart board or bull's eye analogy can be found at http://celebrating200years.noaa.gov/magazine/tct/tct_side1.html. If unfamiliar with the analogy, take some time to review it if you are unfamiliar with it. As you read through the analogy, ask yourself if you agree with it. Is the dart board analogy on target when it comes to the presentation of the concepts of accuracy and precision? Then come back next week to the Lab Blab and Other Gab blog as I take aim at the dart board analogy.






This article is contributed by Tom Henderson. Tom currently teaches Honors ChemPhys (Physics portion) and Honors Chemistry at Glenbrook South High School in Glenview, IL, where he has taught since 1989. Tom invites readers to return next week as he continues to gab about accuracy and precision. Tom plans to take aim at the dart board analogy and hopes to provide some accurate and precise discussion about topics of measurement.

Saturday, September 25, 2010

The Back of the Room

At this time of the year (the first few weeks of school), I'm thinking a lot about the back of the room. Like most science teachers, it is one of my main objectives during the first few weeks to set the stage for the remainder of the course with regard to the role of lab. Part of the stage-setting process is that I get a little preachy about the importance of the laboratory and its uniqueness to the curriculum.

On the first day of class, I tell my students that "Science class is different than other classes because the room is bigger." I pause long enough for them to show those looks of confusion and perplexity; then I repeat the statement.  After repeating the statement, it is clear from their faces that they need and want an explanation. So I explain that the back wall of our classroom is not located behind the last row of seats like it is in their other classes. Unlike math class, history class and english class, the back wall of our science classroom is located a good thirty feet behind the last row of desks. Lab tables and lab equipment fill the extra 30 feet of classroom space. The activity that happens in that space is what makes science class different than other classes.

Then I explain that the room is bigger in science class because the subject of science is different than other subjects. Compared to their other courses, science is unique. It is this extra 30 feet in the back of the room that makes it unique. Borrowing a line from a colleague, I explain that the answers to our questions are found in the back of the room and not in the textbook. Every trip to the back of the room involves an effort to answer a question. The question must reign supreme in the mind of every student as they cross the threshold between where science is talked about and where science is done.

Herein lies the challenge: forming questions that lead students along a path of inquiry and result in a learning experience in the doing of science. For certain, not all questions are created equal. So what makes a good question? Here are some of my quick ponderings on the topic. I think that good questions share some of the following traits:
  1. Good questions are testable questions; the answers can be found within the lab environment. 
  2. Good questions are interesting questions; they engage students.
  3. Good questions are questions that are clear enough to guide (and replace) the procedure.
  4. Good questions are questions whose answers cannot be found in the textbook.
  5. Good questions emerge from students' own curiosities. 
This past week, we started what is perhaps my favorite back of the room experiences. I call the lab Improving Your Image. While the lab is definitely challenging, it is also one of my students' favorite back of the room experiences. When I watch students do this lab, I feel like science is happening. The question and the purpose that is presented to students is
Question: What is the mathematical relationship between the number of images formed by a combination of two plane mirrors and the angle between the mirrors? 
Purpose: To determine the mathematical equation that relates the number of images to the angle between two plane mirrors. 
As is my usual custom, I discuss the question and the purpose during the pre-lab session; students write the Title and Purpose into their lab notebooks as I quickly made last-minute preparations. To explore the question, I have purchased several inexpensive 1-foot square mirrors from a department store and have taped sets of two mirrors together at their edges using duct tape. I showed students the equipment and demonstrated how the angle can be adjusted. I also showed students a protractor that was available at each lab station. I then asked the class, "What will the procedure involve?" The question/purpose is clear enough to provide the answer to that question. Invariably a student is glad to volunteer the procedure. I then asked the class, "What data will you collect?" Once more, the question/purpose is clear enough to provide the answer to that question. Several hands lifted as students eagerly participated in the pre-lab. In effect that question that is posed at the beginning of the lab guides and even replaces the procedure. For the remainder of the lab, the question (and not a step-by-step procedure) will remain at the forefront of the students' minds.

Curiosity piqued during the pre-lab session as I asked the class, "Have any of you ever done this procedure in your bathroom?" This question got some odd looks, but the odd looks quickly subsided as several students responded with audible O' Yeahs. I asked a responder to describe how they have done this procedure in their bathroom. She explained how they have a mirror on the door of their medicine cabinet which opens up towards a second mirror on the wall. They can open and close the medicine cabinet door and adjust the angle that it makes with the wall mirror. If they place their face between the mirrors, they will begin to see varying number of images with a varying angle. The other responders grinned in agreement. Excitement built as more students recognized that they have done the same thing. I dismissed the students to the back of the room to begin investigating the question.

Students began adjusting the angle between the mirrors and counting the number of images that they could see. As I entered the back of the room, I heard a chorus of wow, cool, and ewww. Interest heightened as students begin to manipulate the mirror angle and observe the multitude of images. As the angle grew narrower, the number of images increased. The photo below illustrates the wowness of the lab. Seeing the multiple images of a single object as you scan the 360-degree panorama is a "this rocks" experience for students.

Septuplets


Students adjust the mirror angle and count the number of images. They repeat the count for a variety of angles, collecting sufficient data that would allow them to answer the question - to determine the mathematical equation relating the number of images to the mirror angle. On years in which I wish the challenge to be easier, I suggest angles of 180, 120, 90, 60, 45, 40, 30 and 20 degrees. And on some years I allow them to choose whatever angles they wish. I almost always follow-up the activity on the following class day with a JAVA simulation that models the formation and location of images for varying angles. (View applet exercise.)

This past week, I entered the lab a few minutes after startup. As I approached one lab table, I overheard three students talking like I've never heard them talk before. They were fully engaged in the question. A sense of enthusiasm could be observed in their voices as they were adjusting the mirrors, counting the images, and pondering the question. I tried to keep my presence unknown as I eavesdropped on their conversation. They were discussing how the mirrors were dividing up the space surrounding the apex into sections and an image was present in each section (except for the section the object was in). They were quite animated in their discussion; they used their arms to form angles and began to point out the image locations. One student began sketching in her lab notebook to illustrate the point she was trying to communicate. As their hypothesis developed, they changed the angle and recounted images in an effort to test it. I knew they were doing science. And I knew they were close to the answer when I heard them talk about dividing 360 degrees by the angle. I quickly left for fear that I'd be invited to help answer their question. They were making great progress and enjoying each minute of it; it was time to scram.

A few minutes later I approached another lab table where one of my students was content at working by himself on the problem. Needing a sounding board, he stopped me and remarked, "Mr. H, I've been thinking about this pretty hard ever since you showed us the question." I love ponderers. And I love to hear that phrase I've been thinking. He sounded out his hypothesis about what the equation was and then paused for confirmation from me. Looking at the data table in his lab notebook, I asked him "What does the data say?" With a classroom-wide grin, he said "The data and the equation fit perfectly." Case closed!

I've been reflecting this past week on why do I like this lab so much?  I believe the answer is that this lab, perhaps more than any of my other labs, demonstrates the power of a good question. If labs are to be done with purpose (and not with procedure), then the question must reign supreme in every student mind as they enter the back of the room. This lab demonstrates nearly every trait of a good question. It is testable and answerable. It clearly engages students. It is clear enough to guide the procedure. It does not lead to a verification experience where an answer found in a textbook is verified in the back of the room. Few textbooks (if any) ever discuss the topic; this answer must be found in the back of the room.

For me, two of the greatest challenges to making the back of the room experience scientifically authentic has to with good questions. There are certainly other challenges, but I immediately think of the challenge of ...
  • ... forming questions that are strong enough to guide students along a path of scientific inquiry towards an answer.
  • ... cultivating an attitude among and developing the ability of students to form questions that are testable within a laboratory environment.
Near the top of this post, I mentioned five traits of good questions. The one trait that is not exhibited by this Improving Your Image lab is that the question does not emerge from the students' curiosity. The question certainly conjures up curiosity, yet I was the one who presented the question. This year I will be making efforts to improve students' abilities to form good questions - the kind that can be answered in the lab environment. Once students gain confidence that answers can be found in the back of the room, then it becomes their turn to ask the question. Once students begin asking the question, the level of inquiry will be raised to the highest level. (See previous post on Using Levels of Inquiry in the Classroom.)

Cultivating this attitude of curiosity and developing this student ability to ask questions will be on my radar screen for the rest of the year. Stop back next month as I report on my first effort at improving student ability to ask a good question.


Sunday, September 5, 2010

Emotions, Pride and Lab Journals

For the past few summers, I spent the week after school ended on a week-long bicycle trip. Although stressful to practically “race” out the door at the end of the school year, the trip benefitted me by serving as a mental re-set button. After a few days on the bike, my mind seemed to accept that I was no longer in school-mode. This June, I biked with my uncle, an established researcher in molecular biology, and an established biker as well. The first day was the hardest. Think hills—lots of hills—and pouring rain. We arrived at the end of the day to find our luggage had a similar experience as us—it was drenched from downpours, too. My uncle pulled dripping wet clothes, papers, electronics, and the like from his bag. He was most upset that his journal got soaked. I responded with horror, too, asking: “did you lose experiments?” The most interesting thing to me was his response. He replied that it had been a blank journal, and that he had intended to come up with some experiments on the trip. I realized that if all that emotion is connected to a BLANK lab journal, there must be a whole lot of emotion connected with a full one! It made me reflect on a few specific students this year.

On the first day of the school year, I gave each student a blank notebook that I purchased out-of-pocket for them. I introduced the course briefly and then sent students to the back of the room to try their hands at their first lab. Alayna, however, came straight to me. She wanted to know if she could switch the color of her notebook. You see, she had already color-coded her classes, and physics simply had to be the color green. Although it had only been in her possession for five minutes, Alayna was already thinking of the journal as hers. Naturally, we made the switch to green. Alayna made the journal hers in other ways. I’m linking here to an image of the first page of the table of contents Alayna chose to create at the beginning of her journal to give you a sense of the effort she put into making the journal of value to herself. Alayna took pride in her journal throughout the year. As one of the only seniors in a predominantly junior course, Alayna left school about a month early to complete a senior project. Alayna was a deeply involved student, so I’m sure leaving high school was poignant for her. She promised to stop by before she graduated, and she told me that she’d bring me a present when she did. When I saw Alayna in June for the promised visit, she gave me my present. She presented me with her lab journal so that I could keep it and find ways to use it to help future students.

Kara is synonymous with cheerfulness to me. There was, however, one incident that caused Kara great distress. I allow my students to use their journals during tests and quizzes. As a co-worker said, students’ journals are “like Google for physics class.” The use of journals on tests and quizzes adds to students’ desires to make a useful journal, and it encourages me as a teacher to think about how to move past fact-based, recall-only questions on tests. There is one exception to my policy of allowing journals during tests. Students who are absent on test day are not allowed to use their journal on the make-up test. I provide an equation sheet to use with the make-up test instead. Kara missed a test day because she was sick, and she had to make up the test without the use of her lab journal. Although generally more chipper than a Disney hero, Kara was always sullen upon recalling this particular test. The test wasn’t out of the norm in terms of her test averages. In fact, it didn’t even impact her grade (I checked to see if there was any difference in her quarter grade had she been simply excused from the test—there wasn’t). Regardless, taking a test without her lab journal ranked as a truly depressing memory for her.

For both Kara and Alayna, the journals were a large part of their physics experience. Both students spent a lot of time making the journal theirs and of personal value, and as a result both students had a lot of pride and emotions attached to their journals. Although traditional assessments do have their place, journal work often provides something different for students. Here are just a few reasons why I believe students often take more pride in lab journals than tests and quizzes:
  • More focus on inquiry. As a teacher, having the kids keep a lab journal forces me to consider whether my labs are the most appropriate for journaling. A lab where kids confirm things they already know or simply enter numbers into a bunch of blank boxes often makes me stop and think: “how could I add more inquiry to this lab?”
  • Novelty. I was told during a workshop this summer that the average student takes over 1600 tests and quizzes between grades one and twelve. Why should my test be anything special to them? A journal, however, is different—I guarantee you no student has written over 1600 journals!
  • Personal choice and creativity. I was reading the book Drive by Daniel Pink and was reminded how important autonomy and choice are as motivating factors. Journals are a tool that, when used well, allows students to determine how to best accomplish their lab goals. This choice fosters creativity, investment, and motivation.
  • Authenticity. Real scientists, like my uncle, use lab journals. I didn’t ask my uncle the last time he bubbled in answers to a multiple choice test, but I’m guessing it was a long time ago.
  • Evidence of progression. A test or a quiz is a snapshot at one instant in time. Often, the only data that is recorded in a teacher’s gradebook is a percentage. Here’s a challenge for any teacher. Look at your gradebook from last year. Let’s say you find a student who got an 80% on a quiz. Can you tell me anything else about the student’s compression? Did that student understand 80% of the material perfectly and know nothing about the other 20%, or did the student understand all of the material somewhat but just not completely? There is not enough data to determine the answer from just the percentage score. Look back at a student’s lab journal and you will have a much better picture of the student’s progress and understanding.
I am not suggesting that lab journals are the answer to everything, or that I have mastered their use. Rather, I hope you will reflect along with me about some of the things lab journals do that traditional assessments do not, and vice versa. I invite you to share your insights in the comments section of this post. It will be like a mini “online journal.” Speaking of, that’s my topic for next time.


Until then, 
Debbie 


____________________________________



This month's article is contributed by Debbie Berlin. Debbie is a graduate of Northwestern University in Evanston, Illinois. She has been a high school physics teacher for 12 years. Debbie currently teaches Regular Physics and Honors Physics at Glenbrook South High School in Glenview, IL, where she has taught since 2004.  

The following teachers follow this blog:

Visit The Physics Classroom on FaceBook