Tuesday, October 21, 2014

Conversations on the Rifle Range 12: Teaching to the Authentic Assessment

Barry Garelick, who wrote various letters under the name Huck Finn, published here, is at work writing what will become "Conversations on the Rifle Range". This will be a documentation of his experiences teaching math as a long-term substitute. OILF proudly presents episode number 12:



Back in September, when I was doing my sub-assignment for the high school, I attended a math department meeting the day before school began. Sally from the District office presided, and among the many things she told us at that session was that this year the students in the District would not have to take what is known as the STAR test, by order of the superintendent of the District. “And as you know, the Superintendent is like the Pope. What he says goes.”

While this last was uttered partly in jest, the reaction in the room was celebratory. The STAR exam has been an annual ritual in California and in May of each year about two weeks are devoted to a review and prep for this test, which is keyed to California’s pre-Common Core math and English standards. Such activities inspire accusations that schools are “teaching to the test”. But now in the midst of a transition to implementing Common Core math standards, California was looking at the Smarter Balanced Assessment Consortium (SBAC) exam that would be given officially starting the following school year. (Actually, they don’t call it an exam; they call it an “assessment”. You’ll forgive me if I call it an exam.) For now, however, the state would be field testing the exam. What this meant was anyone’s guess: perhaps this first go-round on SBAC would be to provide a baseline to see how students scored prior to full implementation of Common Core. Or perhaps it was to fine tune the questions. Or both. Or neither.

In any event, when I started my new assignment at the middle school, I had to have my classes take a practice SBAC exam. The day before I was to take all my classes into the computer lab for the practice exam, I attended an after-school faculty meeting.

I had started my assignment at the school earlier that week, so the principal introduced me to the group. I was welcomed by applause, and urgings by fellow teachers to help myself to the tangerines that were brought in for the occasion. I took two tangerines, and as if he were taking that as his cue, the principal started the discussion.

“As you know, we are in transition to the Common Core, and one aspect of this is the SBAC test that will be given this spring. We want students to have a chance to practice with some sample questions. This does two things. It will get them used to the computer interface, because the test is given entirely on computers. And secondly it will get them used to the questions which are not the typical multiple choice question like on the STAR tests. The SBAC is more of an ‘authentic’ test.”

He went on about how Common Core will change the way we teach, but the words all blurred together in my mind amidst phrases like “critical thinking”, “higher order thinking”, and “deeper understanding”. I do recall a conversation between two teachers at my table. One mentioned she saw some of the questions and said "Yes, there are still multiple choice questions on the test. I was very disappointed to see that."

Well, OK, I like open response questions too, but I get rather tired of the “it’s inauthentic if it’s multiple choice” mentality. I took the CSET math exam required in California to be certified to teach math in secondary schools. The multiple choice questions were not exactly easy; I would hesitate to call the exam “inauthentic”. What I find inauthentic is judging seventh and eighth graders’ math ability based on how well they are able to apply prior knowledge to new problems that are substantially different than what they have seen before or have worked with.

On the day of the practice exam the assistant principal took charge of my first group—the first of my three pre-algebra classes and probably the most cooperative of all of my students. He spoke in a loud, commanding voice and gave instructions on how to log on, what page to go to, what things to click on, and had everyone do things at the same time. I only know that I could not duplicate this feat for any of my classes; students would rush ahead, ask me to explain again what I had just said, and inevitably asked “Will the test affect our grades?” I explained that it was for practice and did not affect their grades, nor would the actual test given later in the semester, but the question kept coming up. When it came time to take my fifth period algebra students to the computer lab, I had written on the white board: “No, this test will not affect your grades.”

A boy named Peter exuberantly agreed. “Yes, Mr. G, that’s a great idea, because…” I couldn’t hear the rest amidst the noise of the class, which then followed me outside the classroom to the computer lab. The instructions had to be repeated several times, as I had done throughout the day.

Because this was a practice test, I felt no compunction about giving students help in answering the various questions. For the most part, questions were reasonable, though the students found some difficult. One question I recall on the seventh grade test was “Enter the value of p so that 5/6 - 1/3n is equivalent to p(5-2n). Seventh graders have only learned about how to distribute multiplication, but not to factor. While the answer of 1/6 seems to jump out at adults, this problem presented difficulty to most of my seventh graders, probably because they hadn’t seen anything like it. The variable “n” also was a distraction. I gave them hints like “Can p be a fraction? What fraction would you multiply 5 by to get 5/6?”

On the eighth grade test, one open-response item was quite complex, involving pulse rates versus weights of various animals, which students had to analyze in terms of slope and a “trend line”. One of the questions was “Interpret the slope of the line from Item 1 in the context of the situation.”

At the end of sixth period, I dismissed the students, and went back to my classroom. I realized that when Common Core kicked in students would be “taught to the test” for these all of these particular types of questions. I have no problem with teaching to a test if the test covers material that should be mastered. I do have a problem when part of this is learning how to write explanations that will pass muster according to scoring rubrics.

As I got ready to leave for the day, one of my sixth period students popped her head in the door. “Mr. G, will the test today affect our grade in the class?” I said it wouldn’t, but not for the last time that week.

Sunday, October 19, 2014

Teaching what can be discovered instead of what can't be

Barry Garelick's recent piece Education News, Undoing the ‘Rote Understanding’ Approach to Common Core Math Standards, got me thinking about one of the things that Reform Math has backwards.

Barry talks about the emphasis by today's math educators on ad hoc methods like "making tens" at the expense of traditional algorithms like borrowing and carrying. And so we see more and more worksheets like this one:



And fewer and fewer like this one:


And, in promotional, Common Core-inspired videos like this one, we see just how painfully slow the "making tens" method can be--as well as how it, by itself, does not give students a general method for solving more complex addition problems.

As I wrote in a comment on Barry's piece, I don't remember ever learning officially how to make tens. I remember it instead as something I discovered on my own--in the course of computing all those long columns of sums that students used to be assigned (sometimes upwards of six addends!) and eagerly looking for shortcuts.

The standard algorithms, on the other hand, I most certainly did *not* discover on my own, and am quite glad to have had teachers that were willing and able to teach it to me.

It's ironic how "discovery-based" Reform Math spends more time showing students how to do stuff they might discover on their own than it spends showing them how to do stuff they almost certainly won't learn on their learn own.

Friday, October 17, 2014

Math problems of the week: Common Core-inspired Algebra 2 problems

Two problems from the Spring 2013 North Carolina Measures of Student Learning Common Core Algebra II Exam:




The majority of the 28 problems on this exam, I should note, are straight-up algebra problems, refreshingly free of the excess verbiage and "real life" detail that bog down so many Common Core-inspired math problems.

Wednesday, October 15, 2014

The Finnish Fallacy: drawing the wrong lessons from our favorite international comparison

Last week, CNN ran an opinion piece by Pasi Sahlberg, the former director general in the Finnish Ministry of Education and Culture (and now a visiting professor at Harvard's Graduate School of Education) explaining “Why Finland’s schools are top-notch.”

This is not the first piece that Stahlberg has written for American readers, and one likely reason he’s been getting so much attention is that what he tells us so exactly matches what so many of us want to hear. What makes Finnish schools so great, it turns out, is that they focus more on funding, educational equity, child-centered play, and “the whole child,” and less on testing and “narrow academic achievement”:

There are three things that have positively affected the quality of Finnish schools that are absent in American schools. First, Finland has built a school system that has over time strengthened educational equity. This means early childhood education for all children, funding all schools so they can better serve those with special educational needs, access to health and well-being services for all children in all schools, and a national curriculum that insists that schools focus on the whole child rather than narrow academic achievement.
Stahlberg doesn’t mention that the U.S. spends more per pupil than Finland does, and, in particular, a great deal on special education. Nor does he reconcile the claim that Finland has early childhood education for all children with the fact that Finns famously don’t start school till age 7. As for the implication that U.S. schools are, by comparison, narrowly focused on achievement, he doesn’t mention that Finnish schools lack sports teams, marching bands, and proms.
Second, teachers in Finland have time to work together with their colleagues during the school day. According to the most recent data provided by the OECD the average teaching load of junior high school teachers in Finland is about half what it is in the United States. That enables teachers to build professional networks, share ideas and best practices. This is an important condition to enhancing teaching quality.
But is the only factor? Is networking even the most important factor in teacher quality? Stahlberg doesn’t mention here that Finland recruits its teachers from the top 10% of college graduates, while only 23% of U.S. teachers come from even the top third of college graduates.
Finally, play constitutes a significant part of individual growth and learning in Finnish schools. Every class must be followed by a 15-minute recess break so children can spend time outside on their own activities. Schooldays are also shorter in Finland than in the United States, and primary schools keep the homework load to a minimum so students have time for their own hobbies and friends when school is over.
I agree with Stahlberg that American kids need many more 15-minute outdoor recess breaks, and that our primary schools should assign much less homework. But there are a couple of important distinctions he omits. First, if you include indoor recess, in-class games, and indoor shows and movies—much more common in U.S. schools than elsewhere—American students are getting many more breaks than it first appears. The Finns send their kids outdoors in all kinds of weather; so should we. And if we simply reduce the passive, couch-potato breaks from learning, we can increase the time available for true recess without reducing the time available for true learning.

Secondly, there’s work, and then there’s busywork. American homework is notorious for its busywork components. We can reduce our homework load substantially without decreasing its educational value—simply by reducing the cutting, pasting, coloring, illustrating, assembling, diagramming, and explaining.

Stahlberg goes on to identify three problems in American education that make things worse: excessive testing, school choice, and novice teachers. Nowhere does he suggest that there might be any problems with our educational curricula.

And nowhere does he reference what Finish exchange students have said about the American school system, for example, that much of the high school homework resembles elementary school assignments—e.g., making posters--or that high school tests are often multiple choice rather than essay-based, or that, upon returning to Finland, they ended up having to repeat the school year.

One indication of just how much higher Finland’s academic expectations are is seen in the one big high-stakes exam Finnish students have to take—a refreshing contrast to our Common Core-inspired tests. For that, stay tuned for a later post.

Monday, October 13, 2014

The Tutoring Fallacy: where clear explanations fall short

In the toughest math classes I took in college, it happened a couple of times that a certain classmate of mine would ask me to explain what was going on. He seemed to have more trouble understanding the material than I did, and my verbal explanations seemed to help him understand it better. So much better that he, now an accomplished engineer, went on to score higher than I did on all three class tests.

For many Reform Math acolytes, the ability to communicate your reasoning to others, and to talk about math more generally, is the apotheosis of mathematical understanding. It’s much higher level, supposedly, than “just” being able to get the right answer.

But does saying intelligent things about math necessarily mean that you can actually do math? One situation that often ends up suggesting otherwise is tutoring.

When you tutor someone one-on-one, at length, over a long enough period of time, it’s easy to think that you are comprehensively probing the breadth and depth of their understanding--simply by conversing with them broadly, and by asking the right questions and follow-up questions. Surely what your tutee says reflects what s/he knows. Surely it’s not possible for him or her to carry on articulately about something they don’t fully understand. And surely it’s not necessary to test their ability to do tasks independently in the more traditional, detached testing format of a written examination.

But some of the same things that make tutorials so great—their fluidity and flexibility, and the apparent close-up they provide into students’ understanding—are also their biggest downside. It’s not hard for tutors to accidentally provide more guidance than they intend to; to lead the tutee towards the correct answer; to fail to create situations, complete with awkward silences, in which tutees have to figure things out completely on their own.

Furthermore, when it comes to math in particular--symbolic, quantitative, and visual as it is--verbal discussion only captures so much. One can converse quite intelligently about limits, for example, without actually being able to actually find a limit, or about the properties of functions without being able to construct a formal proof of any of those properties.

Too often I’ve seen tutors grossly overestimate the ability of their more verbally articulate tutees to do the actual math—until they find independent testing turning out results much lower than they expected.

To put it in terms that are only semi-mathematical, clear verbal explanations are neither a necessary nor a sufficient condition for mathematical mastery. And it’s only the latter that correlates significantly with true mathematical understanding.

Saturday, October 11, 2014

College admissions: screening out achievement robots via personality testing

In a recent Op-Ed in the New York Times, Adam Grant, a professor of management and psychology at Wharton, shares his wisdom about America’s college admissions system:

The college admissions system is broken. When students submit applications, colleges learn a great deal about their competence from grades and test scores, but remain in the dark about their creativity and character. Essays, recommendation letters and alumni interviews provide incomplete information about students’ values, social and emotional skills, and capacities for developing and discovering new ideas.
This leaves many colleges favoring achievement robots who excel at the memorization of rote knowledge, and overlooking talented C students. Those with less than perfect grades might go on to dream up blockbuster films like George Lucas and Steven Spielberg or become entrepreneurs like Steve Jobs, Barbara Corcoran and Richard Branson.
Apparently robots who mindlessly memorize things do better on their SATs than humans who read passages carefully, recognize grammatical errors, and know algebra backwards and forwards. And apparently those mindless robots also get better grades than humans who produce careful, thoughtful work—or who are socially savvy enough to know how to please their teachers and motivate them to grade them generously.

In addition, instead of favoring, say, those who look to benefit the most from college-level courses and to offer the most to their fellow classmates (in terms of a diversity of ideas, insights, perspectives, backgrounds, viewpoints, values and character traits), colleges should instead favor:

(1) students with a certain specific character traits, values, and levels of social and emotional skills (a.k.a. personality discrimination).

(2) students who, even prior to matriculation, look like they will have the most ostentatiously impressive careers after they graduate (a.k.a., the” best graduates” as opposed to “best students” approach).

What should replace our college admissions system, thinks Grant, are the so-called “assessment centers” that companies use to evaluate managers and other employees:
Assessment centers give nontraditional students a better chance to display their strengths. For example, imagine that a college wants to focus less on book smarts and more on wisdom and practical intelligence. Rigorous studies demonstrate that we can assess wisdom by asking applicants to give advice on moral dilemmas: What would you say to a friend who is considering suicide? How should a single parent juggle family and work? The answers offer a window into how well students balance different interests and values.
Sure, I’m a big fan of wisdom, but should colleges (as opposed to, say, trade schools) really be favoring practical intelligence over more abstract, theoretical capacities? As for book smarts, when it comes to the street or the lobby or the conference room or the corporate ladder, these may, indeed, be a bug rather than a feature. But isn't academia, even now, still largely about… books?

And is identifying who does or doesn't stumble over what to say to a single parent or a suicidal friend really the best way to gauge who can handle molecular biology, who will get the most out of a history of Islam course, and who will have the most to offer to fellow classmates during a seminar on symbolist poetry?
Similarly, we can identify candidates with strong interpersonal and emotional skills by watching students teach a lesson to a challenging audience — as Teach for America does when assessing applicants. And tests have already been developed to measure creativity and street smarts, which predict college grades over and above high school grades and SAT scores, while reducing differences among ethnic groups. By broadening the range of criteria, assessment centers make it possible to spot diamonds in the rough.
Assessing college applicants for their teaching skills might be a good idea—but not as a tool for personality discrimination (which, have I mentioned this already?, is a bad idea—and unethical to boot).  A student's teaching ability indicates, more directly than it indicates his/her interpersonal and emotional skills, how well s/he can articulate to fellow classmates his or her ideas, insights, perspectives, backgrounds, viewpoints, and values—the diversity of which are a huge part of the college experience.

What about creativity? Isn't the college experience also enlivened by student creativity? The problem here is the limitation of our creativity measurement tools. I’ve blogged earlier about the one that Mr. Grant is referencing here: the Rainbow/Aurora test. One of its sample questions: Number 7 and Number 4 are playing at school, but then they get in a fight. Why aren't 7 and 4 getting along? Is someone who produces a high-scoring answer necessarily going to contribute more to, or get more out of college than someone who snorts and gives the question the kind of answer it deserves?

It’s also not clear how much students' street smarts add to the college experience. To the extent that they really do “predict college grades over and above high school grades and SAT scores” (Grant provides no reference for this particular claim) it may simply be because those with street smarts are especially good at grade grubbing. I’ve seen that up close and personal.
Third, when students submit essays and creative portfolios in the current application system, it is impossible to know how much help they have received from parents and mentors. In an assessment center, we can verify that students are personally responsible for the work they produce.
Right. But there’s a cheaper alternative that doesn’t involve Grant’s assessment centers—cheaper, because it involves processes already in place. These would be those standardized tests that Grant is so eager to jetison. They include, in particular, the SAT critical writing test. This test, via the much-loathed 5-paragraph essay, is the one source of proctored, unaided student writing that college admissions officers have at their disposal (however rarely they actually read these particular essays). When the new SAT is rolled out next year and the essay becomes optional, we will, indeed, need some sort of a replacement.

For the most glaring omission in the college admissions process isn't students’ personality traits or social skills or values or answers to why 7 and 4 don’t get along, but, rather, their ability to write a clear, insightful, well-organized essay without help from others.

Thursday, October 9, 2014

Math problem of the week: a 6th grade Common Core-inspired "Performance Task"

Below is a problem that appeared in a recent article in Education Week, which explains that it comes

from a sample 6th grade performance task for the common-core math assessment being developed by the Smarter Balanced testing consortium. The full five-part problem asks students to find the volume of a cereal box, to label the dimensions on a diagram of a flattened box.., and to determine the surface area of the box. The task culminates in this open-ended question.


Notice the note at the bottom: "This problem has been edited for clarity."

The issue of clear communication comes up in a second way in the Edweek article. The article quotes Doug Sovde, director of content and instructional supports for PARCC, which, besides Smarter Balanced, is the other main institution that has been developing Common Core tests.
Being able to communicate a thought process in words is a critical skill, said Doug Sovde..."I think it's entirely reasonable for somebody to work through something on scratch paper and be able to share their reasoning after they worked through the task," he said. "It's not unlike a student making an outline before they write their research paper."
I'm confused: how is sharing your reasoning after you work through a task like outlining something beforehand?

Yes, clear communication is important, especially when you are inflicting something on millions of students--as opposed to solving a problem in private.