Wednesday, November 16, 2011

Moneyball and Education

Like many of my colleagues in education, I tend to do a lot of professional reading during the school year.  At breakfast, I usually load up Tweetdeck and peruse articles and blogs over coffee and a bowl of cereal.  I try to participate in at least one online education chat over the course of the week.  Most evenings I will find a few more posts on blogs that I follow.  And before bed, I usually thumb through a few pages (sometimes 'pages' turns to 'page', depending on whether my little ones got up during the previous night) of a book that I have on the go.  During the summer, my evening book might be a John Grisham, a Tom Clancy, or a Robert Ludlum, however, I tend to move to more academic titles from late-August to June.

In the past few weeks, I have had less of this academic focus for my reading, as my latest two books on the go are Scorecasting and Moneyball.  Being an avid sports fan and a lover of trivia, statistics, and numbers, each of these books have satiated my appetite for the quantification of talent in sport.

As baseball fans know, there are statistics kept on EVERY aspect of baseball.  Hits, runs and errors.  Batting averages, fielding percentages, and earned run averages. Pitch counts, home runs, and RBIs.  Batting averages during the day, at night, in domes, and outdoors.  Batting success against left handed pitchers versus right handed pitchers. And the list goes on and on and on.

Moneyball details the surprising success of the Oakland Athletics in the 2002 using less conventional and more analytical scouting techniques to pick players for their team.  More specifically, it describes how their general manager used oft-overlooked statistics as a predictor of success for a batter in Major League Baseball.  Prior to the A's in the 1990s, major league scouts would go to little league games, college games and minor league games to watch prospects coming through the system. While a few statistics like batting average, home runs, stolen bases might be a part of the process of evaluating a player, a great deal more weight was put on the impressions of the scout. As a result, the players that would garner the highest recommendations (and subsequent salaries) would be those players who met criteria that were based upon the scout's own knowledge, wisdom and experience in scouting players.  In the face of this 'conventional' scouting, the Oakland A's found that two statistics (on-base percentage and slugging percentage) were often overlooked yet highly effective in predicting offensive success for a hitter in baseball.

This use of data and metrics to predict success helped Oakland have a tremendously successful year in 2002 (which included 20 straight victories) with one of the lowest payrolls of any major league team.  So successful was this concept that the lexicon phrase "the team plays 'Moneyball'" has been coined by baseball pundits to refer to those teams who rely heavily on these metrics in making organizational decisions.

Education is not baseball, of that there is no doubt (I would notice the hotdogs here at the school for certain).  Regardless, I began thinking about how elements this book could apply in education, specifically in teaching and education administration.  Data-driven improvement has been a popular concept for many years, but I am not thinking so much about that as I am trying to determine in my own mind THE characteristics of administrators that lead to school success.  THE characteristics of teachers that lead to student success.   To use the Moneyball analogy, what is that "On Base Percentage" equivalent that can help predict success for administrators?  What is that "Slugging Percentage" equivalent that can help predict success for teachers?

When we visit a classroom, we might observe the teacher and their students for a few minutes and walk out saying "That was good teaching!".  Someone could walk into a school and chat with an Administrator for a half hour and say "She really has it going on, what a great Principal."  But how do we REALLY know?  How are we determining this?  Is it by 'feel' (much like the traditional scouts in Moneyball)?  Is it by 'the numbers', like Billy Beane and the Oakland A's?   Or is it a combination of both, and if so, what should weigh heavier in terms of determining our success in our classrooms and schools?

Several months ago, I wrote a post called "Research: The Educational BS Repellent" about the work of John Hattie in his book Visible Learning.  It is an incredible collection of meta-analyses of the research on over 100 factors that influence student achievement in our schools.  After reading the book (and re-reading it a couple of times), I realized that much like the traditional scouts described in Moneyball, I had preconceived notions of what comprises effective educational practice in classrooms and schools.  Yet after reading the meta-analyses of the meta-analyses (yes, Hattie's book is that comprehensive), I came to the conclusion that in many instances, I have to check some of my beliefs at the door and open my mind when I walk into classes at our school.

In light of the talk of teacher evaluation that is making its way around education circles (and administrator evaluation, I would hope),  it is important for us to work with educators and partner groups to collectively  find those key characteristics that translate into student and school success.  However, when working with our partner groups, it is important to make sure that we temper our feelings and beliefs about teaching and administration with some Moneyball/Hattie-like data that helps us determine effective practices that will truly benefit our students.

If we can work some of the concepts from Moneyball into schools, I can only hope that the hotdogs aren't far behind!

5 comments:

  1. Thoughtful post Cale.
    Firstly, I really enjoyed the movie Moneyball. The concept, along with a large payroll, did help my Red Sox to the title in '04, shortly after the Athletics successes highlighted in the movie. Only wish that they had stuck with some of those principles this year instead of overpaying for free agents such as Carl Crawford.

    Secondly, I think that you raise some good question when you ask what should weigh heavier in determining our successes in the classroom - the feel good factor or statistical data.
    I would argue that you cannot have success in the latter without the former, which comes, in part, through good teaching, well structured lessons, and an atmosphere that encourages student thought and engagement.

    I think, however, that at present, those who are responsible for our educational systems (politicians and bureaucrats) are too focused on statistical data and measurable outcomes. They need these to justify expenditures to a public that is increasingly scrutinizing where their tax dollars are being spent. We need, as educators, to make sure that we can link the improvements in statistical data with the feel-good factor in schools. How one would quantify a feel-good factor within a class is anyone's guess.

    I know, however, that within my own classes, over my career as a teacher, I have found that the more time I take to get to know my students, their concerns, their issues, their problems, the better statistical results I have seen from them. I think that we have to remember that as teachers we are tasked with teaching a whole person, not just achieving a few more percentage points on a test, or a batting average for that matter.

    ReplyDelete
  2. What I like the best here, Cale, is the idea that we can't make decisions -- about teachers, about students, about schools, about principals -- solely on our professional hunches.

    That's a theme that rang true in Atul Gawande's Better -- and it is the reason the Apgar score for measuring the health of babies exists.

    Here's a bit I wrote that lines up nicely with what you've written here:

    http://bit.ly/tEkB3a

    I think the hitch is helping people to understand that we're looking at the wrong indicators right now when judging teachers.

    Instead of looking for the statistics that make the greatest impact, we're looking for the statistics that are the easiest to measure.

    Another hitch is we have a really poorly defined vision of what "successful students" are. That, too, is determined by what we can measure instead of what we really care about.

    Until those two notions are straightened out, I think we're screwed.

    Does that make sense?
    Bill

    ReplyDelete
  3. I first became really interested in (and skeptical of) school data after reading Freakonomics a few years ago. After spending countless hours trying to tie what I see in classrooms to the results that come back months later from the Ministry, I am reasonably sure of one thing: If you know what you want, and you talk about it, focus on it, support it, and then report on it, the culture will change and results will improve. Success is a very subjective concept, but you need to at least know where you are before you can get to where you need to go. Data is essential to improvement. Peter Jory

    ReplyDelete

Thanks for taking the time to comment on my blog!