Does Learning Assessment Help?

What are the biggest changes in higher education in North America over the past twenty-five years? Almost every list would include the development of an intensive "culture of assessment".

TST’s quality assurance processes and accrediting standards require us to do assessment in two areas in particular: What did our students learn in each course? How effective are our programs?

For courses, assessment requires us to identify the learning outcomes that students will achieve, and the ways in which achievement will be measured. For programs, assessment requires us to define educational pathways and expectations, and to ensure perpetual feedback.

Outcomes and inputs

When I was an undergraduate half a century ago, we didn’t look for the "outcomes" of a BA, unless perhaps we were pre-med or had another specific career goal. We chose a school for its "inputs" or resources: great teachers, great libraries, great course offerings, great facilities, great extracurricular and community opportunities, and, let’s be frank, good food in the dining hall. The theory was that by spending four years reading and thinking and thriving in the right place with new friends, we’d somehow come out with the skills, knowledge, humane values, and good judgment of a well-educated person.

For me, it was the right way to learn. I didn’t want to know in advance where someone wanted me to end up.

The culture of assessment

But people paying the bills wanted to know that the system was producing something measurably useful. The payers of bills included government departments of education, taxpayers, donors, foundations, often students' families, sometimes employers. Implicit, frequently, was a pragmatic bottom line: was post-secondary education preparing students for real jobs? For theological schools it was: are graduates being trained to run churches well?

By the 1990s the emphasis on educational assessment was gaining traction. In 1996 the Association of Theological Schools in the United States and Canada revised its accrediting standards to align with assessment requirements that the six big U.S. regional accrediting associations had already established. Further revisions of ATS standards in 2004 and 2010 established the "culture of assessment" as a positive good.

Does assessment help?

Did all this assessment put us on the right track? Has any of it actually improved student learning? A lot of people have started to wonder.

In yesterday’s New York Times "Sunday Review" (February 25), Molly Worthen published a gangbusters commentary titled "The Misguided Drive to Measure 'Learning Outcomes' ". Molly is a specialist in the history of American evangelical religion, and was our colleague at the Toronto School of Theology six years ago. She now teaches at the University of North Carolina.

There are some assertions in her column that I’d want to take issue with, but I think she’s quite right about her main points.

  • Learning assessment devours a lot of time and money "for meagre results".
  • In emphasizing job-ready skills, it can "miss the point".
  • The data that it creates are usually "not very good".
  • The emphasis on program assessment makes people think that educational problems can be fixed by doing something to programs. The real issue is often something else, like the social, financial, and educational disadvantages that our students are struggling with, or inadequate educational resources. Frankly, another impediment can be the numbing bureaucratic ethos of an educational system organized around a culture of assessment.

Molly is by no means a voice in the wilderness. In fact, the critics are pretty numerous. You’ll find them speaking up in books, in the Chronicle of Higher Education and University Affairs, and at academic meetings — including right here at TST.

One such critic is Erik Gilbert, a historian at Arkansas State University. Is it coincidence, he asks, that our focus on assessment is happening just as we’ve come to depend on adjunct faculty members, part-time enrolment options, and online courses? By no means. Assessment, he says, is our way of covering up the fact that students actually aren’t being as well educated as they were when a traditional core faculty built mentoring relationships with full-time students that extended beyond classroom walls and virtual learning portals. (We still have that old-time model in our PhD programs.)

An alternative ideal

What are the essentials of education? There’s a story that U.S. President James Garfield was transformed by a brilliant teacher, a Congregationalist theologian named Mark Hopkins. The experience persuaded Garfield that the ideal college was "Mark Hopkins on one end of a log and a student on the other". You get the same sense when you start reading a Platonic dialogue. No one can predict where the conversation will end up, but you know you’ll learn a lot on the way.

Actually, I think people learn better in community than on a log with one other person, but what I like is the vision for an education that isn’t pre-packaged and assessed.

I suspect that we have prospective students in our churches who, like me fifty years ago, have no vocational end in view, but just want some space to read transformative works and explore ideas with others. For them learning outcomes assessments may get in the way. And I suspect there are teachers besides myself who love it when our students discover knowledge that we hadn’t even thought of, let alone planned.

What’s the word for that?  It might be "grace".

Unplanned educational surprises elude the assessment data, but they’re often the most exciting and life-giving part of education. Let’s not shape our educational processes in ways that inhibit surprises.

~ Alan L. Hayes