intelligent measurement

95 theses on evaluation

November 19, 2008 at 8:19 am Leave a comment

Disturbed by the state of affairs in evaluation, Professor Cronbach and colleagues wrote a 95 theses on reform in evaluation (inspired by Martin Luther’s 95 theses in 1517). They speak of the need for:

“A thoroughgoing transformation. Its priests and patrons have sought from from evaluation what it cannot, probably should not, give.”

Although written 28 years ago, the 95 theses (pdf) makes may pertinent points still valid today.

Here are several favourites that have stood the test of time (no. 75 is my favourite):

9. Commissioners of evaluations complain that the messages from evaluations are not useful, while evaluators complain that the messages are not used.

35. “Evaluate this program” is often a vague charge because a program or a system frequently has no clear boundaries.

49. Communication overload is a common fault; many an evaluation is reported with self-defeating thoroughness.

75. Though the information from an evaluation is typically not used at a foreseeable moment to make a foreseen choice, in many evaluations a deadline set at the start of the study dominates the effort.

95. Scientific quality is not the principle standard; an evaluation should aim to be comprehensible, correct and complete, and credible to partisans on all sides.

Read the full 95 theses (pdf) – despite this poor copy it’s well worth a read. The 95 theses originally appeared in the book “Towards reform of program evaluation“.

Glenn

Entry filed under: Evaluation reporting, General. Tags: cronbach.