Teaching Materials
Ask a Master Teacher
Lesson Plan Gateway
Lesson Plan Reviews
State Standards
Teaching Guides
Digital Classroom
Ask a Digital Historian
Tech for Teachers
Beyond the Chalkboard
History Content
Ask a Historian
Beyond the Textbook
History Content Gateway
History in Multimedia
Museums and Historic Sites
National Resources
Quiz
Website Reviews
Issues and Research
Report on the State of History Education
Research Briefs
Roundtables
Best Practices
Examples of Historical Thinking
Teaching in Action
Teaching with Textbooks
Using Primary Sources
TAH Projects
Lessons Learned
Project Directors Conference
Project Spotlight
TAH Projects
About
Staff
Partners
Technical Working Group
Research Advisors
Teacher Representatives
Privacy
Quiz Rules
Blog
Outreach
Teaching History.org logo and contact info

Quality Evaluation: It Takes Work

play
00:00 00:00
mute
Video Transcription

Evaluating the Goals of the Project

2:57

There doesn't need to be any kind of special evaluation magic related to TAH, but that, really, the concepts of good program evaluation carry over regardless of what kind of project you're working with.

Goals

The sort of basic tenets for effective program evaluation are, first is to base the evaluation actually in the goals of the project. Okay, so, what is it that the project is intending to do?

. . . what is it that the project is intending to do?

Did you and are you actually doing the things that you set out to do within your goals? Those are the, or that is sort of the basic question and questions that you try to answer. In fact, actually when we start off on a project like this we frame this as evaluation questions that tie back to the goals. The next piece is to make sure that the data that you collect in fact actually flows from those evaluation questions. So, if you say that, you know, the goal of your project is to ensure that teachers develop a greater understanding of the roles of freedom, liberty, and democracy as enduring concepts in American history, okay, well, we're going to want to actually find out if in fact teachers can do those things.

The fact of the matter is, though, is that a lot of times people in their evaluations don't follow that basic rule of "evaluate what it is that you're supposed to do."

Role of the Evaluator

So I think the role of the evaluator, whether it's, whoever's doing your evaluation for you, is to be able to pull the project's focus back to the goals. We don't necessarily want to know that people just think that, you know, such and such a presenter was the greatest presenter since, you know, Abraham Lincoln if that doesn't really have anything to do with what it is that the actual outcomes of the project are tied back to.

So, you know, it's our task to sort of pull that along and guide that along and I think sometimes it's easier to do from the outside.

To say that a professional development project is going to be evaluated based on student gains, or gains in student content skills, now, there's not a direct correlation there. And to attempt to prove a direct correlation, I think, inspires an awful lot of gymnastics and magical thinking and, and all sorts of things, which aren't terribly helpful to projects getting done what they need to get done. I think that it looks like the new indicators are about teacher content knowledge, which, at least that makes sense to test. And then issues related to the, you know, the degree of participation, or the depth of participation on the part of participants in the professional development that's being offered, that makes sense as well, it stands to reason to say that if people are going to get something out of a project they have to participate in at least seventy-five percent of the work of the project.

Role of the Evaluator

2:24

I think that all good evaluators for any project, but TAH project specifically, should do formative work as well as summative work. But if there's nobody there to actually, you know, receive the evaluation work that you're doing, then what good is doing it?

A good evaluator should, essentially, be a partner in the project, working with you all along.

A good evaluator should, essentially, be a partner in the project, working with you all along.
Quantitative and Qualitative

There probably are evaluators who say that, you know, we're going to really be summative, we're going to just look at, you know, the quantitative data that's been accumulated over the course of the project. And, no, we don't need to show up at every professional development event or even, you know, a good-sized sample of professional development events. We don't need to do classroom observations. I think that's patent nonsense. I know that the GPRA indicators are largely quantitative and if you're doing good qualitative evaluation you will get the quantitative data that you need, okay. But quantitative is just a small part of this. You know, one of the things that we work with every project to understand, is that just because you said you were going to do the things that you said you were going to do, doesn't mean that they turned out the way that they're supposed to turn out, okay? That latter part, they turned out the way they were supposed to have turned out, is always a qualitative response.

There probably are evaluators who say that [. . . ] no, we don't need to show up at every professional development event or even, you know, a good-sized sample of professional development events. We don't need to do classroom observations. I think that's patent nonsense.

It can't be just quantitative. If your teachers don't do well on their post-test, why? You know, it's not the role of the evaluator, I think, to just simply say, we didn't do very good, you know. We have to be able to explain why. And we have to, you know, feed into a discussion about why. And you have to be there in order to be able to feed into that discussion.

I really wish projects would learn to work better with evaluators. If you are willing to let your evaluator just sort of send you bills and forms then, you know, you get what you pay for, you get what you deserve, I guess I could put it that way. But, you know, if you make your evaluator work for you and find an evaluator who wants to work for you, then, you know, you can really improve your project offerings and improve your outcomes, I think.

Post new comment

  • Web page addresses and e-mail addresses turn into links automatically.
  • Allowed HTML tags: <a> <em> <strong> <cite> <code> <ul> <ol> <li> <dl> <dt> <dd> <b> <i>
  • Lines and paragraphs break automatically.

More information about formatting options

Image CAPTCHA
Enter the characters shown in the image.
 
Content