What to do, what to do

In common with many academic institutions, we've been going through the process of developing ways to assess student learning, both in their general education courses and in the major. In my program (business), we've been talking about using an outside test (the ETS Major Field Test in Business) as a high-stakes test, both in the undergraduate program and in the MBA.

The way this would work looks something like this: During a student's final semester/year, the student would take the ETS Major Field Exam in Business. (If anyone is curious, you can find a description of, and sample questions for the undergrad exam here, and for the MBA exam here.) In order to graduate, the student would have to achieve a minimum score (expressed as a percentile rank). (Our proposed minimum score is low, so I'm not really concerned very much about that.)

In short, the ETS MFT in Business would become a high-stakes exam for students.

Our current proposal allows students two attempts to achieve a "passing" score. (Another campus in the same system allows multiple attempts, with the university paying for the first attempt and the student responsible for paying the test fee for the second and subsequent tries.) The committee considering this is conflicted about the proposal (I'm not),

On the plus side. The test provides some external validation of what we are doing.

On the other side.

1) My reading of the sample questions on the ETS web site (here for the undergrad and here for the MBA) leaves me concerned about whether the tests correlate closely enough with what we do in our program.

2) That raises the possibility that a srudent who has done well in the coursework might not "pass" the ETS test.

3) Which means we would have to consider what the problem is. Does the ETS test fail to test what we emphasize? Does it in fact test what we emphasize, but our grading standards allow students to think they are succeeding when they are not?

A lot of schools use these tests, although I suspect most of them are not the sort of high stakes testing we're considering. But I'm not sold on the idea. And I'm trying to un-sell my colleagues.