Monday, March 14, 2011

Why multiple choice tests are a bad thing

This is a great example of a bad multiple choice test. Multiple choice tests are hard to write because the one writing the question has to anticipate the misunderstandings that can occur. The first two questions are a disaster and the last one isn't any better. Let's take this a question at a time:

1. Sara works full-time at the Big Save store and earns $2,500 per month. Who pays the contributions to Social Security on the $2,500 per month in wages she earns?
A. Only Sara.
B. Only Big Save, her employer.
C. Both Sara and Big Save, her employer.
D. I don’t know.

So the question is "who pays?" I'm sure they are looking for C (even before checking the answers at the bottom), but in fact you could justify every single answer. From an economic perspective, the employer basis the salary it can pay on the added payroll tax and costs, so really Sara is paying for it in a lowered salary - certainly in the aggregate of all the Saras in all the Big Saves of the country. From the perspective of who writes the check, the answer is B. From the perspective of who sees what deduction on their income and or inflation in their payroll expense, that would be C. And of course you could write D just because the question is such a mess.

2. Amanda has $5,000 saved up from working at different jobs. She puts her money in a savings account that pays four percent a year in interest. How much money will be in her account at the end of the first year and at the end of the second year?
A. End of first year: $5,100; end of second year: $5,400.
B. End of first year: $5,200; end of second year: $5,400.
C. End of first year: $5,200; end of second year: $5,408.
D. I don’t know.

Well, this is a compound interest question. They want C. At least A and B are clearly wrong. But the real answer is D - there isn't enough information to know, as you don't know how often the interest compounds. The question assumes annually. I don't know any savings accounts yielding 4% currently, but I have never seen one compound only annually.

5. John drove his car to the local Gas and Shop store. On the way to the store he got distracted while talking to his friend in the car and hit a street sign. Neither he nor his friend was hurt in the accident, but the front end of the car was damaged. What type of automobile insurance coverage will provide reimbursement for damages to his car?
A. liability.
B. collision.
C. comprehensive.
D. I don’t know.

If you get collision *or* comprehensive, you will be covered. Technically when you read down the itemized list of insurance coverage, they may be broken out, but when people think of insurance they often think of levels, so liability is less than collision which is less than comprehensive. So B or C are both financially literate answers, unless you are an insurance agent.

1 comment:

Kevin said...

My favorite multiple choice question was one a professor gave us back in college for a liberal arts class "Principles of Aesthetics"... I don't remember the question but it had several specific answers followed by:

d. a and c
e. none of the above
f. some of the above

I liked f on that one... That professor also used to toss in some "one of the above" choices.

And no, his class wasn't hard at all, it was one of the really light-weight ones... But it did come back to mind on a recent weekend when I checked out the Museum of Modern Art and saw some of the paintings that were discussed in class...