Testing Patience

Following up on my last post, if we want to know whether it is possible to teach students problem solving skills we need to have some method of measuring them.  Let's start with persistence.

Persistence is not a very precise concept, but I have been thinking of a few ways to measure it:

1).  Give a test that is composed of two different kinds of questions.  Some questions are basic skills questions designed to test whether a student has command of a certain technique.  Other questions require application of the technique more than once.  Measure the scores on the two types of questions and create a linear regression to find the best fit line that shows the typical relationship between the two.  If a student has a score on the complex problems that is higher than one would predict based on their mastery of the technique we could call this persistence.  That is, persistence defined as the ability to stick with and process multi-step problems.

To measure whether students are progressing in their persistence you would have to repeat this process throughout the year (and over the years).  If the relationship between basic skills mastery and complex problem mastery changes (the linear relationship changes), then the average persistence of the class has changed.

In order for this type of measure to be meaningful we would have to assume (or better yet, measure) that the relationship between skill mastery and complex problem mastery is somewhat stable.  If it depends too much on subject matter then this measure of persistence would be unreliable.

2) Give students a test that involves a good mix of problems and plenty of time.  Measure how long they spend on each problem.  Then measure whether there is a correlation between how hard the problems are and how long they spend, as well as how long they spend and whether they get them right.  I am not exactly sure what kind of correlation I would get, but I think that students that are not very persistent will not spend very long on the hard problems and get most of them wrong.

3) Give students a test where they can check their answer, but only whether it is right or wrong.  Then test how much time they spend.  The object of letting them check their answer is that some students might be persistent but not very good at checking themselves.  If they know they are wrong, do they try again?

Anyway, ideally a standardized test would be more like a personality test that measures across several dimensions.  It should measure whether you have mastered specific content goals, as well as testing for general problem solving skills like persistence, the ability to move from abstract to concrete, ability to reason and critique arguments, etc.