The Analysis of Sequential Experiments with Feedback to Subjects

Abstract

A problem arising in taste testing, medical, and parapsychology experiments can be modeled as follows. A deck of $n$ cards contains $c_i$ cards labeled $i, 1 \leq i \leq r$. A subject guesses at the cards sequentially. After each guess the subject is told the card just guessed (or at least if the guess was correct or not). We determine the optimal and worst case strategies for subjects and the distribution of the number of correct guesses under these strategies. We show how to use skill scoring to evaluate such experiments in a way which (asymptomatically) does not depend on the subject's strategy.