10900 - So you want to be a 2n-aire?
You are a player on a quiz show. In the beginning, you have $1. For each correct answer this price doubles, but once you give a wrong answer, you lose everything. The game is over either after you decide to stop, or after you answer n questions.
Each time you are given a question, you think for a while, and come up with a possible answer. You can also estimate the probability p that your answer is correct. Based on this p you can decide whether you will stop playing (and take the current price), or try to answer the question.
What is your expected price, if you use an optimal strategy?
Assume that p is a random variable uniformly distributed over the interval [t,1].
First of all, we will explain the last sentence. Take a lot of questions, try to answer each of them, and each time write down the value p. That sentence says that the numbers you have written down will all come from the interval [t,1] and they will be approximately uniformly distributed.
Now, what is the optimal strategy?
Suppose that your current price is C, and that the probability of answering the current question right is p. If you keep the price, you will gain C. If you answer the question, with probability p you will have C and you will still be in the game with one question less to answer, with probability (1 - p) you gain nothing and you are out.
If this is the last question, the expected price if you answer the question is 2pC. If this is more than C, you should answer the question, otherwise you shouldn't.
Now consider the general case. Let f(k) be the answer we seek – the expected price, if you have k questions left and play optimally. How to compute it? If you don't answer, you will get 2n - k dollars. If you do, you have the probability p that you get the price f(k - 1).
When you will be in this situation, you will know the exact value p, thus your expected price will be .
But when we compute our answer, we don't know the question you are going to get. What now? We will simply take an "average over all possible values of p". (As there is an infinite number of possible values, the "average" will actually be an integral.)
We get the following recurrence:
- f(0) = 2n
Using them, we can compute the values f(k) and output the value f(n).
An easy way of computing f(k): split the integral into two parts, on the first part integrate the constant function 2n - k, on the second part integrate the linear function p.f(k - 1).
 A More Formal Approach
First we define a couple of random variables. Let Xk denote the amount that you win when you have k questions left, and let Pk denote the probability that you'll answer the next question correctly when you have k questions left.
We want to determine E(Xn), which is the expected prize when there are n questions left.
We use E(Xk | Pk = p) to denote the conditional expected value of Xk given that Pk = p. This is just the expression that was derived above.
where f(k - 1) = E(Xk - 1).
Now, as explained above, we don't know the value of p. The solution above was to "average over all possible values of p". This is the correct thing to do, but here's the formal justification for this step:
There is a theorem from probability theory which says that E(Xk) = E(E(Xk | Pk)). (See, for example, section 7.5.2, "Computing Expectations by Conditioning", in _A First Course in Probability_ by Sheldon Ross, to see why this theorem holds.)
where is the density function of Pk. Since Pk is uniformly distributed over t..1, it follows that
- for t <= p <= 1 and
- for p < t or p > 1
Plugging the density function and the conditional expected value into the expression for E(Xk) yields the same results as above:
- f(0) = E(X0) = 2n
1 0.5 1 0.3 2 0.6 24 0.25 30 0.8 0 0
1.500 1.357 2.560 230.138 45517159.608