Thursday, May 7, 2015

The Critical Hit Paradox

Occasionally, while browsing 4chan's /v/ or a similar board, I'll see a thread beginning with the following game-themed math problem:
"You hit an enemy twice. At least one of the hits is a crit. Assuming a 50% crit chance, what is the probability both hits are crits?"
A crit, of course, is a "critical hit" and so a 50% crit chance is rather high, but that's not important. This is basically a variant of the second question of The Two Children Problem, also known as the Boy or Girl paradox:
"Mr. Smith has two children. At least one of them is a boy. What is the probability that both children are boys?"
We could also ask the same question in terms of coin flips:
"Two coins are flipped. At least one comes up heads. What is the probability that both coins come up heads?"
This last version, in my opinion, is far less contrived than one involving critical hits or child gender. Any variation of the question, however, is bound to lead to extensive debate despite its seemingly trivial nature. The question as written is ambiguous, hence the controversy surrounding the original formulation of the Boy or Girl paradox, and hence the nauseating debate that ensues whenever the game-themed version of the problem is posed to the denizens of /v/. The person starting the thread always knows this will happen, too. Much like the moving portal paradox, this problem serves little purpose other than to cause trouble and to make people angry.

But I'm more intrigued than angry. Why exactly does such a seemingly simple question cause so much confusion? Why are different people so absolutely certain of so many completely different answers? A math problem like this should have one unambiguous and provable solution, but I count three common answers on which people seem willing to bet their lives: 1/4, 1/3, and 1/2.

In an attempt to lessen the confusion, I will attempt to explain the justifications for each of these answers below. First, however, I'll summarize what we know to be incontrovertibly true about the given scenario. And, because this is a video game blog, I will be working with the "critical hit" variation of the problem, even though I think coin flips are easier to explain.

The Facts (and One Assumption)


The problem, as written, specifies a 50% critical hit chance. In other words, each attack has a 1/2 probability of producing a critical hit (or "Crit"), and a 1/2 probability of producing a non-critical hit (which I will call simply "Hit"). We are given no other combat-related probabilities, so we will have to assume that "Crit" and "Hit" are the only possible results of an attack (i.e., we cannot "miss" or anything else). This seems reasonable, as the problem is very obviously meant to be a variant of the Boy or Girl paradox, and the original question which is the subject of that paradox does not allow any genders other than boy or girl.

In summary, the possible outcomes of a single attack (with associated probabilities) are
Crit - 1/2
Hit - 1/2
Therefore, if we were simply to attack twice, we would expect the possible outcomes (with associated probabilities) to be
Crit Crit - 1/4
Crit Hit - 1/4
Hit Crit - 1/4
Hit Hit - 1/4
where we have listed Crit Hit and Hit Crit separately to preserve information about the order of the attacks (in case we need it) and, perhaps more importantly, to demonstrate more clearly that getting one Crit is twice as likely as getting two Crits. Alternatively, if we really don't care about the order in which the attacks took place, we could write the very same result as
Two Crits - 1/4
One Crit - 1/2
No Crits - 1/4
but the probability of getting two Crits remains the same, and getting one Crit is still twice as probable.

No matter how we write it, one should note that the set of outcomes described above is based solely on the stated 50% critical hit chance for every attack. Let's not forget that we also have some additional information about a particular pair of hits. According to the problem, we hit an enemy twice and at least one of those hits is a critical hit. However, we are not being told which hit is critical.

If we were told that, say, the first hit is critical, the probability of two critical hits would be exceedingly easy to find: It would simply be the probability of the second hit being critical, so 1/2 would be the answer. Equivalently, in terms of our four equally probable outcomes (see above), we can see that two of them would be ruled out, leaving only two equally probable outcomes remaining:
Crit Crit - 1/2
Crit Hit - 1/2
Hit Crit
Hit Hit
Unfortunately, this is not how the problem is phrased. We are only told that at least one hit is critical — it could be one or the other, not necessarily the first — and because of this, depending on how you understand the question, you might get an answer which is very different from 1/2.

Guaranteed Critical Hit: The "1/4" Interpretation


One fairly common interpretation of the problem (with which, for the record, I strongly disagree) is that "at least one Crit" is some kind of gameplay mechanic which is enforced during combat. In other words, in this interpretation, it is predetermined that we are guaranteed one Crit when attacking the enemy twice. Take care to note, by the way, that this is not simply a misunderstanding of what "50% Crit chance" means. Rather, it's like this: The first attack has a 50% chance to produce a Crit. If it does, the second attack then has a 50% chance to produce a Crit. However, if the first attack instead produced a regular Hit, the second attack must then produce a Crit in order to fulfill the requirement.

In other words, Hit Crit is assumed to be the only possible outcome after an initial Hit, and so it has the same probability as that initial Hit. Meanwhile, Crit Crit and Crit Hit are both possible outcomes following an initial Crit, and they share equally the probability of that initial Crit. The probabilities are, therefore,
Crit Crit - 1/4
Crit Hit - 1/4
Hit Crit - 1/2
so the probability of two Crits — and thus our answer to the question — is 1/4.

If you have Python installed, you can easily verify this result for the "guaranteed Crit" scenario using the computer simulation below. The variables first and second represent the first and second attacks, respectively; the Boolean values True and False represent Crit and Hit, respectively; the variable double keeps track of how many times we get two Crits; and the variable trial is a counter for the while loop, which iterates 100000 times. With all this in mind, the algorithm should be self-explanatory.

# Guaranteed Critical Hit (100000 trials)
import random
double = 0
trial = 0
while trial < 100000:
    first = random.choice([True,False])      # first = random
    if first:                                # if first = crit:
        second = random.choice([True,False]) #     second = random
    else:                                    # if first = no crit:
        second = True                        #     second = crit
    if first and second:                     # if two crits:
        double += 1                          #     count double
    trial += 1                               # count trial
print("Chance of two crits: "
      + str(100 * double / trial) + "%")


This simulation will indeed return a result of approximately 25%.

However, the fact that I got a computer to spit out this number does not mean it's the correct answer to the question. The algorithm used in this simulation and the logic used in the preceding discussion are based on an interpretation of the problem which, in my view, is severely flawed. There's no evidence that the statement "at least one of the hits is a crit" is any kind of gameplay rule to be enforced during combat. There's no evidence that it was predetermined before the attacks took place. All we know is that the statement turns out to be true for a particular instance of two attacks.

Furthermore, the notion of a guaranteed Crit makes no sense when we've already been told to assume that the Crit chance is 50%. This interpretation of the problem forces us to say that the Crit chance is sometimes 50% and sometimes 100%. Worse yet, while a strict reading of "50% Crit chance" would normally lead us to assume that the result of each attack (like a coin flip) is independent from the result of any other, in this solution we are forced to admit that the result of the second attack depends on the result of the first attack.

Fortunately, the next two interpretations avoid all of these problems by dropping the dubious assumption that a "guaranteed Crit" exists in gameplay.

Partial Information Provided: The "1/3" Interpretation


Another common interpretation of the problem is as follows: The two attacks take place with no restrictions other than a 50% Crit chance for each, calculated independently. The fact that we have "at least one" Crit is simply information which then happens to be true for this pair of attacks in particular.

As stated previously, when attacking twice, we should expect the possible outcomes to be
Crit Crit
Crit Hit
Hit Crit
Hit Hit
all with equal probability. Now imagine that, upon attacking twice, we don't know the actual result of each individual attack. This information has been hidden from us. All we know, after the fact, is that we happened to get at least one Crit in the process. Perhaps the game simply tells us so, or perhaps we were able to figure it out because our total damage for both attacks is known to be above some threshold. In either case, of all the possible outcomes, this only rules out Hit Hit.

In other words, of the four equally probable things that could have possibly happened, we know merely that one of them did not happen. The remaining three possible outcomes still have equal probability, and so those probabilities are
Crit Crit - 1/3
Crit Hit - 1/3
Hit Crit - 1/3
Hit Hit
Our answer to the question, therefore, is that we have a 1/3 probability of getting two Crits. It really is that simple.

As with the previous interpretation, we can use a computer simulation coded in Python to verify the result for this scenario. The variables first and second again represent the first and second attacks, respectively; the Boolean values True and False again represent Crit and Hit, respectively; and the variable double again keeps track of how many times we get two Crits. Note, however, that the variable trial does not simply count how many times we run through the while loop. It actually counts the number of runs in which we get at least one Crit. (These are the trials which are considered valid for the experiment, and the while loop runs until we have 100000 of these valid trials.) Therefore, the final result is the proportion of two-Crit trials to at-least-one-Crit trials, expressed as a percentage. In other words, the result is the percentage of at-least-one-Crit trials in which we get two Crits, and that's exactly what we want to find.

# Partial Information Provided (100000 trials)
import random
double = 0
trial = 0
while trial < 100000:
    first = random.choice([True,False])  # first = random
    second = random.choice([True,False]) # second = random
    if first or second:                  # if at least one crit:
        trial += 1                       #     count valid trial
        if first and second:             #     if two crits:
            double += 1                  #         count double
print("Chance of two crits: "
      + str(100 * double / trial) + "%")


As expected, this will give a result of approximately 33%.

Partial Result Observed: The "1/2" Interpretation


Yet another common interpretation begins much like the previous one: The two attacks take place with no restrictions other than a 50% Crit chance for each, calculated independently. Therefore, in general, we should expect four possible outcomes, each with equal probability:
Crit Crit
Crit Hit
Hit Crit
Hit Hit
This time, however, information about the result of each individual attack is not completely hidden from us. We need only check the result of an attack, and when we do this for one of the attacks, we see that it happened to be a Crit. In other words, we are not merely being told that we got at least one Crit in the process of attacking twice. We are actually discovering this fact by sampling the data, and seeing that one particular attack — either first or second — did indeed produce a Crit. Having ascertained this, we've then ruled out two of our four possible outcomes. We are left with either
Crit Crit - 1/2
Crit Hit - 1/2
Hit Crit
Hit Hit
or
Crit Crit - 1/2
Crit Hit
Hit Crit - 1/2
Hit Hit
depending on which attack we checked, and it doesn't really matter which. Once it has been established that a particular attack resulted in a Crit, the probability of getting two Crits depends entirely on the Crit chance of the other attack. The answer, therefore, should be 1/2.

Once again, we can use Python to write a computer simulation verifying this result. As in the other simulations, the variables first and second represent the first and second attacks, respectively. Each of these two variables will be assigned a Boolean value: True or False, representing Crit or Hit, respectively. The variable double keeps track of how many times we get two Crits. Finally, the variable trial is a counter for the while loop, which iterates 100000 times. For each trial, the program picks the first or second attack at random (because it should not matter which attack we sample). That attack is then set to Crit (to reflect our finding that the sampled attack is a Crit), while the other attack is randomly set to Crit or Hit (to reflect the fact that its result is unknown).

# Partial Result Observed (100000 trials)
import random
double = 0
trial = 0
while trial < 100000:
    gotCrit = random.randint(1,2)            # pick 1 or 2
    if gotCrit == 1:                         # if picked 1:
        first = True                         #     first = crit
        second = random.choice([True,False]) #     second = random
    elif gotCrit == 2:                       # if picked 2:
        second = True                        #     second = crit
        first = random.choice([True,False])  #     first = random
    if first and second:                     # if two crits:
        double += 1                          #     count double
    trial += 1                               # count trial
print("Chance of two crits: "
      + str(100 * double / trial) + "%")


The result given by the simulation will be approximately 50%, as expected.

Comparison of Solutions


Now this is a conundrum. We have three different answers — 1/4, 1/3, and 1/2 — and it's not just because I'm bad at math. The discrepancy doesn't result from any simple arithmetic blunders; in fact, the Python simulations prove that these numbers aren't just coming out of nowhere. Each of these answers follows directly from a different interpretation of the problem, and all of these interpretations seem to make some sense.

The "guaranteed Crit" interpretation, however, seems to makes the least sense. As explained before, the idea that a guaranteed Crit is predetermined (and that this requirement should be enforced during combat) is an additional assumption not stated in the problem. It's also inconsistent with the stated assumption of an independent 50% Crit chance for each swing. This alone, in my opinion, is enough to lay this dubious interpretation to rest and say definitively that 1/4 is not the correct answer. Besides, the problem discussed here is clearly intended to be a rewording of the second half of The Two Children Problem, for which 1/4 was never a candidate answer.

That original problem is actually a set of two questions. As they are stated on Wikipedia:
  • Mr. Jones has two children. The older child is a girl. What is the probability that both children are girls?
  • Mr. Smith has two children. At least one of them is a boy. What is the probability that both children are boys?
The answer to the first question is very obviously 1/2, because we know the older child is a girl. The probability Mr. Jones of having two girls therefore rests entirely on the probability of the younger child being a girl:
Boy Boy
Boy Girl
Girl Boy - 1/2
Girl Girl - 1/2
The answer to the second question, though it turned out to be controversial, was originally supposed to be 1/3, because we only know that at least one child is a boy. The scenario in which Mr. Smith has two boys is one of three equally probable outcomes, because all we can say is that not both of the children are girls:
Boy Boy - 1/3
Boy Girl - 1/3
Girl Boy - 1/3
Girl Girl
The competing answer to the second question is 1/2 yet again. Why two answers? It depends on how we find out that "at least one" child is a boy. The answer is 1/3 if Mr. Smith simply tells us that he has at least one boy (or, equivalently, not two girls). However, the answer is instead 1/2 if we have seen one of the children and observed that he is a boy, because then we can say that the probability of having two boys depends entirely on the gender of the other child.

If you've been paying attention, you know this is exactly what's going wrong with the critical hit problem (which, as we can see, is almost identical to the Mr. Smith question). We're getting two very different answers — 1/3 and 1/2 — depending on how we find out that we got at least one critical hit. Our intuition tells us this shouldn't matter at all, so we're tempted to say that our preferred answer (whatever it may be) is true regardless of how the information is discovered. However, we've already seen that this is not the case. Each of the two answers is totally correct if you read the problem a certain way.

Let's take another look at the "partial information provided" simulation:

# Partial Information Provided (100000 trials)
import random
double = 0
trial = 0
while trial < 100000:
    first = random.choice([True,False])  # first = random
    second = random.choice([True,False]) # second = random
    if first or second:                  # if at least one crit:
        trial += 1                       #     count valid trial
        if first and second:             #     if two crits:
            double += 1                  #         count double
print("Chance of two crits: "
      + str(100 * double / trial) + "%")


The assumption in this interpretation of the problem is that, after a particular instance of two attacks, the game has simply told us we got at least one Crit. In the simulation of this scenario, we are essentially counting double-Crit instances in a pool of 100000 attack pairs, but in building that pool, we're only considering attack pairs for which there was at least one Crit. The result of this simulation proves without a doubt that, of all attack pairs producing at least one Crit, 1/3 of those attack pairs will produce two Crits.

Now let's take another look at the "partial result observed" simulation:

# Partial Result Observed (100000 trials)
import random
double = 0
trial = 0
while trial < 100000:
    gotCrit = random.randint(1,2)            # pick 1 or 2
    if gotCrit == 1:                         # if picked 1:
        first = True                         #     first = crit
        second = random.choice([True,False]) #     second = random
    elif gotCrit == 2:                       # if picked 2:
        second = True                        #     second = crit
        first = random.choice([True,False])  #     first = random
    if first and second:                     # if two crits:
        double += 1                          #     count double
    trial += 1                               # count trial
print("Chance of two crits: "
      + str(100 * double / trial) + "%")


The assumption in this interpretation of the problem is that, after a particular instance of two attacks, we checked the result of one attack and saw that it was a Crit (and this is how we know that we got at least one Crit). In the simulation, we are again counting double-Crit instances in a pool of 100000 attack pairs, but this time the pool consists of attack pairs for which a particular attack — either first or second, at random — was determined to produce a Crit. The result of this simulation proves without a doubt that, of all attack pairs in which a chosen attack is known to produce a Crit, 1/2 of those attack pairs will produce two Crits.

It seems 1/3 and 1/2 are both correct answers. They're just answers to two slightly different questions which sound very much the same.

Bayes' Theorem


For more fun (and just as many answers), we can attempt to solve the problem using Bayes' theorem, which states
P(A|B) = P(B|A) * P(A) / P(B)
where
P(A|B) is the probability of event A, given that event B occurs
P(B|A) is the probability of event B, given that event A occurs
P(A) is the probability of event A
P(B) is the probability of event B
Once again, if we're attacking twice with a 50% Crit chance, we must consider four equally probable outcomes:
Crit Crit - 1/4
Crit Hit - 1/4
Hit Crit - 1/4
Hit Hit - 1/4
So let's say A is the event that we get two Crits. The probability of this occurring alone is
P(A) = 1/4
Now let's just say B is the event that we get at least one Crit (or, equivalently, the event that we do not get zero Crits). The probability of this occurring alone is
P(B) = 3/4
Meanwhile, the probability of getting at least one Crit, given that we get two Crits, is obviously
P(B|A) = 1
Now Bayes' theorem gives the result: The probability that we get two Crits, given that we get at least one Crit, is
P(A|B) = (1) * (1/4) / (3/4) = 1/3
On the other hand, we could instead say B is the event that a chosen attack is observed to be a Crit. The probability of this occurring alone is
P(B) = 1/2
Meanwhile, the probability that one attack is observed to be a Crit, given that we get two Crits, is obviously
P(B|A) = 1
Now Bayes' theorem gives a different result: The probability that we get two Crits, given that a particular attack is observed to be a Crit, is
P(A|B) = (1) * (1/4) / (1/2) = 1/2
Even when we turn to equations, a subtle difference can change the answer.

Conclusion


So which number — 1/3 or 1/2 — is the correct answer to our problem? I think I've proved quite enough times that 1/3 and 1/2 are each absolutely correct given the right assumption about how the stated information was obtained. However, considering the critical hit problem's roots in the Boy or Girl paradox, I think I'm going to have to choose 1/3 as the better answer. Despite the ambiguous wording, the whole point of Martin Gardner's original 1959 version of The Two Children Problem was that the two questions should have two different answers: 1/2 for the first, and 1/3 (not 1/2 again) for the second. The critical hit problem discussed here is based on the second question.

Even without author intent as backup, the decision to choose 1/3 over 1/2 is not at all indefensible. Getting an answer of 1/2 requires us to make an additional (albeit minor) assumption: that a particular attack results in a Crit. The problem, as written, only asks us to consider that we get at least one Crit. If we simply accept this stated fact, without taking it upon ourselves to imagine how that information has been revealed, we get an answer of 1/3 as explained before.