<note: there is an error in the article>
-
| The chi-square test | The tougher stuff | - | How the test works | A BASIC program to calculate chi-square values |
| Advanced Dungeons & Dragons | - | Dragon #78 | - | Dragon magazine |
It is not surprising that many players have certain favorite dice
-- those they believe tend to roll high (or low, as appropriate).
Selective memory accounts for most of these "high-rollers." A
player will long remember the 20 rolled for a vorpal sword attack
on Tiamat, and easily forget
all the low rolls against the tribe of
orcs. When the bias in the die is purely in the mind of the player,
it does no harm; it provides an illusion of being able to bend
impersonal random reality a little, and adds fun to the game.
But it is possible for a die to be biased, because of tampering or
because of some quirk in the manufacturing process. This can be
a serious problem. The rules of any game specify certain probabilities
of certain events. Using a biased die assigns some different
probability to that event. Even worse, that different probability
is not known. Using a biased die means that, in one
important sense, the actual rules of the game are unknown to
player and referee alike.
It is appropriate to ask at this point: How common are biased
dice? I don't know, and I doubt if anyone else does. I suspect that
it is very rare for commercially manufactured dice to be created
with a bias built in. But such bias can occur, at least in principle.
Uneven wear can create bias; particularly on a d20, rounding
of the edges of one face will make the occurrence of the opposite
face less likely. Uneven wear can also be induced, of course. But
again, tampering that is done with enough skill to not appear
obvious, and that is significant enough to create an important
bias, is probably also quite rare.
On the other hand, the belief that certain dice tend to produce
more results of a certain number, or group of numbers, is exceedingly
common. And it is this belief, due mostly to selective
memory, that makes the procedure outlined below of practical
importance to gamers. For this is how you tell the difference
between the illusion of bias and genuine bias. (I expect
that the
greatest value of this procedure will be to verify the fairness of
a
die that everyone is sure is biased.)
It?s easy to test for a biased die, by the procedure known as a
chi-square test. The rest of this article consists of a simple description
of the test, followed by a more complicated, more general
overview. Finally, a BASIC computer program that makes
the procedure even easier (for those with computers) is provided.
A simple application to illustrate the use of the chi-square test
would be, for instance, an examination of whether each face of
a <d8> has an equal chance of appearing.
1. Preparation. Write each result the die can produce
on the
left side of a piece of paper (e.g., a column of numbers 1-8 for <a
d8>).
Calculate the number of times you will roll the
die for the test by multiplying N × E, where N is the number
of
sides of the die, and E is the number of times each side would be
expected to appear. Ten is usually a good choice for E. Then for
the d8, with E = 10, the total number of rolls (T) is 80. The
number E can be more than 10, and should never be less than 5;
a higher number will yield a more reliable test, but is more work.
2. Data. Roll the die T times, each time recording the
result by
placing a tally mark by the appropriate number. Then total the
number of tally marks for each result.
3. Analysis. Subtract E from each count. Square the result
(multiply the result by itself). Add up these numbers and divide
the total by E. The result is the chi-square. Now, refer to the
Table of Critical Values given later. Use the
row (number of
categories) that corresponds to the number of faces of the die you
are testing. Two values are given, one in the ?.10? column and
one in the ?.01? column. Compare the value of the chi-square
you calculated with those in the table. If your chi-square is less
than the value in column one (labelled .10), the die is almost certainly
fair (or close enough for any reasonable purpose). If your
value is larger than the value in the second column, the die is
almost certainly biased. Specifically, there is a 1-in-100 chance
that a value as large or larger than the value in column two
would have come up by chance, if the die were unbiased. Since
that is so unlikely, you would conclude that the die is biased.
If the value of the chi-square you have calculated is greater
than the value in column one and less than the value in column
two, the result is too close to call. There is a fair chance that the
die is biased, but also a fair chance that it is not. You could
repeat the procedure, possibly choosing a larger number for E.
The tougher stuff
Here are some more complicated explanations and procedures.
(You may want to ready a read magick
spell before going on.) A
more general formula for the chi-square is:
Suppose you have observed the behavior of a particular sixsided
die, and it seems that the numbers 1, 3, and 4 occur rarely,
2 and 6 appear as often as one would expect, and 5 appears quite
often. In this case it might make more sense to test directly
whether this observation is really accurate, rather than simply
making the general test described earlier. If what you suspect is
true, a specialized test will show the bias more readily.
Define an occurrence of a 1, 3, or 4 as category #1, a 2 or 6 as
category #2, and a 5 as category #3. Perform the tallying procedure
as described above, only this time record each roll by category
(#1, #2, or #3) rather than by the actual number rolled.
(Again, E = 10, for a total of 60 rolls, is a good choice.) The formula
would be applied like this: The chi-square equals (count in
category #1 - 30)²/30 plus (count in category #2 - 20)²/20
plus
(count in category #3 - 10)²/10.
Note that the value of E for each category varies. For instance,
since category #l includes 3 out of the 6 possible results, this
category would occur on the average of 30 times out of 60 if the
die were unbiased. The row to use in the table for evaluating the
chi-square is 3 (the number of categories considered).
How the test works
A proof of why this procedure works is suitable only for an
advanced course in mathematical statistics. An intuitive explanation
may prove helpful, though. Ei in the equation is the
number of times an event in category i
is expected to occur in the
long run, if the die is unbiased. In a test, the number of occurrences
tallied for each category tend to vary somewhat from Ei.
The critical question is, when is that variation too much?
And when is it so much that the assumption that the die is fair
becomes unbelievable? What the equation does is convert the discrepancies
(01 - Ei) into a number (the chi-square statistic) that
makes answering the question relatively easy.
Tables for the probabilities of chi-square have existed for
many years and are widely available. In addition, some programmable
calculators have programs available to calculate the
exact probability of a chi-square. The table in this article is an
abbreviated version of the tables that usually accompany statistics
texts; it includes those values of particular use in testing dice.
If the die is really fair, the chance of the chi-square being greater
than the value in column two (.01) is less than 1 in 100, as the
article pointed out earlier. So it is unlikely that a fair die will
produce a chi-square in excess of this value. The chance is only 1
in 10 of a fair die producing a chi-square greater than the value
in column one (.10). If the die being tested has any appreciable
bias, it will most likely cause the chi-square to exceed this value,
and will generally cause it to exceed the critical value in column
2 as well.
The chi-square result of 7.6 is less than 12.017, the value in
column one, in the "8" row. Conclusion: the die is unbiased.
| Number of categories | .10 | .01 |
| 2 .................. | 2.706 | 6.635 |
| 3 .................. | 4.605 | 9.210 |
| 4 ................. | 6.251 | 11.341 |
| 5 ................. | 7.779 | 13.277 |
| 6 ................. | 9.236 | 15.086 |
| 7 ................. | 10.645 | 16.812 |
| 8 ................. | 12.017 | 18.475 |
| 9 ................ | 13.362 | 20.090 |
| 10 ............... | 14.684 | 21.666 |
| 11 .............. | 15.987 | 23.209 |
| 12 .............. | 17.275 | 24.725 |
| 13 ............. | 18.549 | 26.217 |
| 14 .............. | 19.812 | 27.688 |
| 15 ............ | 21.064 | 29.141 |
| 16 ............ | 22.307 | 30.578 |
| 17 ............ | 23.542 | 32.000 |
| 18 ........... | 24.769 | 33.409 |
| 19 ............ | 25.989 | 34.805 |
| 20 ............. | 27.204 | 36.191 |
A BASIC program to calculate chi-square values
The program first asks you for the number of faces on the die;
enter that number. Then it asks for E, the number of times each
face is expected; enter that number. (If you enter less than 5, it
will set E = 10.) The program next asks if you want the categories
grouped. If you are only doing a simple test, enter 0. If you want
to group some categories together (as in the second example),
then enter 1.
If you use group categories, you will need to enter the number
of groups desired. The example in the article used three groups:
l-3-4, 2-6, and 5. Then enter the group code for each face, in
order. Again, using the example, you would enter 1, 2, 1, 1, 3, 2,
for faces 1-6 in that order. That assigns a roll of 1, 3, or 4 to
group 1, a roll of 2 or 6 to group 2, and rolls of 5 are the only
element of group 3.
The next step is entry of the data. Roll the die and enter the
results until the computer tells you it's done. If you enter invalid
data (e.g., a 7 when testing a d6) the program gives you an error
message and requests re-entry. When data entry is done, the program
will display the results and wait for some input to give you
Time to look at it. If you used the grouping option, the program
will then display the results by group.
Finally, the chi-square value will be displayed, along with a
message indicating the result of the test -- no bias, uncertain, or
bias likely.
Program listing
OCTOBER 1983
Missing line
Dear Dragon:
In issue #78 you had a program to calculate
the chi-square values. On line 520 you referenced
line 660, but line 660 was not in the
source code. Could you please tell me what
line 660 says?
Bret Bradford
Torrance, Calif.
(Dragon #79)
Our apologies to you, Bret. . . and to anyone
else who tried to make the program work the
way it was printed. We tried to consolidate
some program lines in the printed version, for
space-saving purposes, and wound up making
a very basic mistake (no pun intended). There
was a line 660 in the original program, containing
the PRINT statement that comprises
the second part of line 650. We combined the
two lines into one, not realizing that this
would mess up the program. (Simply changing
line 520 to read "GO TO 650" won't solve
the problem, because then the program would
be jumping to a NEXT statement without
encountering the FOR statement that precedes
it.) To solve the problem, just create a line 660
that contains the second part of line 650, make
line 650 read "NEXT J" and the program will
run the way it's supposed to.
-- KM
(Dragon #79)
Dear Editor:
I have just discovered another error in the
listing of my chi-square program (#78).
Line 670
should read NT=NF. . . instead of NT=NC. . . .
The result of this error is that in standard (ungrouped)
mode, the message regarding the test
will be wrong (always biased in most BASICs)
The chi-square value is still correct, however.
I?m not sure of the source of this error. It
doesn't appear on my program listing, but since I
didn't have a printer at that time, what I sent you
was a typed transcript of the listing, and I can't
find a copy of that now. Well, if it was my error,
sorry. Regardless, I would appreciate your informing
your readers of the correction.
David G. Weeks
Webster Groves, Mo.
(Dragon #82)
I pulled David’s original listing of the chisquare
program from our files and discovered
that the mistake did appear in the typed tran-
script he sent. I really do understand how these
kinds of mistakes can happen (because I?ve made
so many of them myself), and I?m grateful to
David and a few other readers who pointed it out
to us.
(Dragon #82)
Die-sastrous!
Dear Dragon:
Fellows, the April issue is 6 months away. How
did Mr. Weeks' article ("Be thy die ill-wrought?
") wind up in the October issue?
First of all, who wants to test their dice for
bias? Secondly, the computer program was all
messed up (i.e., line 520 with goto 660, and no
line 660!). Thirdly, who wants to roll their twenty
siders 200 times each, and lastly, the formula
doesn't work!
After typing in the program, I tested my 4-
sider three times, and the result was biased. I
then split the numbers into groups and did the
whole thing over, and the result was no trace of
bias! That didn't bother me much. So I tested a
20-sider and a 6-sider, and they all came up
biased.
Seriously, if my first DRAGON copy I ever
bought had that article in it, I would have never
subscribed.
John Biederman
Ringwood, Ill.
(Dragon #80)
Okay, John . . . first, apparently you want to
test your dice for bias, because you did check out
at least a few of the dice you own. (It does seem,
though, like you were motivated by a desire to
find fault with the procedure, instead of a desire
to find fault with your dice.)
Second, you're right: the computer program
was a little "messed up." We made our apologies
for that, and fixed the mistake, in last month's
letters column.
Third, I don't know of anyone who "wants to
roll their twenty siders 200 times," but we were
willing to play the odds. Out of the hundreds of
thousands of people who read this magazine, we
figured there would be at least a few thousand
who'd be interested in going through the
motions. Judging by the number of letters we got
with questions about the mistake in the program
listing, lots of people were interested enough to
type in the program and use it. We don't expect
to please everybody, but it seems like we pleased
enough people so that we don't have to feel like
printing the article was a bad decision.
And last, I don't understand why you came to
the conclusion that the chi-square procedure
doesn't work. Maybe the dice you tested were
biased. Maybe you didn't repair the mistake in
the program properly, so that it wasn't running
properly when you put in your test results. And
maybe your problem with the 4-sider arises from
the simple fact that random generation can
produce some strange results. For instance, the
odds against a 4-sider coming up "1" twenty
times in a row are very small, but it is possible.
And if your test rolls were very unevenly distributed,
it stands to reason that the chi-square
procedure would be thrown off accordingly.
Sincerely, I?m sorry you didn't like the article,
and (no offense) I hope there aren't too many
other people who felt the same way. And last but
not least, thanks for giving me a terrific idea for
next April's issue.
-- KM
(Dragon #80)
How many sides?
Dear Editor:
I appreciate the article about chi-squares, but I
have one problem: How do you test the d4's that
have 8 sides? Is the number of categories 4 or 8?
Marvin Somerman
Los Angeles, Calif.
(Dragon #80)
You can solve this problem in one of two ways.
depending on the results you want to get. If you.
want to test for the occurrence of each of the eight
sides of the die, mark half of the sides (1A
through 4A) so you can tell them apart from the
other four (1B through 4B). Then test the die as if
it was a d8 to see if each of the sides shows up
with roughly the same frequency. Or, you can
simply treat the die as a "normal" d4 (one with
four sides) and test to see if each of the four
possible number results comes up with the same
frequency I'd recommend the second method,
since what you're interested in is how often a
certain result occurs, not how often a particular
side is showing.
-- KM
(Dragon #80)