Online internet courses and in-house courses in Resampling Methods and R are now available.  Check them out. New classes start all the time.

Errata for my textbooks

CrossOver Designs

Dublin '02Comparing Variances

Testing for Correlation

Now, ANOV has competition.   Permutation tests are preferable for unbalanced k-sample comparisons,  One-way, but are of only marginal utility in the analysis of multi-factor designs, Two-Way.

The group theory underlying the generation of synchronized rearrangements.  And the R code for generating them.

'Tis a Gift to be simple A comparison of the bootstrap and the permutation test that Lynne Stokes, editor of the American Statistician, says is unlikely to be of interest to readers while Martin Tanner, JASA's editor describes it as too short to publish. Hey, Marty, 'tis a gift to be simple.

New concepts of exchangeability--see page 243-247. 

Resampling methods are best for testing hypotheses about models

The Use and Misuse of Permutation Tests

The Analysis of Crossover Designs


newbangThe third edition of  Permutation Tests has been retitled Permutation, Parametric, and Bootstrap Tests of Hypotheses


Applying Statistics in the Courtroom.  Order now from CRC.

 A Manager’s Guide to the Design and Conduct of Clinical Trials. ISBN: 0-471-22615-7, Cloth, 240 Pages, August 2002, US $64.95

Common Errors in Statistics and How to Avoid Them, with James Hardin.

has just been named Wiley’s book-of-the-year for 2003/4.

    Errata:  Georg F Beilhack notes that the formula on page 106 of the second edition should read

          SE = SD/√n

newspinIntroduction to Statistics via Resampling Methods and R

Errata:  Georg F Beilhack notes that on p.59 where I explain the Poisson distribution,
it should say:
Pr{no cells in a square} = 1*e^-1/1 = 0.37 
Pr{exactly one cell in a square} = 1*e^-1/1 = 0.37 
Pr{two or more cells in a square} = 1 - 0.37 - 0.37 = 0.26 

 Click here to download Resampling Software

Join the immortals; send us your answers to the exercises and we’ll post your name and your answer here.   Send to

Click here to download data sets for use with Resampling Methods text


Corrections to errors in the third edition

Corrections to errors in the second edition

Corrections to errors in the first edition

Permutation Tests, 2nd edition

by Phillip Good

Corrections to errors in the text of the 2nd edition of Permutation Tests

Thanks to Haru Martinez for pointing out that in the example on page 41, the correct significance level is 4/70th.

FAQ and Corrections to errors in the text of Resampling Methods 1st edition


Q1 Couldn't I run through the permutations rather than the combinations?
A1 Sure. But you'll get the same p-value using just combinations and you can play FreeCell or Wizardry with the time you save. Take the case of two samples of size three. When you permute within the sample, you leave the value of the test statistic unchanged. You'll get !3!3 times as many extreme values, using all the permutations, but you'll also have !3!3 more values in the total.


Q2 Does your freeware use Aly's statistic when testing for the difference between variances? 

A2  No, it uses the less powerful, asymptotically exact test based on deviations from the median.

pg. 14, Thanks to Dan Nordlund
last paragraph of section 1.4 ...For our first sample, the variance is (4 + 1 + 0 + 1 + 4)/4 = 10/4 grams^2, the standard deviation is 1.323 grams, and the L1-norm is (2 + 1 + 0 + 1 + 2)/4 = 1.5 grams. For our second sample, the variance is (9 + 1 + 0 + 1 + 9)/4 = 5 grams^2, the standard deviation is 2.236 grams, and the L1-norm is (3 + 1 + 0 + 1 + 3)/4 = 2 grams.

pg. 35, top line
there are 2^10, or 1,024 possible outcomes;

p. 51, Thanks to Dan Nordlund
top 3 lines exercise--we find 19 of the (10 choose 5) = 252 possible rearrangements have sums that are as small as or smaller than that of our original sample. Two samples this different will be drawn from the same population just over 7.5% of the time by chance. p. 51 transformed samples (3.1) first value in second line should be 310, instead of 305 295 320 329 315 310 290 325 320

pg. 59, Thanks to Bill Teel
Estimating Permutation Test Significance Level
Get the data and pack it into single linear vector
Resample from the data
Select ...
Select at random without replacement

pp. 59-60 - Aric Agmon notes that 2 to the power of 20 is only a million (or thereabouts), not a million million.

Page 109, Thanks to Barbara Heller
6.1.3 Significance Level and Power
In selecting a statistical method, statisticians work with two closely related concepts, significance level and power. The significance level of a test, denoted throughout the text by the Greek letter (alpha), is the probability of making a Type I error; that is, is the probability of deciding erroneously on the alternative when the hypothesis is true.
.... To test a hypothesis, we divide the set of possible outcomes into two or more regions. We reject the hypothesis and risk a Type I error when our test statistic lies in the rejection region R; we accept the hypothesis and risk a Type II error when our test statistic lies in the acceptance region A; and we may take additional observations when our test statistic lies in the boundary region of indifference, I.

Page 121, thank to Alson C H Look
The f..! is part of R, and not of Q. Halter JH. A rigorous derivation of the exact contingency formula. Proc Cambridge Phil Soc 1969; 65:527-530.

Page 139, Page 121, thank to Alson C H Look
The estimate of the significance level "p" that results from a Monte Carlo simulation is actually a Binomial random variable with standard deviation SQRT[p(1-p)/n] where n is the number of simulations. With p=5% and n=400, the standard deviation is about 1%.

pg. 208-209, Appendix 1; Thanks to Bill Teel and Richard Saba

p. 213 first line of text, Thanks to Dan Nordlund
require that we select elements without replacement.

Corrections to errors in the second edition

Page 11 Thomas Gatliffe points out that the product of 7 observations should be raised to the 1/7 power.

Page 35 Mike Chernick notes that most roulette wheels have a green 0 and 00 (even a 000 in some unsavory parts).  Thus the probability of even a single red is less than one half.  Even if there were no zeros, the probability of ten reds in a row would be 1 in 1000.  The probability of ten reds in a row or an event as or more extreme (like ten blacks in a row) would be 1 in 500.

Page 46 (with thanks to Alson Look and Dwayne Schindler) The "original value of Aly's test statistic"  should be 43 = (8+18+15+2).  For the rearrangement {2,1,1,2}.{4,6,6,4} = 28.  Moreover, there are three arrangements that are as or more extreme; these include (2,3,2.5.,0.5), (2, 3,2.5,2), and (1,3,2.5,2).

Corrections to errors in the third edition

Page 52, see Page 46 of second edition above

Page 85. thanks to vincent vinh-hung for pointing out that the last line of the R program should read cnt/MC.


Return to top of page


This counter provided through the courtesy of