An Introduction to Bootstrap Methods with Applications to R

Preface xi

Acknowledgments xv

List of Tables xvii

1 Introduction 1

1.1 Historical Background 1

1.2 Definition and Relationship to the Delta Method and Other Resampling Methods 3

1.2.1 Jackknife 6

1.2.2 Delta Method 7

1.2.3 Cross-Validation 7

1.2.4 Subsampling 8

1.3 Wide Range of Applications 8

1.4 The Bootstrap and the R Language System 10

1.5 Historical Notes 25

1.6 Exercises 26

References 27

2 Estimation 30

2.1 Estimating Bias 30

2.1.1 Bootstrap Adjustment 30

2.1.2 Error Rate Estimation in Discriminant Analysis 32

2.1.3 Simple Example of Linear Discrimination and Bootstrap Error Rate Estimation 42

2.1.4 Patch Data Example 51

2.2 Estimating Location 53

2.2.1 Estimating a Mean 53

2.2.2 Estimating a Median 54

2.3 Estimating Dispersion 54

2.3.1 Estimating an Estimate’s Standard Error 55

2.3.2 Estimating Interquartile Range 56

2.4 Linear Regression 56

2.4.1 Overview 56

2.4.2 Bootstrapping Residuals 57

2.4.3 Bootstrapping Pairs (Response and Predictor Vector) 58

2.4.4 Heteroscedasticity of Variance: The Wild Bootstrap 58

2.4.5 A Special Class of Linear Regression Models: Multivariable Fractional Polynomials 60

2.5 Nonlinear Regression 60

2.5.1 Examples of Nonlinear Models 61

2.5.2 A Quasi-Optical Experiment 63

2.6 Nonparametric Regression 63

2.6.1 Examples of Nonparametric Regression Models 64

2.6.2 Bootstrap Bagging 66

2.7 Historical Notes 67

2.8 Exercises 69

References 71

3 Confidence Intervals 76

3.1 Subsampling, Typical Value Theorem, and Efron’s Percentile Method 77

3.2 Bootstrap-t 79

3.3 Iterated Bootstrap 83

3.4 Bias-Corrected (BC) Bootstrap 85

3.5 BCa and ABC 85

3.6 Tilted Bootstrap 88

3.7 Variance Estimation with Small Sample Sizes 90

3.8 Historical Notes 94

3.9 Exercises 96

References 98

4 Hypothesis Testing 101

4.1 Relationship to Confidence Intervals 103

4.2 Why Test Hypotheses Differently? 105

4.3 Tendril DX Example 106

4.4 Klingenberg Example: Binary Dose–Response 108

4.5 Historical Notes 109

4.6 Exercises 110

References 111

5 Time Series 113

5.1 Forecasting Methods 113

5.2 Time Domain Models 114

5.3 Can Bootstrapping Improve Prediction Intervals? 115

5.4 Model-Based Methods 118

5.4.1 Bootstrapping Stationary Autoregressive Processes 118

5.4.2 Bootstrapping Explosive Autoregressive Processes 123

5.4.3 Bootstrapping Unstable Autoregressive Processes 123

5.4.4 Bootstrapping Stationary ARMA Processes 123

5.5 Block Bootstrapping for Stationary Time Series 123

5.6 Dependent Wild Bootstrap (DWB) 126

5.7 Frequency-Based Approaches for Stationary Time Series 127

5.8 Sieve Bootstrap 128

5.9 Historical Notes 129

5.10 Exercises 131

References 131

6 Bootstrap Variants 136

6.1 Bayesian Bootstrap 137

6.2 Smoothed Bootstrap 138

6.3 Parametric Bootstrap 139

6.4 Double Bootstrap 139

6.5 The m-Out-of-n Bootstrap 140

6.6 The Wild Bootstrap 141

6.7 Historical Notes 141

6.8 Exercises 142

References 142

7 Chapter Special Topics 144

7.1 Spatial Data 144

7.1.1 Kriging 144

7.1.2 Asymptotics for Spatial Data 147

7.1.3 Block Bootstrap on Regular Grids 148

7.1.4 Block Bootstrap on Irregular Grids 148

7.2 Subset Selection in Regression 148

7.2.1 Gong’s Logistic Regression Example 149

7.2.2 Gunter’s Qualitative Interaction Example 153

7.3 Determining the Number of Distributions in a Mixture 155

7.4 Censored Data 157

7.5 P-Value Adjustment 158

7.5.1 The Westfall–Young Approach 159

7.5.2 Passive Plus Example 159

7.5.3 Consulting Example 160

7.6 Bioequivalence 162

7.6.1 Individual Bioequivalence 162

7.6.2 Population Bioequivalence 165

7.7 Process Capability Indices 165

7.8 Missing Data 172

7.9 Point Processes 174

7.10 Bootstrap to Detect Outliers 176

7.11 Lattice Variables 177

7.12 Covariate Adjustment of Area Under the Curve Estimates for Receiver Operating Characteristic (ROC) Curves 177

7.13 Bootstrapping in SAS 179

7.14 Historical Notes 182

7.15 Exercises 183

References 185

8 When the Bootstrap is Inconsistent and How to Remedy it 190

8.1 Too Small of a Sample Size 191

8.2 Distributions with Infinite Second Moments 191

8.2.1 Introduction 191

8.2.2 Example of Inconsistency 192

8.2.3 Remedies 193

8.3 Estimating Extreme Values 194

8.3.1 Introduction 194

8.3.2 Example of Inconsistency 194

8.3.3 Remedies 194

8.4 Survey Sampling 195

8.4.1 Introduction 195

8.4.2 Example of Inconsistency 195

8.4.3 Remedies 195

8.5 m-Dependent Sequences 196

8.5.1 Introduction 196

8.5.2 Example of Inconsistency When Independence Is Assumed 196

8.5.3 Remedy 197

8.6 Unstable Autoregressive Processes 197

8.6.1 Introduction 197

8.6.2 Example of Inconsistency 197

8.6.3 Remedies 197

8.7 Long-Range Dependence 198

8.7.1 Introduction 198

8.7.2 Example of Inconsistency 198

8.7.3 A Remedy 198

8.8 Bootstrap Diagnostics 199

8.9 Historical Notes 199

8.10 Exercises 201

References 201

Author Index 204

Subject Index 210