Advanced Kalman Filtering, Least-Squares and Modeling: A Practical HandbookISBN: 978-0-470-52970-6
Hardcover
632 pages
March 2011
This is a Print-on-Demand title. It will be printed specifically to fill your order. Please allow an additional 15-20 days delivery time. The book is not returnable.
|
PREFACE xv
1 INTRODUCTION 1
1.1 The Forward and Inverse Modeling Problem 2
1.2 A Brief History of Estimation 4
1.3 Filtering, Smoothing, and Prediction 8
1.4 Prerequisites 9
1.5 Notation 9
1.6 Summary 11
2 SYSTEM DYNAMICS AND MODELS 13
2.1 Discrete-Time Models 14
2.2 Continuous-Time Dynamic Models 17
2.2.1 State Transition and Process Noise Covariance Matrices 19
2.2.2 Dynamic Models Using Basic Function Expansions 22
2.2.3 Dynamic Models Derived from First Principles 25
2.2.4 Stochastic (Random) Process Models 31
2.2.5 Linear Regression Models 42
2.2.6 Reduced-Order Modeling 44
2.3 Computation of State Transition and Process Noise Matrices 45
2.3.1 Numeric Computation of Φ 45
2.3.2 Numeric Computation of QD 57
2.4 Measurement Models 58
2.5 Simulating Stochastic Systems 60
2.6 Common Modeling Errors and System Biases 62
2.7 Summary 65
3 MODELING EXAMPLES 67
3.1 Angle-Only Tracking of Linear Target Motion 67
3.2 Maneuvering Vehicle Tracking 69
3.2.1 Maneuvering Tank Tracking Using Multiple Models 69
3.2.2 Aircraft Tracking 73
3.3 Strapdown Inertial Navigation System (INS) Error Model 74
3.4 Spacecraft Orbit Determination (OD) 80
3.4.1 Geopotential Forces 83
3.4.2 Other Gravitational Attractions 86
3.4.3 Solar Radiation Pressure 87
3.4.4 Aerodynamic Drag 88
3.4.5 Thrust Forces 89
3.4.6 Earth Motion 89
3.4.7 Numerical Integration and Computation of Φ 90
3.4.8 Measurements 92
3.4.9 GOES I-P Satellites 96
3.4.10 Global Positioning System (GPS) 97
3.5 Fossil-Fueled Power Plant 99
3.6 Summary 99
4 LINEAR LEAST-SQUARES ESTIMATION: FUNDAMENTALS 101
4.1 Least-Squares Data Fitting 101
4.2 Weighted Least Squares 108
4.3 Bayesian Estimation 115
4.3.1 Bayesian Least Squares 115
4.3.2 Bayes’ Theorem 117
4.3.3 Minimum Variance or Minimum Mean-Squared Error (MMSE) 121
4.3.4 Orthogonal Projections 124
4.4 Probabilistic ApproachesMaximum Likelihood and Maximum A Posteriori 125
4.4.1 Gaussian Random Variables 126
4.4.2 Maximum Likelihood Estimation 128
4.4.3 Maximum A Posteriori 133
4.5 Summary of Linear Estimation Approaches 137
5 LINEAR LEAST-SQUARES ESTIMATION: SOLUTION TECHNIQUES 139
5.1 Matrix Norms, Condition Number, Observability, and the Pseudo-Inverse 139
5.1.1 Vector-Matrix Norms 139
5.1.2 Matrix Pseudo-Inverse 141
5.1.3 Condition Number 141
5.1.4 Observability 145
5.2 Normal Equation Formation and Solution 145
5.2.1 Computation of the Normal Equations 145
5.2.2 Cholesky Decomposition of the Normal Equations 149
5.3 Orthogonal Transformations and the QR Method 156
5.3.1 Givens Rotations 158
5.3.2 Householder Transformations 159
5.3.3 Modified Gram-Schmidt (MGS) Orthogonalization 162
5.3.4 QR Numerical Accuracy 165
5.4 Least-Squares Solution Using the SVD 165
5.5 Iterative Techniques 167
5.5.1 Sparse Array Storage 167
5.5.2 Linear Iteration 168
5.5.3 Least-Squares Solution for Large Sparse Problems Using Krylov Space Methods 169
5.6 Comparison of Methods 175
5.6.1 Solution Accuracy for Polynomial Problem 175
5.6.2 Algorithm Timing 181
5.7 Solution Uniqueness, Observability, and Condition Number 183
5.8 Pseudo-Inverses and the Singular Value Transformation (SVD) 185
5.9 Summary 190
6 LEAST-SQUARES ESTIMATION: MODEL ERRORS AND MODEL ORDER 193
6.1 Assessing the Validity of the Solution 194
6.1.1 Residual Sum-of-Squares (SOS) 194
6.1.2 Residual Patterns 195
6.1.3 Subsets of Residuals 196
6.1.4 Measurement Prediction 196
6.1.5 Estimate Comparison 197
6.2 Solution Error Analysis 208
6.2.1 State Error Covariance and Confidence Bounds 208
6.2.2 Model Error Analysis 212
6.3 Regression Analysis for Weighted Least Squares 237
6.3.1 Analysis of Variance 238
6.3.2 Stepwise Regression 239
6.3.3 Prediction and Optimal Data Span 244
6.4 Summary 245
7 LEAST-SQUARES ESTIMATION: CONSTRAINTS, NONLINEAR MODELS, AND ROBUST TECHNIQUES 249
7.1 Constrained Estimates 249
7.1.1 Least-Squares with Linear Equality Constraints (Problem LSE) 249
7.1.2 Least-Squares with Linear Inequality Constraints (Problem LSI) 256
7.2 Recursive Least Squares 257
7.3 Nonlinear Least Squares 259
7.3.1 1-D Nonlinear Least-Squares Solutions 263
7.3.2 Optimization for Multidimensional Unconstrained Nonlinear Least Squares 264
7.3.3 Stopping Criteria and Convergence Tests 269
7.4 Robust Estimation 282
7.4.1 De-Weighting Large Residuals 282
7.4.2 Data Editing 283
7.5 Measurement Preprocessing 285
7.6 Summary 286
8 KALMAN FILTERING 289
8.1 Discrete-Time Kalman Filter 290
8.1.1 Truth Model 290
8.1.2 Discrete-Time Kalman Filter Algorithm 291
8.2 Extensions of the Discrete Filter 303
8.2.1 Correlation between Measurement and Process Noise 303
8.2.2 Time-Correlated (Colored) Measurement Noise 305
8.2.3 Innovations, Model Validation, and Editing 311
8.3 Continous-Time Kalman-Bucy Filter 314
8.4 Modifications of the Discrete Kalman Filter 321
8.4.1 Friedland Bias-FreeBias-Restoring Filter 321
8.4.2 Kalman-Schmidt Consider Filter 325
8.5 Steady-State Solution 328
8.6 Wiener Filter 332
8.6.1 Wiener-Hopf Equation 333
8.6.2 Solution for the Optimal Weighting Function 335
8.6.3 Filter Input Covariances 336
8.6.4 Equivalence of Weiner and Steady-State Kalman-Bucy Filters 337
8.7 Summary 341
9 FILTERING FOR NONLINEAR SYSTEMS, SMOOTHING, ERROR ANALYSISMODEL DESIGN, AND
MEASUREMENT PREPROCESSING 343
9.1 Nonlinear Filtering 344
9.1.1 Linearized and Extended Kalman Filters 344
9.1.2 Iterated Extended Kalman Filter 349
9.2 Smoothing 352
9.2.1 Fixed-Point Smoother 353
9.2.2 Fixed-Lag Smoother 356
9.2.3 Fixed-Interval Smoother 357
9.3 Filter Error Analysis and Reduced-Order Modeling 370
9.3.1 Linear Analysis of Independent Error Sources 372
9.3.2 Error Analysis for ROM Defi ned as a Transformed Detailed Model 380
9.3.3 Error Analysis for Different Truth and Filter Models 382
9.4 Measurement Preprocessing 385
9.5 Summary 385
10 FACTORED (SQUARE-ROOT) FILTERING 389
10.1 Filter Numerical Accuracy 390
10.2 U-D Filter 392
10.2.1 U-D Filter Measurement Update 394
10.2.2 U-D Filter Time Update 396
10.2.3 RTS Smoother for U-D Filter 401
10.2.4 U-D Error Analysis 403
10.3 Square Root Information Filter (SRIF) 404
10.3.1 SRIF Time Update 405
10.3.2 SRIF Measurement Update 407
10.3.3 Square Root Information Smoother (SRIS) 408
10.3.4 Dyer-McReynolds Covariance Smoother (DMCS) 410
10.3.5 SRIF Error Analysis 410
10.4 Inertial Navigation System (INS) Example Using Factored Filters 412
10.5 Large Sparse Systems and the SRIF 417
10.6 Spatial Continuity Constraints and the SRIF Data Equation 419
10.6.1 Flow Model 421
10.6.2 Log Conductivity Spatial Continuity Model 422
10.6.3 Measurement Models 424
10.6.4 SRIF Processing 424
10.6.5 Steady-State Flow Constrained Iterative Solution 425
10.7 Summary 427
11 ADVANCED FILTERING TOPICS 431
11.1 Maximum Likelihood Parameter Estimation 432
11.1.1 Calculation of the State Transition Partial Derivatives 434
11.1.2 Derivatives of the Filter Time Update 438
11.1.3 Derivatives of the Filter Measurement Update 439
11.1.4 Partial Derivatives for Initial Condition Errors 440
11.1.5 Computation of the Log Likelihood and Scoring Step 441
11.2 Adaptive Filtering 449
11.3 Jump Detection and Estimation 450
11.3.1 Jump-Free Filter Equations 452
11.3.2 Stepwise Regression 454
11.3.3 Correction of Jump-Free Filter State 455
11.3.4 Real-Time Jump Detection Using Stepwise Regression 456
11.4 Adaptive Target Tracking Using Multiple Model Hypotheses 461
11.4.1 Weighted Sum of Filter Estimates 462
11.4.2 Maximum Likelihood Filter Selection 463
11.4.3 Dynamic and Interactive Multiple Models 464
11.5 Constrained Estimation 471
11.6 Robust Estimation: H-Infi nity Filters 471
11.7 Unscented Kalman Filter (UKF) 474
11.7.1 Unscented Transform 475
11.7.2 UKF Algorithm 478
11.8 Particle Filters 485
11.9 Summary 490
12 EMPIRICAL MODELING 493
12.1 Exploratory Time Series Analysis and System Identification 494
12.2 Spectral Analysis Based on the Fourier Transform 495
12.2.1 Fourier Series for Periodic Functions 497
12.2.2 Fourier Transform of Continuous Energy Signals 498
12.2.3 Fourier Transform of Power Signals 502
12.2.4 Power Spectrum of Stochastic Signals 504
12.2.5 Time-Limiting Window Functions 506
12.2.6 Discrete Fourier Transform 509
12.2.7 Periodogram Computation of Power Spectra 512
12.2.8 Blackman-Tukey (Correlogram) Computation of Power Spectra 514
12.3 Autoregressive Modeling 522
12.3.1 Maximum Entropy Method (MEM) 524
12.3.2 Burg MEM 525
12.3.3 Final Prediction Error (FPE) and Akaike Information Criteria (AIC) 526
12.3.4 Marple AR Spectral Analysis 528
12.3.5 Summary of MEM Modeling Approaches 529
12.4 ARMA Modeling 531
12.4.1 ARMA Parameter Estimation 532
12.5 Canonical Variate Analysis 534
12.5.1 CVA Derivation and Overview 536
12.5.2 Summary of CVA Steps 539
12.5.3 Sample Correlation Matrices 540
12.5.4 Order Selection Using the AIC 541
12.5.5 State-Space Model 543
12.5.6 Measurement Power Spectrum Using the State-Space Model 544
12.6 Conversion from Discrete to Continuous Models 548
12.7 Summary 551
APPENDIX A SUMMARY OF VECTORMATRIX OPERATIONS 555
A.1 Definition 555
A.1.1 Vectors 555
A.1.2 Matrices 555
A.2 Elementary Vector Matrix Operations 557
A.2.1 Transpose 557
A.2.2 Addition 557
A.2.3 Inner (Dot) Product of Vectors 557
A.2.4 Outer Product of Vectors 558
A.2.5 Multiplication 558
A.3 Matrix Functions 558
A.3.1 Matrix Inverse 558
A.3.2 Partitioned Matrix Inversion 559
A.3.3 Matrix Inversion Identity 560
A.3.4 Determinant 561
A.3.5 Matrix Trace 562
A.3.6 Derivatives of Matrix Functions 563
A.3.7 Norms 564
A.4 Matrix Transformations and Factorization 565
A.4.1 LU Decomposition 565
A.4.2 Cholesky Factorization 565
A.4.3 Similarity Transformation 566
A.4.4 Eigen Decomposition 566
A.4.5 Singular Value Decomposition (SVD) 566
A.4.6 Pseudo-Inverse 567
A.4.7 Condition Number 568
APPENDIX B PROBABILITY AND RANDOM VARIABLES 569
B.1 Probability 569
B.1.1 Definitions 569
B.1.2 Joint and Conditional Probability, and Independence 570
B.2 Random Variable 571
B.2.1 Distribution and Density Functions 571
B.2.2 Bayes’ Theorem for Density Functions 572
B.2.3 Moments of Random Variables 573
B.2.4 Gaussian Distribution 574
B.2.5 Chi-Squared Distribution 574
B.3 Stochastic Processes 575
B.3.1 Wiener or Brownian Motion Process 576
B.3.2 Markov Process 576
B.3.3 Differential and Integral Equations with White Noise Inputs 577
BIBLIOGRAPHY 579
INDEX 599