Advanced Digital Signal Processing and Noise Reduction, 4th Edition

Preface xix

Acknowledgements xxiii

Symbols xxv

Abbreviations xxix

1 Introduction 1

1.1 Signals, Noise and Information 1

1.2 Signal Processing Methods 3

1.3 Applications of Digital Signal Processing 6

1.4 A Review of Sampling and Quantisation 22

1.5 Summary 32

Bibliography 32

2 Noise and Distortion 35

2.1 Introduction 35

2.2 White Noise 37

2.3 Coloured Noise; Pink Noise and Brown Noise 39

2.4 Impulsive and Click Noise 39

2.5 Transient Noise Pulses 41

2.6 Thermal Noise 41

2.7 Shot Noise 42

2.8 Flicker (I/f ) Noise 43

2.9 Burst Noise 44

2.10 Electromagnetic (Radio) Noise 45

2.11 Channel Distortions 46

2.12 Echo and Multi-path Reflections 47

2.13 Modelling Noise 47

Bibliography 50

3 Information Theory and Probability Models 51

3.1 Introduction: Probability and Information Models 52

3.2 Random Processes 53

3.3 Probability Models of Random Signals 57

3.4 Information Models 64

3.5 Stationary and Non-Stationary Random Processes 73

3.6 Statistics (Expected Values) of a Random Process 76

3.7 Some Useful Practical Classes of Random Processes 87

3.8 Transformation of a Random Process 98

3.9 Search Engines: Citation Ranking 103

3.10 Summary 104

Bibliography 105

4 Bayesian Inference 107

4.1 Bayesian Estimation Theory: Basic Definitions 108

4.2 Bayesian Estimation 117

4.3 Expectation-Maximisation (EM) Method 128

4.4 Cramer–Rao Bound on the Minimum Estimator Variance 131

4.5 Design of Gaussian Mixture Models (GMMs) 134

4.6 Bayesian Classification 136

4.7 Modelling the Space of a Random Process 143

4.8 Summary 145

Bibliography 146

5 Hidden Markov Models 147

5.1 Statistical Models for Non-Stationary Processes 147

5.2 Hidden Markov Models 149

5.3 Training Hidden Markov Models 155

5.4 Decoding Signals Using Hidden Markov Models 161

5.5 HMMs in DNA and Protein Sequences 164

5.6 HMMs for Modelling Speech and Noise 165

5.7 Summary 171

Bibliography 171

6 Least Square ErrorWiener-Kolmogorov Filters 173

6.1 Least Square Error Estimation:Wiener-Kolmogorov Filter 173

6.2 Block-Data Formulation of theWiener Filter 178

6.3 Interpretation ofWiener Filter as Projection in Vector Space 179

6.4 Analysis of the Least Mean Square Error Signal 181

6.5 Formulation ofWiener Filters in the Frequency Domain 182

6.6 Some Applications ofWiener Filters 183

6.7 Implementation ofWiener Filters 188

6.8 Summary 191

Bibliography 191

7 Adaptive Filters: Kalman, RLS, LMS 193

7.1 Introduction 194

7.2 State-Space Kalman Filters 195

7.3 Extended Kalman Filter (EFK) 206

7.4 Unscented Kalman Filter (UFK) 208

7.5 Sample Adaptive Filters – LMS, RLS 211

7.6 Recursive Least Square (RLS) Adaptive Filters 213

7.7 The Steepest-Descent Method 217

7.8 Least Mean Squared Error (LMS) Filter 220

7.9 Summary 223

Bibliography 224

8 Linear Prediction Models 227

8.1 Linear Prediction Coding 227

8.2 Forward, Backward and Lattice Predictors 236

8.3 Short-Term and Long-Term Predictors 243

8.4 MAP Estimation of Predictor Coefficients 245

8.5 Formant-Tracking LP Models 247

8.6 Sub-Band Linear Prediction Model 248

8.7 Signal Restoration Using Linear Prediction Models 249

8.8 Summary 254

Bibliography 254

9 Eigenvalue Analysis and Principal Component Analysis 257

9.1 Introduction – Linear Systems and Eigen Analysis 257

9.2 Eigen Vectors and Eigenvalues 261

9.3 Principal Component Analysis (PCA) 264

9.4 Summary 269

Bibliography 270

10 Power Spectrum Analysis 271

10.1 Power Spectrum and Correlation 271

10.2 Fourier Series: Representation of Periodic Signals 272

10.3 Fourier Transform: Representation of Non-periodic Signals 274

10.4 Non-Parametric Power Spectrum Estimation 279

10.5 Model-Based Power Spectrum Estimation 283

10.6 High-Resolution Spectral Estimation Based on Subspace Eigen-Analysis 287

10.7 Summary 293

Bibliography 293

11 Interpolation – Replacement of Lost Samples 295

11.1 Introduction 295

11.2 Polynomial Interpolation 301

11.3 Model-Based Interpolation 306

11.4 Summary 319

Bibliography 319

12 Signal Enhancement via Spectral Amplitude Estimation 321

12.1 Introduction 321

12.2 Spectral Subtraction 324

12.3 Bayesian MMSE Spectral Amplitude Estimation 333

12.4 Estimation of Signal to Noise Ratios 335

12.5 Application to Speech Restoration and Recognition 336

12.6 Summary 338

Bibliography 338

13 Impulsive Noise: Modelling, Detection and Removal 341

13.1 Impulsive Noise 341

13.2 Autocorrelation and Power Spectrum of Impulsive Noise 344

13.3 Probability Models of Impulsive Noise 345

13.4 Impulsive Noise Contamination, Signal to Impulsive Noise Ratio 349

13.5 Median Filters for Removal of Impulsive Noise 350

13.6 Impulsive Noise Removal Using Linear Prediction Models 351

13.7 Robust Parameter Estimation 355

13.8 Restoration of Archived Gramophone Records 357

13.9 Summary 358

Bibliography 358

14 Transient Noise Pulses 359

14.1 Transient NoiseWaveforms 359

14.2 Transient Noise Pulse Models 361

14.3 Detection of Noise Pulses 364

14.4 Removal of Noise Pulse Distortions 366

14.5 Summary 369

Bibliography 369

15 Echo Cancellation 371

15.1 Introduction: Acoustic and Hybrid Echo 371

15.2 Echo Return Time: The Sources of Delay in Communication Networks 373

15.3 Telephone Line Hybrid Echo 375

15.4 Hybrid (Telephone Line) Echo Suppression 377

15.5 Adaptive Echo Cancellation 377

15.6 Acoustic Echo 381

15.7 Sub-Band Acoustic Echo Cancellation 384

15.8 Echo Cancellation with Linear Prediction Pre-whitening 385

15.9 Multi-Input Multi-Output Echo Cancellation 386

15.10 Summary 389

Bibliography 389

16 Channel Equalisation and Blind Deconvolution 391

16.1 Introduction 391

16.2 Blind Equalisation Using Channel Input Power Spectrum 398

16.3 Equalisation Based on Linear Prediction Models 400

16.4 Bayesian Blind Deconvolution and Equalisation 402

16.5 Blind Equalisation for Digital Communication Channels 409

16.6 Equalisation Based on Higher-Order Statistics 414

16.7 Summary 420

Bibliography 421

17 Speech Enhancement: Noise Reduction, Bandwidth Extension and Packet Replacement 423

17.1 An Overview of Speech Enhancement in Noise 424

17.2 Single-Input Speech Enhancement Methods 425

17.3 Speech Bandwidth Extension–Spectral Extrapolation 442

17.4 Interpolation of Lost Speech Segments–Packet Loss Concealment 447

17.5 Multi-Input Speech Enhancement Methods 455

17.6 Speech Distortion Measurements 462

Bibliography 464

18 Multiple-Input Multiple-Output Systems, Independent Component Analysis 467

18.1 Introduction 467

18.2 A note on comparison of beam-forming arrays and ICA 469

18.3 MIMO Signal Propagation and Mixing Models 469

18.4 Independent Component Analysis 472

18.5 Summary 490

Bibliography 490

19 Signal Processing in Mobile Communication 491

19.1 Introduction to Cellular Communication 491

19.2 Communication Signal Processing in Mobile Systems 497

19.3 Capacity, Noise, and Spectral Efficiency 498

19.4 Multi-path and Fading in Mobile Communication 500

19.5 Smart Antennas – Space–Time Signal Processing 505

19.6 Summary 508

Bibliography 508

Index 509