Bayesian Approach to Inverse Problems

Introduction 15
Jérôme IDIER

PART I. FUNDAMENTAL PROBLEMS AND TOOLS 23

Chapter 1. Inverse Problems, Ill-posed Problems 25
Guy DEMOMENT, Jérôme IDIER

1.1. Introduction 25

1.2. Basic example 26

1.3. Ill-posed problem 30

1.3.1. Case of discrete data 31

1.3.2. Continuous case 32

1.4. Generalized inversion 34

1.4.1. Pseudo-solutions 35

1.4.2. Generalized solutions 35

1.4.3. Example 35

1.5. Discretization and conditioning 36

1.6. Conclusion 38

1.7. Bibliography 39

Chapter 2. Main Approaches to the Regularization of Ill-posed Problems 41
Guy DEMOMENT, Jérôme IDIER

2.1. Regularization 41

2.1.1. Dimensionality control 42

2.1.2. Minimization of a composite criterion 44

2.2. Criterion descent methods 48

2.2.1.Criterion minimization for inversion 48

2.2.2. The quadratic case 49

2.2.3. The convex case 51

2.2.4. General case 52

2.3. Choice of regularization coefficient 53

2.3.1. Residual error energy control 53

2.3.2. “L-curve” method 53

2.3.3. Cross-validation 54

2.4. Bibliography 56

Chapter 3. Inversion within the Probabilistic Framework 59
Guy DEMOMENT, Yves GOUSSARD

3.1. Inversion and inference 59

3.2. Statistical inference 60

3.2.1. Noise law and direct distribution for data 61

3.2.2. Maximum likelihood estimation 63

3.3. Bayesian approach to inversion 64

3.4. Links with deterministic methods 66

3.5. Choice of hyperparameters 67

3.6. A priori model68

3.7. Choice of criteria 70

3.8. The linear, Gaussian case 71

3.8.1. Statistical properties of the solution 71

3.8.2. Calculation of marginal likelihood 73

3.8.3. Wiener filtering 74

3.9. Bibliography 76

PART II. DECONVOLUTION 79

Chapter 4. Inverse Filtering and Other Linear Methods 81
Guy LE BESNERAIS, Jean-François GIOVANNELLI, Guy DEMOMENT

4.1. Introduction 81

4.2. Continuous-time deconvolution 82

4.2.1. Inverse filtering 82

4.2.2. Wiener filtering 84

4.3. Discretization of the problem 85

4.3.1. Choice of a quadrature method 85

4.3.2. Structure of observation matrix H 87

4.3.3. Usual boundary conditions 89

4.3.4. Problem conditioning 89

4.3.5.Generalized inversion 91

4.4. Batch deconvolution 92

4.4.1. Preliminary choices 92

4.4.2. Matrix form of the estimate 93

4.4.3. Hunt’s method (periodic boundary hypothesis) 94

4.4.4. Exact inversion methods in the stationary case 96

4.4.5. Case of non-stationary signals 98

4.4.6. Results and discussion on examples 98

4.5. Recursive deconvolution 102

4.5.1. Kalman filtering 102

4.5.2. Degenerate state model and recursive least squares 104

4.5.3. Autoregressive state model 105

4.5.4. Fast Kalman filtering 108

4.5.5. Asymptotic techniques in the stationary case 110

4.5.6. ARMA model and non-standard Kalman filtering 111

4.5.7. Case of non-stationary signals 111

4.5.8. On-lineprocessing: 2Dcase 112

4.6. Conclusion 112

4.7. Bibliography 113

Chapter 5. Deconvolution of Spike Trains 117
Frédéric CHAMPAGNAT, Yves GOUSSARD, Stéphane GAUTIER, Jérôme IDIER

5.1. Introduction 117

5.2. Penalization of reflectivities, L2LP/L2Hy deconvolutions 119

5.2.1. Quadratic regularization 121

5.2.2. Non-quadratic regularization 122

5.2.3. L2LPorL2Hy deconvolution 123

5.3. Bernoulli-Gaussian deconvolution 124

5.3.1. Compound BG model 124

5.3.2. Various strategies for estimation 124

5.3.3. General expression for marginal likelihood 125

5.3.4. An iterative method for BG deconvolution 126

5.3.5. Other methods 128

5.4. Examples of processing and discussion 130

5.4.1. Nature of the solutions 130

5.4.2. Setting the parameters 132

5.4.3. Numerical complexity 133

5.5. Extensions 133

5.5.1. Generalization of structures of R and H 134

5.5.2. Estimation of the impulse response . . . 134

5.6. Conclusion 136

5.7. Bibliography 137

Chapter 6. Deconvolution of Images 141
Jérôme IDIER, Laure BLANC-FÉRAUD

6.1. Introduction 141

6.2. Regularization in the Tikhonov sense 142

6.2.1. Principle 142

6.2.2. Connection with image processing by linear PDE 144

6.2.3. Limits of Tikhonov’s approach 145

6.3. Detection-estimation 148

6.3.1. Principle 148

6.3.2. Disadvantages 149

6.4. Non-quadratic approach 150

6.4.1. Detection-estimation and non-convex penalization 154

6.4.2. Anisotropic diffusion by PDE 155

6.5. Half-quadratic augmented criteria 156

6.5.1. Duality between non-quadratic criteria and HQ criteria 157

6.5.2. Minimization of HQ criteria 158

6.6. Application in image deconvolution 159

6.6.1. Calculation of the solution 159

6.6.2. Example 161

6.7. Conclusion 164

6.8. Bibliography 165

PART III. ADVANCED PROBLEMS AND TOOLS 169

Chapter 7. Gibbs-Markov Image Models 171
Jérôme IDIER

7.1. Introduction 171

7.2. Bayesian statistical framework 172

7.3. Gibbs-Markov fields 173

7.3.1. Gibbs fields 174

7.3.2. Gibbs-Markov equivalence 177

7.3.3. Posterior law of a GMRF 180

7.3.4. Gibbs-Markov models for images 181

7.4. Statistical tools, stochastic sampling 185

7.4.1. Statistical tools 185

7.4.2. Stochastic sampling 188

7.5. Conclusion 194

7.6. Bibliography 195

Chapter 8. Unsupervised Problems 197
Xavier DESCOMBES, Yves GOUSSARD

8.1. Introduction and statement of problem 197

8.2. Directly observed field 199

8.2.1. Likelihood properties 199

8.2.2. Optimization 200

8.2.3. Approximations 202

8.3. Indirectly observed field 205

8.3.1. Statement of problem 205

8.3.2. EM algorithm 206

8.3.3. Application to estimation of the parameters of a GMRF 207

8.3.4. EM algorithm and gradient 208

8.3.5. Linear GMRF relative to hyperparameters 210

8.3.6. Extensions and approximations 212

8.4. Conclusion 215

8.5. Bibliography 216

PART IV. SOME APPLICATIONS 219

Chapter 9. Deconvolution Applied to Ultrasonic Non-destructive Evaluation 221
Stéphane GAUTIER, Frédéric CHAMPAGNAT, Jérôme IDIER

9.1. Introduction 221

9.2. Example of evaluation and difficulties of interpretation 222

9.2.1. Description of the part to be inspected 222

9.2.2. Evaluation principle 222

9.2.3. Evaluation results and interpretation 223

9.2.4. Help with interpretation by restoration of discontinuities 224

9.3. Definition of direct convolution model 225

9.4. Blind deconvolution 226

9.4.1. Overview of approaches for blind deconvolution 226

9.4.2. DL2Hy/DBGd econvolution 230

9.4.3. Blind DL2Hy/DBG deconvolution 232

9.5. Processing real data 232

9.5.1. Processing by blind deconvolution 233

9.5.2. Deconvolution with a measured wave 234

9.5.3. Comparison between DL2Hy and DBG 237

9.5.4. Summary 240

9.6. Conclusion 240

9.7. Bibliography 241

Chapter 10. Inversion in Optical Imaging through Atmospheric Turbulence 243
Laurent MUGNIER, Guy LE BESNERAIS, Serge MEIMON

10.1. Optical imaging through turbulence 243

10.1.1. Introduction 243

10.1.2. Image formation 244

10.1.4. Imaging techniques 249

10.2. Inversion approach and regularization criteria used 253

10.3. Measurement of aberrations 254

10.3.1. Introduction 254

10.3.2. Hartmann-Shack sensor 255

10.3.3. Phase retrieval and phase diversity 257

10.4. Myopic restoration in imaging 258

10.4.1. Motivation and noise statistic 258

10.4.2. Data processing in deconvolution from wavefront sensing 259

10.4.3. Restoration of images corrected by adaptive optics 263

10.4.4. Conclusion 267

10.5. Image reconstruction in optical interferometry (OI) 268

10.5.1. Observation model 268

10.5.2. Traditional Bayesian approach 271

10.5.3. Myopic modeling 272

10.5.4. Results 274

10.6. Bibliography 277

Chapter 11. Spectral Characterization in Ultrasonic Doppler Velocimetry 285
Jean-François GIOVANNELLI, Alain HERMENT

11.1. Velocity measurement in medical imaging 285

11.1.1. Principle of velocity measurement in ultrasound imaging 286

11.1.2. Information carried by Doppler signals 286

11.1.3.Some characteristics and limitations 288

11.1.4. Data and problems treated 288

11.2. Adaptive spectral analysis 290

11.2.1. Least squares and traditional extensions 290

11.2.2. Long AR models – spectral smoothness – spatial continuity 291

11.2.3. Kalman smoothing 293

11.2.4. Estimation of hyperparameters 294

11.2.5. Processing results and comparisons 296

11.3. Tracking spectral moments 297

11.3.1. Proposed method 298

11.3.2. Likelihood of the hyperparameters 302

11.3.3. Processing results and comparisons 304

11.4. Conclusion 306

11.5. Bibliography 307

Chapter 12. Tomographic Reconstruction from Few Projections 311
Ali MOHAMMAD-DJAFARI, Jean-Marc DINTEN

12.1. Introduction 311

12.2. Projection generation model 312

12.3. 2D analytical methods 313

12.4. 3D analytical methods 317

12.5. Limitations of analytical methods 317

12.6. Discrete approach to reconstruction 319

12.7. Choice of criterion and reconstruction methods 321

12.8. Reconstruction algorithms 323

12.8.1. Optimization algorithms for convex criteria 323

12.8.2. Optimization or integration algorithms 327

12.9. Specific models for binary objects 328

12.10. Illustrations 328

12.10.1.2D reconstruction 328

12.10.2.3Dreconstruction 329

12.11. Conclusions 331

12.12. Bibliography 332

Chapter 13. Diffraction Tomography 335
Hervé CARFANTAN, Ali MOHAMMAD-DJAFARI

13.1. Introduction 335

13.2. Modeling the problem 336

13.2.1. Examples of diffraction tomography applications 336

13.2.2. Modeling the direct problem 338

13.3. Discretization of the direct problem 340

13.3.1. Choice of algebraic framework 340

13.3.2. Method of moments 341

13.3.3. Discretization by the method of moments 342

13.4. Construction of criteria for solving the inverse problem 343

13.4.1. First formulation: estimation of x 344

13.4.2. Second formulation: simultaneous estimation of x and φ 345

13.4.3. Properties of the criteria 347

13.5. Solving the inverse problem 347

13.5.1. Successive linearizations 348

13.5.2. Joint minimization 350

13.5.3. Minimizing MAP criterion 351

13.6. Conclusion 353

13.7. Bibliography 354

Chapter 14. Imaging from Low-intensity Data 357
Ken SAUER, Jean-Baptiste THIBAULT

14.1. Introduction 357

14.2. Statistical properties of common low-intensity image data 359

14.2.1. Likelihood functions and limiting behavior 359

14.2.2. Purely Poisson measurements 360

14.2.3. Inclusion of background counting noise 362

14.2.4. Compound noise models with Poisson information 362

14.3. Quantum-limited measurements in inverse problems 363

14.3.1. Maximum likelihood properties 363

14.3.2. Bayesian estimation 366

14.4. Implementation and calculation of Bayesian estimates 368

14.4.1. Implementation for pure Poisson model 368

14.4.2. Bayesian implementation for a compound data model 370

14.5. Conclusion 372

14.6. Bibliography 372

List of Authors 375

Index 377