Video Tracking: Theory and Practice

About the authors xv

Preface xvii

Acknowledgements xix

Notation xxi

Acronyms xxiii

1 What is video tracking? 1

1.1 Introduction 1

1.2 The design of a video tracker 2

1.2.1 Challenges 2

1.2.2 Main components 6

1.3 Problem formulation 7

1.3.1 Single-target tracking 7

1.3.2 Multi-target tracking 10

1.3.3 Definitions 11

1.4 Interactive versus automated tracking 12

1.5 Summary 13

2 Applications 15

2.1 Introduction 15

2.2 Media production and augmented reality 16

2.3 Medical applications and biological research 17

2.4 Surveillance and business intelligence 20

2.5 Robotics and unmanned vehicles 21

2.6 Tele-collaboration and interactive gaming 22

2.7 Art installations and performances 22

2.8 Summary 23

References 24

3 Feature extraction 27

3.1 Introduction 27

3.2 From light to useful information 28

3.2.1 Measuring light 28

3.2.2 The appearance of targets 30

3.3 Low-level features 32

3.3.1 Colour 32

3.3.2 Photometric colour invariants 39

3.3.3 Gradient and derivatives 42

3.3.4 Laplacian 47

3.3.5 Motion 49

3.4 Mid-level features 50

3.4.1 Edges 50

3.4.2 Interest points and interest regions 51

3.4.3 Uniform regions 56

3.5 High-level features 61

3.5.1 Background models 62

3.5.2 Object models 63

3.6 Summary 65

References 65

4 Target representation 71

4.1 Introduction 71

4.2 Shape representation 72

4.2.1 Basic models 72

4.2.2 Articulated models 73

4.2.3 Deformable models 74

4.3 Appearance representation 75

4.3.1 Template 76

4.3.2 Histograms 78

4.3.3 Coping with appearance changes 83

4.4 Summary 84

References 85

5 Localisation 89

5.1 Introduction 89

5.2 Single-hypothesis methods 90

5.2.1 Gradient-based trackers 90

5.2.2 Bayes tracking and the Kalman filter 95

5.3 Multiple-hypothesis methods 98

5.3.1 Grid sampling 99

5.3.2 Particle filter 101

5.3.3 Hybrid methods 105

5.4 Summary 111

References 111

6 Fusion 115

6.1 Introduction 115

6.2 Fusion strategies 116

6.2.1 Tracker-level fusion 116

6.2.2 Measurement-level fusion 118

6.3 Feature fusion in a Particle Filter 119

6.3.1 Fusion of likelihoods 119

6.3.2 Multi-feature resampling 121

6.3.3 Feature reliability 123

6.3.4 Temporal smoothing 126

6.3.5 Example 126

6.4 Summary 128

References 128

7 Multi-target management 131

7.1 Introduction 131

7.2 Measurement validation 132

7.3 Data association 134

7.3.1 Nearest neighbour 134

7.3.2 Graph matching 136

7.3.3 Multiple-hypothesis tracking 139

7.4 Random Finite Sets for tracking 143

7.5 Probabilistic Hypothesis Density filter 145

7.6 The Particle PHD filter 147

7.6.1 Dynamic and observation models 149

7.6.2 Birth and clutter models 151

7.6.3 Importance sampling 151

7.6.4 Resampling 152

7.6.5 Particle clustering 156

7.6.6 Examples 160

7.7 Summary 163

References 165

8 Context modeling 169

8.1 Introduction 169

8.2 Tracking with context modelling 170

8.2.1 Contextual information 170

8.2.2 Influence of the context 171

8.3 Birth and clutter intensity estimation 173

8.3.1 Birth density 173

8.3.2 Clutter density 179

8.3.3 Tracking with contextual feedback 181

8.4 Summary 184

References 184

9 Performance evaluation 185

9.1 Introduction 185

9.2 Analytical versus empirical methods 186

9.3 Ground truth 187

9.4 Evaluation scores 190

9.4.1 Localisation scores 190

9.4.2 Classification scores 193

9.5 Comparing trackers 196

9.5.1 Target life-span 197

9.5.2 Statistical significance 198

9.5.3 Repeatibility 198

9.6 Evaluation protocols 199

9.6.1 Low-level protocols 199

9.6.2 High-level protocols 203

9.7 Datasets 207

9.7.1 Surveillance 207

9.7.2 Human-computer interaction 212

9.7.3 Sport analysis 215

9.8 Summary 220

References 220

Epilogue 223

Further reading 225

Appendix A Comparative results 229

A.1 Single versus structural histogram 229

A.1.1 Experimental setup 229

A.1.2 Discussion 230

A.2 Localisation algorithms 233

A.2.1 Experimental setup 233

A.2.2 Discussion 235

A.3 Multi-feature fusion 238

A.3.1 Experimental setup 238

A.3.2 Reliability scores 240

A.3.3 Adaptive versus non-adaptive tracker 242

A.3.4 Computational complexity 248

A.4 PHD filter 248

A.4.1 Experimental setup 248

A.4.2 Discussion 250

A.4.3 Failure modalities 251

A.4.4 Computational cost 255

A.5 Context modelling 257

A.5.1 Experimental setup 257

A.5.2 Discussion 257

References 261

Index 263