Visual Media Coding and Transmission

VISNET II Researchers xiii

Preface xv

Glossary of Abbreviations xvii

1 Introduction 1

2 Video Coding Principles 7

2.1 Introduction 7

2.2 Redundancy in Video Signals 7

2.3 Fundamentals of Video Compression 8

2.3.1 Video Signal Representation and Picture Structure 8

2.3.2 Removing Spatial Redundancy 9

2.3.3 Removing Temporal Redundancy 14

2.3.4 Basic Video Codec Structure 16

2.4 Advanced Video Compression Techniques 17

2.4.1 Frame Types 17

2.4.2 MC Accuracy 19

2.4.3 MB Mode Selection 20

2.4.4 Integer Transform 21

2.4.5 Intra Prediction 22

2.4.6 Deblocking Filters 22

2.4.7 Multiple Reference Frames and Hierarchical Coding 24

2.4.8 Error-Robust Video Coding 24

2.5 Video Codec Standards 28

2.5.1 Standardization Bodies 28

2.5.2 ITU Standards 29

2.5.3 MPEG Standards 29

2.5.4 H.264/MPEG-4 AVC 31

2.6 Assessment of Video Quality 31

2.6.1 Subjective Performance Evaluation 31

2.6.2 Objective Performance Evaluation 32

2.7 Conclusions 35

References 36

3 Scalable Video Coding 39

3.1 Introduction 39

3.1.1 Applications and Scenarios 40

3.2 Overview of the State of the Art 41

3.2.1 Scalable Coding Techniques 42

3.2.2 Multiple Description Coding 45

3.2.3 Stereoscopic 3D Video Coding 47

3.3 Scalable Video Coding Techniques 48

3.3.1 Scalable Coding for Shape, Texture, and Depth for 3D Video 48

3.3.2 3D Wavelet Coding 68

3.4 Error Robustness for Scalable Video and Image Coding 74

3.4.1 Correlated Frames for Error Robustness 74

3.4.2 Odd–Even Frame Multiple Description Coding for Scalable H.264/AVC 82

3.4.3 Wireless JPEG 2000: JPWL 91

3.4.4 JPWL Simulation Results 94

3.4.5 Towards a Theoretical Approach for Optimal Unequal Error Protection 96

3.5 Conclusions 98

References 99

4 Distributed Video Coding 105

4.1 Introduction 105

4.1.1 The Video Codec Complexity Balance 106

4.2 Distributed Source Coding 109

4.2.1 The Slepian–Wolf Theorem 109

4.2.2 The Wyner–Ziv Theorem 110

4.2.3 DVC Codec Architecture 111

4.2.4 Input Bitstream Preparation – Quantization and Bit Plane Extraction 112

4.2.5 Turbo Encoder 112

4.2.6 Parity Bit Puncturer 114

4.2.7 Side Information 114

4.2.8 Turbo Decoder 115

4.2.9 Reconstruction: Inverse Quantization 116

4.2.10 Key Frame Coding 117

4.3 Stopping Criteria for a Feedback Channel-based Transform Domain Wyner–Ziv Video Codec 118

4.3.1 Proposed Technical Solution 118

4.3.2 Performance Evaluation 120

4.4 Rate-distortion Analysis of Motion-compensated Interpolation at the Decoder in Distributed Video Coding 122

4.4.1 Proposed Technical Solution 122

4.4.2 Performance Evaluation 126

4.5 Nonlinear Quantization Technique for Distributed Video Coding 129

4.5.1 Proposed Technical Solution 129

4.5.2 Performance Evaluation 132

4.6 Symmetric Distributed Coding of Stereo Video Sequences 134

4.6.1 Proposed Technical Solution 134

4.6.2 Performance Evaluation 137

4.7 Studying Error-resilience Performance for a Feedback Channel-based Transform Domain Wyner–Ziv Video Codec 139

4.7.1 Proposed Technical Solution 139

4.7.2 Performance Evaluation 140

4.8 Modeling the DVC Decoder for Error-prone Wireless Channels 144

4.8.1 Proposed Technical Solution 145

4.8.2 Performance Evaluation 149

4.9 Error Concealment Using a DVC Approach for Video Streaming Applications 151

4.9.1 Proposed Technical Solution 152

4.9.2 Performance Evaluation 155

4.10 Conclusions 158

References 159

5 Non-normative Video Coding Tools 161

5.1 Introduction 161

5.2 Overview of the State of the Art 162

5.2.1 Rate Control 162

5.2.2 Error Resilience 164

5.3 Rate Control Architecture for Joint MVS Encoding and Transcoding 165

5.3.1 Problem Definition and Objectives 165

5.3.2 Proposed Technical Solution 166

5.3.3 Performance Evaluation 169

5.3.4 Conclusions 171

5.4 Bit Allocation and Buffer Control for MVS Encoding Rate Control 171

5.4.1 Problem Definition and Objectives 171

5.4.2 Proposed Technical Approach 172

5.4.3 Performance Evaluation 177

5.4.4 Conclusions 179

5.5 Optimal Rate Allocation for H.264/AVC Joint MVS Transcoding 179

5.5.1 Problem Definition and Objectives 179

5.5.2 Proposed Technical Solution 180

5.5.3 Performance Evaluation 181

5.5.4 Conclusions 182

5.6 Spatio-temporal Scene-level Error Concealment for Segmented Video 182

5.6.1 Problem Definition and Objectives 182

5.6.2 Proposed Technical Solution 183

5.6.3 Performance Evaluation 187

5.6.4 Conclusions 188

5.7 An Integrated Error-resilient Object-based Video Coding Architecture 189

5.7.1 Problem Definition and Objectives 189

5.7.2 Proposed Technical Solution 189

5.7.3 Performance Evaluation 195

5.7.4 Conclusions 195

5.8 A Robust FMO Scheme for H.264/AVC Video Transcoding 195

5.8.1 Problem Definition and Objectives 195

5.8.2 Proposed Technical Solution 195

5.8.3 Performance Evaluation 197

5.8.4 Conclusions 198

5.9 Conclusions 199

References 199

6 Transform-based Multi-view Video Coding 203

6.1 Introduction 203

6.2 MVC Encoder Complexity Reduction using a Multi-grid Pyramidal Approach 205

6.2.1 Problem Definition and Objectives 205

6.2.2 Proposed Technical Solution 205

6.2.3 Conclusions and Further Work 208

6.3 Inter-view Prediction using Reconstructed Disparity Information 208

6.3.1 Problem Definition and Objectives 208

6.3.2 Proposed Technical Solution 208

6.3.3 Performance Evaluation 210

6.3.4 Conclusions and Further Work 211

6.4 Multi-view Coding via Virtual View Generation 212

6.4.1 Problem Definition and Objectives 212

6.4.2 Proposed Technical Solution 212

6.4.3 Performance Evaluation 215

6.4.4 Conclusions and Further Work 216

6.5 Low-delay Random View Access in Multi-view Coding Using a Bit Rate-adaptive Downsampling Approach 216

6.5.1 Problem Definition and Objectives 216

6.5.2 Proposed Technical Solution 216

6.5.3 Performance Evaluation 219

6.5.4 Conclusions and Further Work 222

References 222

7 Introduction to Multimedia Communications 225

7.1 Introduction 225

7.2 State of the Art: Wireless Multimedia Communications 228

7.2.1 QoS in Wireless Networks 228

7.2.2 Constraints on Wireless Multimedia Communications 231

7.2.3 Multimedia Compression Technologies 234

7.2.4 Multimedia Transmission Issues in Wireless Networks 235

7.2.5 Resource Management Strategy in Wireless Multimedia Communications 239

7.3 Conclusions 244

References 244

8 Wireless Channel Models 247

8.1 Introduction 247

8.2 GPRS/EGPRS Channel Simulator 247

8.2.1 GSM/EDGE Radio Access Network (GERAN) 247

8.2.2 GPRS Physical Link Layer Model Description 250

8.2.3 EGPRS Physical Link Layer Model Description 252

8.2.4 GPRS Physical Link Layer Simulator 256

8.2.5 EGPRS Physical Link Layer Simulator 261

8.2.6 E/GPRS Radio Interface Data Flow Model 268

8.2.7 Real-time GERAN Emulator 270

8.2.8 Conclusion 271

8.3 UMTS Channel Simulator 272

8.3.1 UMTS Terrestrial Radio Access Network (UTRAN) 272

8.3.2 UMTS Physical Link Layer Model Description 279

8.3.3 Model Verification for Forward Link 290

8.3.4 UMTS Physical Link Layer Simulator 298

8.3.5 Performance Enhancement Techniques 307

8.3.6 UMTS Radio Interface Data Flow Model 309

8.3.7 Real-time UTRAN Emulator 312

8.3.8 Conclusion 313

8.4 WiMAX IEEE 802.16e Modeling 316

8.4.1 Introduction 316

8.4.2 WIMAX System Description 317

8.4.3 Physical Layer Simulation Results and Analysis 323

8.4.4 Error Pattern Files Generation 324

8.5 Conclusions 328

8.6 Appendix: Eb/No and DPCH_Ec/Io Calculation 329

References 330

9 Enhancement Schemes for Multimedia Transmission over Wireless Networks 333

9.1 Introduction 333

9.1.1 3G Real-time Audiovisual Requirements 333

9.1.2 Video Transmission over Mobile Communication Systems 335

9.1.3 Circuit-switched Bearers 339

9.1.4 Packet-switched Bearers 348

9.1.5 Video Communications over GPRS 350

9.1.6 GPRS Traffic Capacity 351

9.1.7 Error Performance 354

9.1.8 Video Communications over EGPRS 357

9.1.9 Traffic Characteristics 357

9.1.10 Error Performance 358

9.1.11 Voice Communication over Mobile Channels 359

9.1.12 Support of Voice over UMTS Networks 360

9.1.13 Error-free Performance 361

9.1.14 Error-prone Performance 362

9.1.15 Support of Voice over GPRS Networks 362

9.1.16 Conclusion 363

9.2 Link-level Quality Adaptation Techniques 365

9.2.1 Performance Modeling 365

9.2.2 Probability Calculation 367

9.2.3 Distortion Modeling 368

9.2.4 Propagation Loss Modeling 368

9.2.5 Energy-optimized UEP Scheme 369

9.2.6 Simulation Setup 370

9.2.7 Performance Analysis 372

9.2.8 Conclusion 373

9.3 Link Adaptation for Video Services 373

9.3.1 Time-varying Channel Model Design 374

9.3.2 Link Adaptation for Real-time Video Communications 379

9.3.3 Link Adaptation for Streaming Video Communications 389

9.3.4 Link Adaptation for UMTS 396

9.3.5 Conclusion 402

9.4 User-centric Radio Resource Management in UTRAN 403

9.4.1 Enhanced Call-admission Control Scheme 403

9.4.2 Implementation of UTRAN System-level Simulator 403

9.4.3 Performance Evaluation of Enhanced CAC Scheme 410

9.5 Conclusions 411

References 413

10 Quality Optimization for Cross-network Media Communications 417

10.1 Introduction 417

10.2 Generic Inter-networked QoS-optimization Infrastructure 418

10.2.1 State of the Art 418

10.2.2 Generic of QoS for Heterogeneous Networks 420

10.3 Implementation of a QoS-optimized Inter-networked Emulator 422

10.3.1 Emulation System Physical Link Layer Simulation 426

10.3.2 Emulation System Transmitter/Receiver Unit 428

10.3.3 QoS Mapping Architecture 428

10.3.4 General User Interface 438

10.4 Performances of Video Transmission in Inter-networked Systems 442

10.4.1 Experimental Setup 442

10.4.2 Test for the EDGE System 443

10.4.3 Test for the UMTS System 445

10.4.4 Tests for the EDGE-to-UMTS System 445

10.5 Conclusions 452

References 453

11 Context-based Visual Media Content Adaptation 455

11.1 Introduction 455

11.2 Overview of the State of the Art in Context-aware Content Adaptation 457

11.2.1 Recent Developments in Context-aware Systems 457

11.2.2 Standardization Efforts on Contextual Information for Content Adaptation 467

11.3 Other Standardization Efforts by the IETF and W3C 476

11.4 Summary of Standardization Activities 479

11.4.1 Integrating Digital Rights Management (DRM) with Adaptation 480

11.4.2 Existing DRM Initiatives 480

11.4.3 The New ‘‘Adaptation Authorization’’ Concept 481

11.4.4 Adaptation Decision 482

11.4.5 Context-based Content Adaptation 488

11.5 Generation of Contextual Information and Profiling 492

11.5.1 Types and Representations of Contextual Information 492

11.5.2 Context Providers and Profiling 494

11.5.3 User Privacy 497

11.5.4 Generation of Contextual Information 498

11.6 The Application Scenario for Context-based Adaptation of Governed Media Contents 499

11.6.1 Virtual Classroom Application Scenario 500

11.6.2 Mechanisms using Contextual Information in a Virtual Collaboration Application 502

11.6.3 Ontologies in Context-aware Content Adaptation 503

11.6.4 System Architecture of a Scalable Platform for Context-aware and DRM-enabled Content Adaptation 504

11.6.5 Context Providers 507

11.6.6 Adaptation Decision Engine 510

11.6.7 Adaptation Authorization 514

11.6.8 Adaptation Engines Stack 517

11.6.9 Interfaces between Modules of the Content Adaptation Platform 544

11.7 Conclusions 552

References 553

Index 559