Apply Now
MULTIMEDIA WEB
CONTENTS
Preface
I Multimedia Authoring and Data Representations
1 Introduction to Mullimedia
1.1 What is Multimedia? 3
1.1.1 Components of Multimedia 3
1.1.2 Multimedia Research Topics and Projects 4
1.2 Multimedia and Hypermedia 5
1.2.1 History of Mu1timedia 5
1.2.2 Hypermedia and Multimedia 7
1.3 World Wide Web 8
1.3.1 History ofthe WWW 8
1.3.2 HyperText Transfer Protocol (HTTP) 9
1.3.3 HyperText Markup Language (HTML) 10
1.3.4 Extensible Markup Language (XML) 11
1.3.5 Synchronized Multimedia Integration Language (SMIL) 12
1.4 Overview of Mu1timedia Software Tools 14
1.4.1 Music Sequencing and Notation 14
1.4.2 Digital Audio 15
1.4.3 Graphics and Image Editing 15
1.404 Video Editing 15
104.5 Animation 16
1.4.6 Mu1timedia Authorjng 17
1.5 Further Exploration 17
1.6 Exercises 18
1.7 References 19
xiv
1
3
2 l\Iultimedia Authoring and Tools 20
2.1 Multimedia Authoring 20
2.1.1 Multimedia Authoring Metaphors 21
2.1.2 Multimedia Production 23
2.1.3 Multimedia Presentation 25
2.1.4 Automatic Authoring 33
2.2 Some Useful Editing and Authoring Tools 37
2.2.1 Adobe Premiere 37
2.2.2 Macromedia Director 40
2.2.3 Macromedia Flash 46
2.2.4 Dreamweaver 51
2.3 VRML 51
2.3.1 Overview 51
2.3.2 Animation and Interactions 54
2.3.3 VR1vfL Specifics 54
2.4 Further Exploration 55
2.5 Exercises 56
2.6 References 59
V
vi
3 Graphics and hnage Data Representations
3.1 GraphicslImage Data Types 60
3.1.1 l·Bit Images 61
3.1.2 8-Bit Gray·Level Images 61
3.1.3 Image Data Types 64
3.1.4 24~Bit Color Images 64
3.1.5 8-Bit Color Images 65
3.1.6 Color Lookup Tables (LUTs) 67
3.2 Popular File Formats 71
3.2.1 GIF 71
3.2.2 JPEG 75
3.2.3 PNG 76
3.2.4 TIFF 77
3.2.5 EXIF 77
3.2.6 Graphics Animation Files 77
3.2.7 PS and PDF 78
3.2.8 Windows WMF 78
3.2.9 Windows BMF 78
3.2.10 Macintosh PAINT and PICT 78
3.2.11 X Windows PPM 79
3.3 Further Exploration 79
3.4 Exercises 79
3.5 References 81
4 Color in Image and Video
4. [ Color Science 82
4.1.1 Light and Spectra 82
4.1.2 Human Vision 84
4.1.3 Spectral Sensitivity of the Eye 84
4.1.4 Image Formation 85
4.1.5 Camera Systems 86
4.1.6 Gamma Correction 87
4.1.7 Color-Matching Functions 89
4.1.8 CIE Chromaticity Diagram 91
4.1.9 Color Monitor Specifications 94
4.1.10 Out-of-Gamut Colors 95
4.1.11 White-Point Correction 96
4.1.12 XYZ to RGB Transform 97
4.1.13 Transform with Gamma Correction 97
4.1.14 L*a*b* (CIELAB) Color Model 98
4.1.15 More Color-Coordinate Schemes 100
4.1.16 Munsell Color Naming System 100
4.2 Color'Models in Images 100
4.2.1 RGB Color Model for CRT Displays 100
4.2.2 Subtractive Color: CMY Color Model 101
4.2.3 Tr1!nsformation from RGB to CMY 101
4.2.4 Undercolor Removal: CMYK System 102
4.2.5 Printer Gamuts 102
4.3 Color Models in Video 104
4.3.1 Video Color Transforms 104
4.3.2 YUV Color Model 104
60
82
4.3.3 YIQ Color Model 105
4.3.4 YCbCr Color Model 107
4.4 Further Exploration 107
4.5 Exercises 108
4.6 References III
5 Fundamental Concepts in Video
5.1 Types of Video Signals 112
5.1.1 Component Video 112
5.1.2 Composite Video 113
5.1.3 S-Video 113
5.2 Analog Video 113
5.2.1 NTSC Video 116
5.2.2 PAL Video 119
5.2.3 SECAM Video 119
5.3 Digital Video 119
5.3.1 Chroma Subsampllng 120
5.3.2 CCIR Standards for Digital Video 120
5.3.3 High Definition TV (HDTV) 122
5.4 Further Exploration 124
5.5 Exercises 124
5.6 References 125
6 Basics of Digital Audio
6.1 Digitization of Sound 126
6.1.1 What Is Sound? 126
6.1.2 Digitization 127
6.1.3 Nyquist Theorem 128
6.1.4 Signal-to-Noise Ratio (SNR) 131
6.1.5 Signal-to-Quantization-Noise Ratio (SQNR) 131
6.1.6 Linear and Nonlinear Quantization 133
6.1.7 Audio Filtering 136
6.1.8 Audio Quality versus Data Rate 136
6.1.9 Synthetic Sounds 137
6.2 MIDI: Musical Instrument Digital Interface 139
6.2.1 MIDI Overview 139
6.2.2 Hardware Aspects of MIDI 142
6.2.3 Structure of MIDI Messages 143
6.2.4 General MIDI 147
6.2.5 MIDI-to-WAV Conversion 147
6.3 Quantization and Transmission of Audio 147
6.3.1 Coding of Audio 147
6.3.2 Pulse Code Modulation 148
6.3.3 Differential Coding of Audio 150
6.3.4 Lossless Predictive Coding 151
6.3.5 DPCM 154
6.3.6 DM 157
6.3.7 ADPCM 158
6.4 Further Exploration 159
6.5 Exercises 160
6.6 References 163
vii
112
126
viii
II lVlultimedia Data Compression
7 Lossless Compression Algorithms
7.1 Introduction 167
7.2 Basics of Infonnation Theory 168
7.3 Run-Length Coding 171
7.4 Variable-Length Coding (VLC) 171
7.4.1 Shannon-Fano Algorithm 171
7.4.2 Huffman Coding 173
7.4.3 Adaptive Huffman Coding 176
7.5 Dictionary-Based Coding 181
7.6 Arithmetic Coding 187
7.7 Lossless Image Compression 191
7.7.1 Differential Coding of Images 191
7.7.2 Lossless JPEG 193
7.8 Further Exploration 194
7.9 Exercises 195
7.10 References 197
8 Lossy Compression Algorithms
8.1 Introduction 199
8.2 Distortion Measures 199
8.3 The Rate-Distortion Theory 200
8.4 Quantization 200
8.4.1 Uniform Scalar Quantization 201
8.4.2 Nonunifonn Scalar Quantization 204
8.4.3 Vector Quantization* 206
8.5 Transform Coding 207
8.5.1 Discrete Cosine Transfonn (DCT) 207
8.5.2 Karhunen-Loeve Transform* 220
8.6 Wavelet-Based Coding 222
8.6.1 Introduction 222
8.6.2 Continuous Wavelet Transfonn* 227
8.6.3 Discrete Wavelet Transfonn* 230
8.7 Wavelet Packets 240
8.8 Embedded Zerotree of Wavelet Coefficients 241
8.8.1 The Zerotree Data Structure 242
8.8.2 Successive Approximation Quantization 244
8.8.3 EZW Example 244
8.9 Set Partitioning in Hierarchical Trees (SPIHT) 247
8.10 Further Exploration 248
8.11 Exercises 249
8.12 References 252
9 Image Compl'ession Standards
9.1 The JPEG Standard 253
9.1.1 Main Steps in IPEG Image Compression 253
9.1.2 JPEG Modes 262
9.1.3 AGlance at the JPEG Bitstream 265
9.2 The JPEG2000 Standard 265
9.2.1 Main Steps of IPEG2000 Image Compression* 267
165
167
199
253
9.2.2 Adapting EBCOT to JPEG2000 275
9.2.3 Region-of-Interest Coding 275
9.2.4 Comparison ofJPEG and JPEG2000 Performance 277
9.3 The JPEG-LS Standard 277
93.1 Prediction 280
9.3.2 Context Determination 281
9.3.3 Residual Coding 281
9.3.4 Near-Lossless Mode 281
9,4 Bilevel Image Compression Standards 282
9.4.1 The JBIG Standard 282
9.4.2 The JBIG2 Standard 282
9.5 Further Exploration 284
9.6 Exercises '285
9.7 References 287
10 Basic Video Compression Techniques
10.1 Introduction to Video Compression 288
10.2 Video Compression Based on Motion Compensation 288
10.3 Search for Motion Vectors 290
10.3.1 Sequential Search 290
10.3.2 2D Logarithmic Search 291
10.3.3 Hierarchical Search 293
10.4 H.261 295
10.4.1 Intra-Frame (I-Frame) Coding 297
10.4.2 Inter-Frame (P-Frame) Predictive Coding 297
10.4.3 Quantization in H.261 297
10.4.4 H.261 Encoder and Decoder 298
10.4.5 A Glance at the H.261 Video Bitstream Syntax 301
10.5 H.263 303
10.5.1 Motion Compensation in H.263 304
10.5.2 Optional H.263 Coding Modes 305
10.5.3 H.263+ and H.263++ 307
10.6 Further Exploration 308
10.7 Exercises 309
10.8 References 310
11 MPEG Video Coding I - MPEG-1 and 2
1L1 Overview 312
lL2 MPEG-1 312
11.2.1 Motion Compensation in MPEG-l 313
11.2.2 Other Major Differences from H.261 315
11.2.3 MPEG-l Video Bitstream 318
11.3 MPEG-2 319
11.3.1 Supporting Interlaced Video 320
11.3,2 MPEG-2 Scalabilities 323
11.3.3 Other Major Differences from MPEG-l 329
11.4 Further Exploration 330
11.5 Exercises 330
11.6 References 331
ix
288
312
x
12 MPEG Video Coding 11- MPEG-4, 7, and Beyond
12.1 Overview ofMPEG-4 332
12.2 Object-Based Visual Coding in MPEGA 335
12.2.1 VOP-Based Coding vs. Frame-Based Coding 335
12.2.2 Motion Compensation 337
12.2.3 Texture Coding 341
12.2.4 Shape Coding 343
12.2.5 Static Texture Coding 346
12.2.6 Sprite Coding 347
12.2.7 Global Motion Compensation 348
12.3 Synthetic Object Coding in MPEG-4 349
12.3.1 2D Mesh Object Coding 349
12.3.2 3D Model-based Coding 354
12.4 :tvlPEG-4 Object types, Profiles and Levels 356
12.5 MPEG-4 PartlOlH.264 357
12.5.1 Core Features 358
12.5.2 Baseline Profile Features 360
12.5.3 Main Profile Features 360
12.5.4 Extended Profile Features 361
12.6 MPEG-7 361
12.6.1 Descriptor CD) 363
12.6.2 Description Scheme CDS) 365
12.6.3 Description Definition Language CDDL) 368
12.7 MPEG-21 369
12.8 Further Exploration 370
12.9 Exercises 370
12.10 References 371
13 Basic Audio Compl'ession Techniques
13.1 ADPCM in Speech Coding 374
13.1.1 ADPCM 374
13.2 G.726 ADPCM 376
13.3 Vocoders 378
13.3.1 Phase Insensitivity 378
13.3.2 Channel Vocoder 378
13.3.3 Formant Vocoder 380
13.3.4 Linear Predictive Coding 380
13.3.5 CELl' 383
13.3.6 Hybrid Excitation Vocoders* 389
13.4 Further Exploration 392
13.5 Exercises 392
13.6 References 393
14 MPEG Audio Compl'ession
14.1 Psychoacoustics 395
14.1.1 Equal-Loudness Relations 396
14.1.2 Frequency Masking 398
14.1.3 Temporal Masking 403
14.2 MPEG Audio 405
14.2.1 MPEG Layers 405
332
374
395
14.2.2 MPEG AudIo Strategy 406
14.2.3 MPEG Audio Compression Algorithm 407
14.2.4 MPEG·2 AAC (Advanced Audio Coding) 412
14.2.5 MPEGA Audio 4]4
14.3 Other Commercial Audio Codecs 415
14.4 The Future: MPEG-7 and MPEG-21 415
14.5 FurtherExploration 416
14.6 Exercises 416
14.7 References 417
III Multimedia Communication and Retrieval
15 Computer and Multimedia Networks
15.1 Basics of Computer and Multimedia Networks 421
15.1.1 OSI Network Layers 421
15.1.2 TCPIIP Protocols 422
15.2 Multiplexing Technologies 425
15.2.1 Basics of Multiplexing 425
]5.2.2 Integrated Services Digital Network (ISDN) 427
15.2.3 Synchronous Optical NETwork (SONET) 428
15.2.4 Asymmetric Digital Subscriber Line (ADSL) 429
15.3 LAN and WAN 430
15.3.1 Local Area Networks (LANs) 431
15.3.2 Wide Area Networks (WANs) 434
15.3.3 Asynchronous Transfer Mode (ATM) 435
15.3.4 Gigabit and lO-Gigabit Ethernets 438
15.4 Access Networks 439
15.5 Common Peripheral Interfaces 441
15.6 Further Exploration 441
15.7 Exercises 442
15.8 References 442
16 Multimedia Network Communications and Applications
16.1 Quality of Multimedia Data Transmission 443
16.1.1 Quality of Service (QoS) 443
16.1.2 QoS forIPProtocols 446
16.1.3 Prioritized Delivery 447
16.2 Multimedia over IP 447
]6.2.1 IF-Multicast 447
16.2.2 RTP (Real-time Transport Protocol) 449
16.2.3 Real Time Control Protocol (RTCP) 451
16.2.4 Resource ReSerVation Protocol (RSVP) 451
16.2.5 Real·Time Streaming Protocol (RTSP) 453
16.2.6 Internet Telephony 455
16.3 Multimedia over ATM Networks 459
16.3.1 Video Bitrates over ATM 459
16.3.2 ATM Adaptation Layer (AAL) 460
16.3.3 1vIPEG-2 Convergence to ATM 461
16.3.4 Multicast over ATM 462
xi
419
421
443
xii
16.4 Transport of MPEG-4 462
16.4.1 DMIF in MPEG-4 462
16.4.2 MPEG-4 over IP 463
16.5 Media~on-Demand (MOD) 464
16.5.1 Interactive TV (ITV) and Set-Top Box (STB) 464
16.5.2 Broadcast Schemes for Video-an-Demand 465
16.5.3 Buffer Management 472
16.6 Further Exploration 475
16.7 Exercises 476
16.8 References 477
17 Wil-eless NetwOl'ks
17.1 Wireless Networks 479
17.1.1 Analog Wireless Networks 480
17.1.2 Digital Wireless Networks 481
17.1.3 TDMA and GSM 481
17.1.4 Spread Spectrum and CDMA 483
17.1.5 Analysis ofCDMA 486
17.1.6 3G Digital \Vueless Networks 488
17.1.7 Wireless LAN (WLAN) 492
17.2 Radio Propagation Models 493
17.2.1 Multipath Fading 494
17.2.2 Path Loss 496
17.3 Multimedia over Wireless Networks 496
17.3.1 Synchronization Loss 497
17.3.2 Error Resilient Entropy Coding 499
17.3.3 Error Concealment 501
17.3.4 Forward Error Correction (FEe) 503
17.3.5 Trends in Wireless Interactive Multimedia 506
17.4 Further Exploration 508
17.5 Exercises 508
17.6 References 510
18 Content-Based Retdeval in Digital UbI'aries
18.1 How Should We Retrieve Images? 511
18.2 C-BIRD - A Case Study 513
18.2.1 C-BIRD Gill 514
18.2.2 Color Histogram 514
18.2.3 Color Density 516
18.2.4 Color Layout 516
18.2.5 Texture Layout 517
18.2.6 Search by Illumination Invariance 519
18.2.07 Search by Object Model 520
18.3 Synopsis of Current Image Search Systems 533
18.3.1 QBIC 535
18.3.2 UC Santa Barbara Search Engines 536
18.3.3 Berkeley Digital Library Project 536
18.3.4 Chabot 536
18.3.5 Blobworld 537
18.3.6 Columbia University Image Seekers 537
479
511
xiii
18.3.7 Informedia 537
18.3.8 MetaSEEk 537
18.3.9 Photobook and FourEyes 538
18.3.10 MARS 538
18.3.11 Virage 538
18.3.12 Viper 538
18.3.13 Visual RetrievalWare 538
18.4 Relevance Feedback 539
18.4.1 MARS 539
18.4.2 iFind 541
18.5 Quantifying Results 541
18.6 Querying on Videos 542
18.7 Querying on Other Formats 544
18.8 Outlook for Content-Based Retrieval 544
18.9 Further Exploration 545
18.10 Exercises 546
18.11 References 547
Index 551