Video creates motion by switching still images rapidly, based on the same principle as a "flipbook." However, storing a sequence of still images as it is would result in an enormous amount of data. Now that video streaming has become commonplace, we will learn about the technologies of digital video representation and compression that support it behind the scenes.
The Nature of Video — Frames and Frame Rate
Video represents motion by displaying still images called frames in succession. The number of frames displayed per second is called the frame rate, and its unit is fps (frames per second). Television and many videos use 30fps (or 29.97fps), movies have traditionally used 24fps, and 60fps or higher is also used in games and sports footage when smoothness is desired. The higher the frame rate, the smoother the motion, but the amount of data also increases in proportion to the number of frames. Video files contain audio in addition to video, and store both in synchronization.
Data Size of Uncompressed Video — Why Compression Is Essential
The data size of uncompressed video can be estimated as data size of one frame × frame rate × number of seconds（＋audio）. One frame of Full HD (1920×1080・24-bit color) is approximately 6.2MB. At 30fps, a video produces approximately 187MB per second, amounting to approximately 11GB per minute. This would not even fit on a DVD, making online distribution impossible. The reason actual videos can be watched easily on smartphones is powerful compression technology that reduces the data size to an order of one-hundredth or less.
How Video Compression Works — Intra-Frame and Inter-Frame Compression
The key to video compression is a combination of two types of compression. ①Intra-frame compression（spatial compression）: compressing each frame as a single image, like JPEG. ②Inter-frame compression（temporal compression）: taking advantage of the fact that consecutive frames in a video are almost identical, and recording only the differences between adjacent frames（the changed parts）. For example, in footage of a news anchor speaking with a fixed camera, the background barely changes, so the data can be greatly reduced by recording only the differences in the moving mouth and facial expressions. The basic structure is to insert a complete reference frame (a keyframe) at regular intervals and connect the frames between them using difference information. Because of this method, footage with frequent scene changes is difficult to compress, while footage with little movement compresses well. Representative compression methods (codecs) include H.264（MPEG-4 AVC）, its successor H.265, and AV1.
Containers and Codecs, and Streaming
The video file format MP4 is, strictly speaking, a format for storing video, audio, subtitles, and other elements together in a single "container." The "codec" (such as H.264) determines how the video inside is compressed, so containers and codecs are separate concepts (the relationship is that a box called MP4 contains video compressed with H.264 and audio compressed with AAC). Video distribution methods include the download method, in which the entire file is saved before playback, and the streaming method, in which playback occurs while the data is being received. Streaming also uses technology that automatically switches the image quality (bit rate) according to the viewer's communication speed. This is why image quality becomes rougher when the connection slows down. Bit rate (the amount of data per second, measured in bps) is a central indicator that determines the balance between image quality and data size.
- Video＝a sequence of frames (still images)＋audio. Frame rate (fps)＝the number of frames per second.
- Uncompressed Full HD at 30fps is approximately 11GB per minute. Compression is essential for video.
- Intra-frame compression＝compressing each frame as an image. Inter-frame compression＝recording only the differences between adjacent frames.
- The structure consists of keyframes＋differences. Footage with less movement compresses better.
- A codec (H.264, etc.＝compression method) and a container (MP4, etc.＝storage format) are separate concepts.
- Streaming＝playing video while receiving it. The bit rate (bps) determines image quality and data size.
Practice Questions
- Explain what frame rate is, and describe how videos at 30fps and 60fps differ in appearance and data size.
- Calculate approximately how many MB per second are in an uncompressed Full HD (1920×1080・24-bit color) video at 30fps (assume 1MB＝10 to the 6th power bytes. Exclude audio).
- Explain how inter-frame compression works, and state which can be compressed more effectively—a lecture video filmed with a fixed camera or an action movie—along with the reason.
- Explain the relationship between "MP4" and "H.264" using the terms container and codec.
Answers and Explanations
- Answer：Frame rate is the number of still images (frames) displayed per second in a video, and its unit is fps. Video at 60fps appears smoother than video at 30fps, but because the number of frames doubles, the data size also becomes approximately twice as large.
Explanation：The key is whether both the definition (the number of frames per second) and the "trade-off between smoothness and data size" are included. - Answer：1 frame＝1920×1080×24 bits＝49,766,400 bits＝6,220,800 bytes (approximately 6.2MB). Since the frame rate is 30fps, the amount per second is 6,220,800 × 30 ＝ 186,624,000 bytes ≒ approximately 187MB.
Explanation：Calculate in two stages: "data size of one frame×frame rate." This enormous size is why compression is essential. For one minute, multiply by 60 to get approximately 11GB. - Answer：Inter-frame compression takes advantage of the fact that consecutive frames are almost identical. Except for the keyframes that serve as references, it records only the differences (the changed parts) from the preceding and following frames. A lecture video filmed with a fixed camera can be compressed more effectively. Because the background barely changes, the differences between frames are small, so less information needs to be recorded. In an action movie with scene changes and intense movement, the differences are large, making compression less effective.
Explanation：The answer should include both the mechanism of "recording only the differences" and the application that "less change＝smaller differences＝better compression." - Answer：H.264 is a codec (compression method) that specifies how video is compressed, while MP4 is a container (storage format) for storing compressed video, audio, subtitles, and other elements together. The relationship is that "a box called MP4 contains video compressed with H.264 and audio compressed with AAC."
Explanation：The essential point is distinguishing between a "compression method" and a "container." This also explains why the same MP4 may fail to play if the codec inside is different.