TL;DR
Video encoding converts raw video into a compressed digital bitstream that is practical to store, transmit, and play. A codec such as H.264, VP8, VP9, or AV1 performs the compression; a container such as MP4 or WebM packages encoded tracks and metadata.
Most streaming and real-time video uses lossy compression because it can reduce bandwidth dramatically while preserving acceptable visual quality. Lossless compression retains all source information but usually produces files or bitrates too large for consumer streaming. For calls and interactive experiences, the best codec is not simply the one with the highest theoretical quality—it must also encode quickly, work across target devices, and adapt to changing networks.
What Video Encoding Actually Means
Video encoding is the process of converting raw or lightly processed video frames into a smaller digital representation suitable for storage, distribution, or real-time transmission. An encoder analyzes the frames, identifies information that can be represented more efficiently, and outputs a compressed bitstream.
Without encoding, video data becomes impractically large. Consider a raw 1920 × 1080 video with 24-bit color at 30 frames per second:
1920 × 1080 × 24 × 30 ≈ 1.49 billion bits per second
That is roughly 186 MB per second before audio or other overhead. A single minute would consume more than 11 GB. Higher frame rates, 4K resolution, and greater color depth increase the requirement further.
An encoder reduces this volume by exploiting patterns such as:
- Similar colors within the same frame
- Background areas that change very little
- Motion that can be described instead of storing complete new images
- Visual details that viewers are less likely to notice
- Repeated information across adjacent frames
The result is not ordinarily a sequence of independent images. It is a structured bitstream containing compressed image data, references between frames, timing information, and instructions a compatible decoder can interpret.
Encoding appears throughout the video lifecycle. Cameras may encode video while recording, editing software encodes exported files, streaming platforms create multiple renditions, and real-time communication systems encode every participant’s camera feed before sending it across the network.
Decoding: Turning the Bitstream Back Into Frames
Decoding reverses the encoded representation sufficiently to produce frames for display. The receiving device reads the bitstream, reconstructs predicted frames, applies motion information, and outputs images that can be rendered on screen.
With lossless encoding, the decoded output can reproduce the original input exactly. With lossy encoding, the decoder reconstructs a close approximation because some source information was intentionally removed.
Smooth playback depends on more than decoding alone. The viewer’s device must support the codec, have enough processing capacity, and receive data quickly enough to avoid an empty playback buffer. In a real-time call, buffering must remain minimal, so decoding delays and missing packets are much more visible than they may be during on-demand playback.
A useful distinction is:
- Encoding prepares captured or stored video for efficient use.
- Decoding reconstructs frames that a viewer can display.
- Playback coordinates decoding, timing, rendering, and audio-video synchronization.
Transcoding: Creating a Different Encoded Version
Transcoding converts an existing encoded video into another codec, resolution, frame rate, or bitrate. It normally involves decoding the source and then encoding a new version.
For example, a platform might receive a 1080p H.264 stream and generate:
- 1080p at 5 Mbps
- 720p at 2.5 Mbps
- 480p at 1 Mbps
- 360p at 500 Kbps
A player can then choose a rendition based on available bandwidth and screen size. This process supports adaptive bitrate streaming and avoids sending high-resolution video to a device or network that cannot use it effectively.
Transcoding solves compatibility problems as well. A service may retain a high-quality source while generating H.264 for broad device support and newer codecs for clients that can decode them efficiently.
However, transcoding is computationally expensive, and repeated lossy transcoding can introduce visible degradation. When possible, platforms should preserve a high-quality master rather than using an already compressed delivery file as the source for every new version.
Why Video Must Be Encoded
The immediate reason for encoding is file-size reduction, but the practical benefits extend across the entire delivery system.
Lower storage and delivery requirements
Compressed files require less disk space, take less time to upload, and consume less network bandwidth. For a video platform, these savings affect storage capacity, content delivery costs, startup time, and the number of concurrent viewers the infrastructure can support.
A lower bitrate does not automatically mean a better result. Excessive compression can create blockiness, blurred detail, color banding, or unstable motion. The goal is to reach a bitrate that the network can sustain without degrading the image more than the use case permits.
Playback across different devices
Phones, laptops, browsers, televisions, and embedded devices do not all support the same codecs or decoding profiles. Encoding content into compatible formats helps it play across the intended device set.
Hardware decoding also matters. A codec may be supported in software but still consume substantial CPU and battery power. This can make an efficient modern codec less attractive for a real-time mobile application if target devices cannot accelerate it.
Streaming over changing networks
Network capacity can fluctuate from one moment to the next. Encoded video gives streaming systems levers such as bitrate, frame rate, resolution, and keyframe frequency. These parameters can be adjusted to reduce stalling or preserve conversational continuity.
For on-demand video, the player can buffer ahead and switch between pre-encoded renditions. Real-time communication has far less room for buffering, so the sender may need to adapt the active stream as conditions change.
Predictable real-time performance
Interactive calls, live classes, remote assistance, and conferences require more than small files. Frames must be encoded and delivered quickly enough for people to communicate naturally.
A configuration that is excellent for offline movie distribution may be unsuitable for a live call because it uses long processing windows or complex frame dependencies. Real-time encoding prioritizes low delay, fast recovery from packet loss, and consistent performance alongside picture quality.
The Building Blocks: Codecs and Containers
Two concepts cause much of the confusion around video formats: codecs and containers. They work together, but they solve different problems.
Video Codecs
Codec is short for coder-decoder. The encoder compresses video into a bitstream, while the decoder interprets that bitstream and reconstructs displayable frames.
A video codec typically uses two broad forms of prediction:
- Intra-frame compression finds redundancy within one frame, similar in principle to compressing a still image.
- Inter-frame compression stores differences or motion relationships between frames rather than representing every frame from scratch.
Many codecs use keyframes, also called intra-coded frames, as independent reference points. Frames between them may depend on earlier or later frames. Longer intervals between keyframes can improve compression, but they may make seeking and recovery from packet loss slower.
Choosing a codec involves several tradeoffs:
- Compression efficiency at the desired visual quality
- Encoding and decoding latency
- Browser, operating-system, and hardware support
- CPU, GPU, memory, and battery consumption
- Resilience to packet loss
- Licensing and distribution requirements
- Suitability for live or on-demand workflows
The same codec can also produce very different results depending on its profile, bitrate, resolution, frame rate, keyframe interval, and encoder implementation. Naming the codec alone does not fully describe stream quality.
Video Containers and File Formats
A container packages one or more encoded media tracks plus information needed to coordinate them. It may include:
- A video track
- One or more audio tracks
- Subtitles or captions
- Timestamps
- Chapter information
- Descriptive metadata
MP4, WebM, MOV, and Matroska are container formats. H.264, H.265, VP9, and AV1 are codecs. This distinction explains why a file extension does not always reveal how its video was compressed.
For example, an .mp4 file might contain H.264 video and AAC audio, but MP4 can support other combinations. Likewise, WebM commonly contains VP8, VP9, or AV1 video with Opus audio.
When someone asks which “video format” to use, clarify whether they mean:
- The codec that compresses the picture
- The container that packages the tracks
- The streaming protocol that transports the content
- The complete combination required by a device or platform
Compatibility depends on the complete chain. A browser might recognize a container but not the codec inside it. A live WebRTC session also transports encoded media as real-time packets rather than treating it like a downloadable MP4 file.
How Video Compression Works
Video compression removes or represents redundant information more efficiently. It makes playback and transmission possible on devices and networks that could not handle raw video.
Encoders generally exploit two kinds of redundancy:
- Spatial redundancy: Neighboring pixels within one frame often have similar values.
- Temporal redundancy: Consecutive frames often share most of the same scene.
For temporal compression, an encoder can divide a frame into regions, estimate how those regions moved, and store motion vectors plus residual differences. A stationary background may therefore require far less data than encoding it as an entirely new image 30 times each second.
Compression is controlled by several connected parameters:
- Bitrate: The amount of encoded data produced per second
- Resolution: The number of pixels in each frame
- Frame rate: The number of frames shown per second
- Keyframe interval: How often independently decodable frames appear
- Quantization: How aggressively numerical detail is simplified
Reducing bitrate without changing anything else generally increases compression artifacts. A better response to limited bandwidth may be to reduce resolution or frame rate so the available bits describe fewer pixels or frames more effectively.
The right compromise depends on the content. A talking-head call may remain clear at a modest frame rate, while sports, gaming, or demonstrations with rapid movement need more temporal detail. Screen sharing may prioritize sharp text over smooth motion.
Lossy Compression
Lossy compression permanently discards information to achieve a much greater reduction in data size. It is the standard approach for consumer video delivery and real-time communication.
The encoder attempts to remove information in ways that are less noticeable to human vision. It may reduce fine texture, simplify color detail, or quantize transform coefficients more aggressively. When the bitrate is sufficient, viewers may not notice the missing information under normal conditions.
When compression becomes too aggressive, common artifacts include:
- Block-shaped regions
- Smearing during movement
- Soft facial or text detail
- Banding in gradients
- Mosquito noise around sharp edges
- Delayed recovery after packet loss
Lossy compression is appropriate when practical transmission and storage matter more than preserving every original value. The relevant question is not whether quality is lost—it usually is—but whether the remaining quality meets the experience’s needs.
For a video call, a slightly softer image may be preferable to frozen playback. For medical review, professional mastering, or computer-vision analysis, losing subtle detail may be unacceptable.
Lossless Compression
Lossless compression reduces data size while allowing every encoded source value to be reconstructed exactly. It looks for repetition and efficient representations without permanently discarding information.
This is different from uncompressed video. Both can preserve full source information, but lossless video still uses compression; uncompressed video stores samples directly and therefore requires even more bandwidth.
Lossless compression is useful for:
- Archival or preservation workflows
- Intermediate files used during editing
- Scientific or technical imagery
- Visual effects pipelines
- Sources that will undergo further processing
Its limitation is compression ratio. Natural video contains complex motion and texture, so lossless output remains much larger than a lossy delivery stream. That makes it unsuitable for most internet calls and consumer live streaming.
A common production strategy is to retain a high-quality or lossless master and create lossy versions for distribution. This protects future editing and transcoding quality without forcing every viewer to receive an unnecessarily large stream.
Widely Used Video Codecs
No codec is universally best. The practical choice depends on compatibility, latency, hardware acceleration, operational cost, and the quality required at a given bitrate.
H.264 (AVC)
H.264, also known as AVC, remains one of the most widely supported video codecs. It offers a strong balance among visual quality, encoder speed, hardware acceleration, and device compatibility.
Its broad support makes it a common default for web video, mobile playback, live streaming, and real-time communication. H.264 is especially useful when an application must work across a mixed fleet of older and newer devices.
Its compression efficiency is lower than that of several newer codecs, meaning it may need more bandwidth for comparable quality. Licensing considerations can also matter depending on how a product is distributed and used.
H.265 (HEVC)
H.265, or HEVC, was designed to provide better compression than H.264, particularly for high-resolution content. It is frequently used for 4K delivery, broadcasting, and device ecosystems with established hardware support.
The tradeoffs include greater computational complexity and less consistent browser support. Licensing can also be more complicated. Teams should verify their exact client matrix rather than assuming that modern hardware guarantees seamless web compatibility.
VP8
VP8 has been widely associated with browser-based real-time communication. It offers reasonable quality, relatively manageable encoding complexity, and broad WebRTC-era support.
It is less efficient than VP9 or AV1 at many quality levels, but predictable real-time performance and mature support can outweigh compression gains—especially on devices where newer codecs lack hardware acceleration.
VP9
VP9 improves compression efficiency over VP8 and is commonly used for web video. It can deliver stronger quality at lower bitrates, which is valuable for higher resolutions and bandwidth-sensitive playback.
Encoding may demand more processing, and support varies by browser, platform, and hardware. Applications should test CPU usage and battery impact instead of evaluating bandwidth savings in isolation.
AV1
AV1 targets high compression efficiency and royalty-free distribution. It can reduce the bitrate needed for a given visual quality, making it attractive for large-scale video delivery and increasingly for real-time media.
Its main constraint is computational cost, particularly on devices without hardware acceleration. Real-time AV1 support is improving, but an application serving a broad audience may still need fallback codecs.
For current browser compatibility details, consult resources such as the MDN Web Docs guide to web video codecs. Test actual target devices because declared support does not guarantee acceptable performance at every resolution and frame rate.
A sensible selection process is:
- Identify the oldest devices and browsers that must be supported.
- Set maximum acceptable latency and CPU consumption.
- Measure quality at realistic network bitrates.
- Confirm hardware encode and decode support.
- Provide a compatible fallback when using a newer codec.
How to Build an App Like This with Tencent RTC
A custom calling, conferencing, or interactive live application should not treat encoding as an isolated offline step. Capture, encoding, network adaptation, transport, decoding, and rendering must operate as one low-latency pipeline.
Tencent RTC Engine provides that cross-platform real-time media core. Using trtc-sdk-v5, a web application captures local camera and microphone input, encodes and publishes it into a room, subscribes to remote streams, and adapts delivery as network conditions change.
The high-level architecture is:
- Application server: Authenticates the user and securely generates a
userSig. The secret key must never be embedded in browser code. - Web client: Uses Tencent RTC Engine to enter a room and capture local media.
- RTC infrastructure: Routes encoded streams with low-latency transport and weak-network adaptation.
- Remote clients: Subscribe to streams, decode them, and render synchronized audio and video.
- Application backend: Stores room membership and business state, but does not need to relay raw media.
Follow the Tencent RTC Engine overview and integration documentation to create an application, obtain its SDKAppID, install the appropriate SDK, and configure authentication.
First, create an instance, enter the room, and begin local capture. Starting local audio and video causes the SDK to capture, encode, and publish the tracks:
import TRTC from "trtc-sdk-v5";
const client = TRTC.create();
await client.enterRoom({
sdkAppId: 1400000000,
userId: "host-42",
userSig: userSigFromYourServer,
roomId: 9001
});
await client.startLocalVideo({
view: document.getElementById("local-video")
});
await client.startLocalAudio();Next, listen for remote video availability and attach each subscribed stream to its intended rendering element:
client.on(TRTC.EVENT.REMOTE_VIDEO_AVAILABLE, async ({
userId,
streamType
}) => {
const view = document.getElementById(`remote-${userId}`);
await client.startRemoteVideo({
userId,
streamType,
view
});
});In production, complete the implementation in this order:
- Generate room credentials on a trusted server.
- Request camera and microphone permission with clear UI context.
- Enter the Tencent RTC room before starting media publication.
- Capture and publish only the tracks the user has enabled.
- Subscribe to remote streams as participants publish them.
- Handle device changes, permission failures, and users leaving.
- Monitor network-quality events and expose clear reconnecting states.
- Stop local tracks and exit the room when the session ends.
Codec choice should follow the application’s device requirements rather than a theoretical efficiency ranking. H.264 is often the safest compatibility baseline, while newer codecs may reduce bandwidth on supported clients. Test real devices under constrained bandwidth, packet loss, and CPU load. For interactive video, stable end-to-end latency and graceful degradation usually matter more than maximizing compression in a laboratory test.
FAQ
What is the simplest video encoding definition?
Video encoding converts raw video into a compressed digital bitstream that is easier to store, transmit, and play. A decoder reconstructs frames from that bitstream for viewing.
Is MP4 a codec or a video format?
MP4 is a container format, not a codec. It packages encoded video, audio, captions, and metadata. The video inside an MP4 file may use H.264, H.265, or another supported codec.
Is lossy or lossless compression better for streaming?
Lossy compression is usually better for streaming because it reduces bandwidth far more effectively. Lossless compression preserves every source value but generally produces data rates too high for consumer live video and real-time calls.