Detects file type by reading magic number (hex signature) and validates against file extension
Before standardized magic numbers, operating systems relied exclusively on file extensions or filename patterns to determine file types. This approach proved unreliable as users could rename files arbitrarily, leading to misinterpretation and execution errors. Early versions of Unix began exploring the concept of inspecting file content to identify type, laying groundwork for what would become the "magic number" system.
The term "magic number" originated in the Unix operating system, where the file command was developed to identify file types by looking for specific byte patterns. In 1973, the first version of the file command appeared in Unix Research Version 4, using a simple text database of magic number patterns. This allowed the system to distinguish between binary executables, text files, shell scripts, and other formats regardless of their extension. [Wikipedia, file command]
In 1986, Ian Darwin wrote a more sophisticated version of the file command that used a configuration file (/etc/magic) to define byte offsets and expected values. This database approach allowed the system to be updated with new file formats without recompiling the tool. The magic database grew to include hundreds of file signatures, from common formats like JPEG and PDF to obscure proprietary formats. [Wikipedia, Magic Number]
The concept of magic numbers was later adopted by many operating systems and programming languages. The libmagic library, derived from the file command, became the standard for MIME type detection in web servers, file managers, and development tools. Today, magic number databases contain thousands of file signatures maintained by the open source community.
With the growth of the web and need for accurate MIME type detection, libraries such as file-type (JavaScript), python-magic (Python), and Apache Tika (Java) emerged. These libraries implement the same magic number principles but are optimized for modern applications, providing fast, memory efficient detection. The importance of magic numbers remains critical for security: web browsers use MIME sniffing to prevent malicious files from masquerading as safe content types. [Wikipedia, Content Sniffing]
A magic number is a sequence of bytes at a fixed position (usually the beginning) of a file that uniquely identifies the file format. Different file formats have distinct "signatures" stored in their headers. When a file verifier reads the first few bytes of a file and compares them against a database of known signatures, it can determine the actual file type regardless of the file extension or user assigned name. [Wikipedia, List of file signatures]
Magic numbers are often represented in hexadecimal (base-16) notation because it provides a compact way to view raw byte values. Each byte corresponds to two hex characters (0-9, A-F). For example, the magic number for PDF files is "%PDF" which in hex is "25 50 44 46". Some formats use little endian or big endian byte ordering, meaning the same value might appear reversed depending on architecture. Detection libraries handle both variants.
Many signatures include "wildcard" bytes (placeholders) where the value may vary. For example, the JPEG format uses "FF D8" as a start marker but the exact following bytes vary by JPEG variant. Advanced detection uses bitmasks to ignore variable portions while matching the fixed signature pattern.
Modern file type detection follows these steps:
The file-type library used in this tool implements the same algorithm used by thousands of web applications and is maintained by the open source community, supporting over 100 unique file formats including images, documents, archives, and audio/video containers. [GitHub, file-type]
Below is a reference list of widely used file signatures. The hex column shows the byte sequence typically found at the beginning of the file.
Note: This is not an exhaustive list. The file-type library used in this tool recognizes over 100 distinct formats. Many newer formats (AVIF, HEIC, etc.) also have well-defined magic numbers but may require deeper inspection.