Large file support

Encyclopedia : L : LA : LAR : Large file support

Large file support, often abberviated to LFS, is the term frequently applied to the support for files larger than 2 GB on 32-bit operating systems.

Traditionally, many operating systems and their underlying file system implementations used 32-bit integers to represent file sizes and positions. Consequently no file could be larger than 2³² bytes (4 GB). The problem was exacerbated by treating the sizes as signed numbers, which further lowered the limit to 2³¹ bytes (2 GB). Files larger than 2 GB, too large for 32-bit operating systems to handle, came to be known as large files.

While the 2 GB limit was quite acceptable at a time when disks were smaller, the general increase in storage capacity combined with more intense server and desktop file usage, especially for database and multimedia files, lead to intense pressure for OS vendors to remove the limitation.

Eventually vendors responded by switching to 64-bit numbers to represent file sizes, thus introducing large file support. (Merely ensuring the sizes were treated as unsigned numbers would not have been sufficient, except as a stop-gap measure, because it would only up the limit from 2 GB to 4 GB, which would not make a difference to most users.) It is worth noting that 64-bit operating systems, such as Tru64 UNIX, never had the 32-bit limit to begin with, and hence needed no additional "large file support".

This switch caused deployment issues and required design choices the consequences of which can still be seen:

The change to 64-bit file sizes frequently required incompatible changes to file system layout, which means that large file support sometimes necessitated a file system change. For example, Windows's FAT32 file system does not support files larger than 4 GB; one has to use NTFS instead.
To support binary compatibility with old applications, operating system interfaces had to retain their use of 32-bit file sizes and new interfaces had to be designed specifically for large file support.
To support writing portable code that makes use of LFS where possible, C standard library authors devised mechanisms that, depending on preprocessor constants, transparently redefined the functions to the 64-bit large file aware ones.
Many old interfaces, especially C-based ones, explicitly specified argument types in a way that did not allow straightforward nor transparent transition to 64-bit types. For example, the C functions fseek and ftell operate on file positions of type long int, which is typically 32 bits wide on 32-bit architecture, and cannot be made larger without sacrificing backward compatibility. (This was resolved by introducing new functions fseeko and ftello.)
The above efforts notwithstanding, all applications had to be recompiled to make them LFS-aware. The resulting binaries were typically not runnable on older releases of the same operating system. This was, and to some extent still remains, a problem for some application vendors.

Because of these transition issues many present-day applications still do not support large files.

External links

White Paper "[Adding Large File Support to the Single UNIX® Specification]"
[Large File Support in Linux]

From Wikipedia, the Free Encyclopedia. Original article here. Support Wikipedia by contributing or donating.
All text is available under the terms of the GNU Free Documentation License See Wikipedia Copyrights for details.