GB 18030
Encyclopedia : G : GB : GB1 : GB 18030
| Unicode |
|---|
| Encodings |
| UCS |
| Mapping |
| Bi-directional text |
| BOM |
| Han unification |
| Unicode and HTML |
| Unicode and e-mail |
| Unicode typefaces |
GB18030 can be considered as a Unicode Transformation Format (i.e. an encoding of all Unicode code points) that maintains compatibility with a legacy character set. In other words, it is a Chinese equivalent of UTF-8 (which maintains compatibility with ASCII). Like UTF-8, GB18030 is a superset of ASCII and can represent the whole range of Unicode code points. Due to its Unicode equivalence, GB18030 supports both simplified and traditional Chinese characters.
GB18030 also maintains compatibility with GBK (with the exception of the euro sign which is given a single byte code of 0x80 in Microsoft's later versions of GBK and a two byte code of A2 E3 in GB18030), which was the pre-existing standard character encoding used in the PRC, with the aim of simplifying the upgrade of data and software to use GB18030. Part of the mapping data is from a lookup table (similarly to GBK). The rest is calculated algorithmically. Unfortunately it also inherits the bad aspects of the legacy standards its based on (most notably needing special code to safely find ASCII characters in a GB18030 sequence).
Most major computer companies had already standardised on some version of Unicode as the primary format for use in their binary formats and OS calls. However, they mostly had only supported code points in the BMP originally defined in Unicode 1.0, which supported only 65,536 codepoints and was often encoded in 16 bits as UCS-2.
In a move of historic significance for software supporting Unicode, the PRC decided to mandate support of certain code points outside the BMP. This means that software can no longer get away with treating characters as 16 bit fixed width entities (UCS-2). Therefore they must either process the data in a variable width format (such as UTF-8 or UTF-16), which are the most common choices, or move to a larger fixed width format (such as UCS-4 or UTF-32). Microsoft made the change from UCS-2 to UTF-16 with Windows 2000.
The SimSun 18030 font enables the display of the GB 18030 characters, which includes all the characters in Unicode 2.1 plus new characters found in the Unicode CJK Unified Ideographs Extension A section.
Technical details
The four byte scheme can be thought of as consisting of two units, each of two bytes. Each unit has a similar format to a GBK two byte character but with a range of values for the second byte of 0x30-0x39 (the ASCII codes for decimal digits). The first byte has the range 0x81 to 0xFE, as before. This means that a string search routine that is safe for GBK should also be reasonablly safe for GB18030 (in much the same way that a basic byte-oriented search routine is reasonably safe for EUC).This gives a total of 1,587,600 (126*10*126*10) possible 4 byte sequences, which is easily sufficient to cover Unicode's 1,112,046 (17 × 65536 - 2048 surrogates - 18 noncharacters) code points.
Unfortunately, to further complicate matters there are no simple rules to translate between a 4 byte sequence and its corresponding code point. Instead, codes are allocated sequentially (with the first byte containing the most significant part and the last the least significant part) only to Unicode code points that are not mapped in any other manner. For example:
U+00DE (Þ) → 81 30 89 37 U+00DF (ß) → 81 30 89 38 U+00E0 (à) → A8 A4 U+00E1 (á) → A8 A2 U+00E2 (â) → 81 30 89 39 U+00E3 (ã) → 81 30 8A 30
See also
External links
- [IANA Charset Registration for GB18030]
- [English language summary of GB 18030-2000]
- [Introduction to GB18030 including evolution from GB2312 and GBK]
- [Authoritative mapping table between GB18030 and Unicode] (Large xml file, does not make as much use of range tags as it could. Seems to hang firefox!).
- [ICU Converter Explorer: GB18030]
- [Unicode CJK Unified Ideographs Extension A] (PDF, 1.5MB)
- [Unicode CJK Unified Ideographs Extension B] (PDF, 13 MB)
- [GB18030 Support Package for Windows 2000/XP, including Chinese, Tibetan, Yi, Mongolian and Thai font by Microsoft]
- [SIL's freeware fonts, editors and documentation]
From Wikipedia, the Free Encyclopedia. Original article here. Support Wikipedia by contributing or donating.
All text is available under the terms of the GNU Free Documentation License See Wikipedia Copyrights for details.