2 Codesets and Codeset Conversion

The Compaq Tru64 UNIX operating system fully supports the following Korean codesets by including locales and codeset conversion support:

• DEC Korean
• Korean EUC (Extended UNIX Code)

It also provides codeset conversion support for the following codesets:

• KSC5601 (Unified Hangul)
• ISO-2022-KR
• UCS-2/UCS-4
• UTF-8

2.1 DEC Korean

The ASCII, KSC5636-1993 (KS Roman), and KSC5601-1992 character sets (excluding the additional Hangul characters defined an Annex 3 of the standard) are combined to form the DEC Korean codeset, which is denoted as deckorean.

DEC Korean uses a two-byte data representation for symbols and ideographic characters defined in KSC5601-1992. To differentiate KSC5601-1992 characters from ASCII, the most significant bit (MSB) of both bytes of KSC5601 characters is always set on.

Figure 2-1: Representations of DEC Korean Characters

The first byte of a two-byte code determines its row number, while the second determines its column number. The following formula illustrates the code of a two-byte KSC5601 character in relation to its row and column numbers:

1st byte = A0 + row number

2nd byte = A0 + column number

For example, if a character is at the first column of the 36th row, its encoded value is calculated as follows:

In this case, the character code is C4A1.

Figure 2-2 illustrates the division of a two-byte code space and the position of KSC5601-1992 characters.

Figure 2-2: Code Space for KSC5601-1992

2.2 Korean EUC

Extended UNIX Code (EUC) is an encoding methodology that allows concurrent use of up to four code sets in a data stream. Korean EUC uses that method to combine ASCII and KSC5601. Korean EUC is currently identical to DEC Korean, and is denoted as eucKR.

2.3 KSC5601 (Unified Hangul)

Microsoft has developed Unified Hangul Code (UHC) also known as "Extended Wansung" for its Windows 95 operating system. It is an optional character set of Win95K. Microsoft calls this Code Page 949.

UHC provides full compatibility with KSC5601-1992 EUC encoding, but adds additional encoding ranges to hold additional precombined Hangul characters (more precisely, the 8,822 that are needed to fully support the Johab character set). The following table provides the encoding ranges for UHC encoding:

Two-Byte Standard Characters	Encoding Ranges
First byte range	0x81-0xFE
Second byte ranges	0x41-0x5A, 0x61-0x7A and 0x81-0xFE

One-Byte Characters	Encoding Range
ASCII	0x21-0x7E

Note that the encoding ranges 0xA1A1 through 0xFEFE are identical in terms of character-to-code allocation with KSC5601-1992 in EUC Encoding.

2.4 ISO-2022-KR

The ISO-2022-KR codeset consists of the following character sets:

• ASCII
• KSC5601-1992

It is assumed that the starting code of the text is ASCII. ASCII and Korean characters are distinguished by use of the shift function. For example, the code SO indicates that the upcoming bytes are Korean characters as defined in KSC5601. To return to ASCII the SI code is used.

Therefore, the escape sequence, shift function and character set used in a text are as follows:

Currently, the ISO-2022-KR codeset can be used in codeset conversion.

2.5 UCS-2/UCS-4

UCS is a standard character encoding for the universal character set specified in the Unicode and ISO/IEC 10646 standards. UCS has two forms; UCS-2 (16-bit, or 2 octet units) and UCS-4 (32-bit, or 4 octet units). Unicode uses the UCS-2 form, which is commonly used on perconal computers. ISO/IEC allows either UCS-2 or UCS-4 encoding. UCS-4 encoding is in use on systems that can support the larger data unit size.

The current version of the Compaq Tru64 UNIX operating system supports both UCS-2 and UCS-4 encoding. UCS-4 is available in some Korean locales, and can be used in codeset conversion. For information about codeset conversion, see Section 2.7. For information about locales, see Chapter 3, Locales.

2.6 UTF-8

Unicode and ISO/IEC 10646 standards define transformation formats for the universal character set. For the most part, the following UCS transformation formats (UTFs) exist to transform UCS values into sequences of bytes to be handled by various byte-oriented protocols:

• UTF-8, the standard method for transforming UCS-4 or UCS-2 data into a sequence of 8-bit bytes and ensuring interchange transparency for characters from the ASCII character set (code positions 0 through 127).
• UTF-7, the standard interchange format for environments that strip the eighth bit from each byte.
• UTF-16, a transformation format that allows systems that can process only 16-bit units (specified by UCS-2 encoding) to support the extended character definition space that is included in UCS-4. In Compaq Tru64 UNIX, the UCS-2 encoding is the same as UTF-16.

The current version of the Compaq Tru64 UNIX operating system supports UTF-8 and UTF-16 but not UTF-7. UTF-8 can be used in codeset conversion and in the UTF-8 locales. For information about codeset conversion, see Section 2.7. For information about locale variants, see Chapter 3, Locales.

2.7 Codeset Conversion

The iconv utility provided by Compaq Tru64 UNIX converts the encoding of characters in one codeset to another and writes the results to standard output. Korean codeset converters provided are shown in Table 2-1.

Table 2-1: Codeset Conversion

For example, you can enter the following command to convert a DEC Korean file to a Korean UTF-8 file:

% iconv -f deckorean -t UTF-8 <file>

Table 2-2 shows the codesets and the strings you use as parameters to the iconv utility.

Table 2-2: Codeset Names

2.8 Codeset for Peripheral Devices

The Compaq Tru64 UNIX operating system provides a mechanism by which you configure your system to run applications with peripherals, such as terminals and printers, supporting different codesets. You can specify the codesets for the applications, terminals, and printers independently as shown in Table 2-3.

Table 2-3: Feasible Korean Codeset for Applications, Terminals, and Printers

For details about setting up terminal code and printer code, see Writing Software for the International Market.