1. Introduction

   ISO/IEC 10646 [ISO.10646] defines a large character set called the
   Universal Character Set (UCS), which encompasses most of the world's
   writing systems.  The same set of characters is defined by the
   Unicode standard [UNICODE], which further defines additional
   character properties and other application details of great interest
   to implementers.  Up to the present time, changes in Unicode and
   amendments and additions to ISO/IEC 10646 have tracked each other, so
   that the character repertoires and code point assignments have
   remained in sync.  The relevant standardization committees have
   committed to maintain this very useful synchronism.

   ISO/IEC 10646 and Unicode define several encoding forms of their
   common repertoire: UTF-8, UCS-2, UTF-16, UCS-4 and UTF-32.  In an
   encoding form, each character is represented as one or more encoding
   units.  All standard UCS encoding forms except UTF-8 have an encoding
   unit larger than one octet, making them hard to use in many current
   applications and protocols that assume 8 or even 7 bit characters.

   UTF-8, the object of this memo, has a one-octet encoding unit.  It
   uses all bits of an octet, but has the quality of preserving the full
   US-ASCII [US-ASCII] range: US-ASCII characters are encoded in one
   octet having the normal US-ASCII value, and any octet with such a
   value can only stand for a US-ASCII character, and nothing else.

   UTF-8 encodes UCS characters as a varying number of octets, where the
   number of octets, and the value of each, depend on the integer value
   assigned to the character in ISO/IEC 10646 (the character number,
   a.k.a. code position, code point or Unicode scalar value).  This
   encoding form has the following characteristics (all values are in
   hexadecimal):

   o  Character numbers from U+0000 to U+007F (US-ASCII repertoire)
      correspond to octets 00 to 7F (7 bit US-ASCII values).  A direct
      consequence is that a plain ASCII string is also a valid UTF-8
      string.

   o  US-ASCII octet values do not appear otherwise in a UTF-8 encoded
      character stream.  This provides compatibility with file systems
      or other software (e.g., the printf() function in C libraries)
      that parse based on US-ASCII values but are transparent to other
      values.

   o  Round-trip conversion is easy between UTF-8 and other encoding
      forms.

   o  The first octet of a multi-octet sequence indicates the number of
      octets in the sequence.

   o  The octet values C0, C1, F5 to FF never appear.

   o  Character boundaries are easily found from anywhere in an octet
      stream.

   o  The byte-value lexicographic sorting order of UTF-8 strings is the
      same as if ordered by character numbers.  Of course this is of
      limited interest since a sort order based on character numbers is
      almost never culturally valid.

   o  The Boyer-Moore fast search algorithm can be used with UTF-8 data.

   o  UTF-8 strings can be fairly reliably recognized as such by a
      simple algorithm, i.e., the probability that a string of
      characters in any other encoding appears as valid UTF-8 is low,
      diminishing with increasing string length.

   UTF-8 was devised in September 1992 by Ken Thompson, guided by design
   criteria specified by Rob Pike, with the objective of defining a UCS
   transformation format usable in the Plan9 operating system in a non-
   disruptive manner.  Thompson's design was stewarded through
   standardization by the X/Open Joint Internationalization Group XOJIG
   (see [FSS_UTF]), bearing the names FSS-UTF (variant FSS/UTF), UTF-2
   and finally UTF-8 along the way.