MIT Scheme Reference: Characters

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

# 5. Characters

Characters are objects that represent printed characters, such as letters and digits.(5)

> 5.1 External Representation of Characters |  |
> 5.2 Comparison of Characters |  |
> 5.3 Miscellaneous Character Operations |  |
> 5.4 Internal Representation of Characters |  |
> 5.5 ISO-8859-1 Characters |  |
> 5.6 Character Sets |  |
> 5.7 Unicode |  |

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

## 5.1 External Representation of Characters

Characters are written using the notation `#\character` or `#\character-name`. For example:

    #\a                     ; lowercase letter
    #\A                     ; uppercase letter
    #\(                     ; left parenthesis
    #\space                 ; the space character
    #\newline               ; the newline character

Case is significant in `#\character`, but not in `#\character-name`. If character in `#\character` is a letter, character must be followed by a delimiter character such as a space or parenthesis. Characters written in the `#\` notation are self-evaluating; you don't need to quote them.

A character name may include one or more bucky bit prefixes to indicate that the character includes one or more of the keyboard shift keys Control, Meta, Super, Hyper, or Top (note that the Control bucky bit prefix is not the same as the ASCII control key). The bucky bit prefixes and their meanings are as follows (case is not significant):

    Key             Bucky bit prefix        Bucky bit
    ---             ----------------        ---------

    Meta            M- or Meta-                 1
    Control         C- or Control-              2
    Super           S- or Super-                4
    Hyper           H- or Hyper-                8
    Top             T- or Top-                 16

For example,

    #\c-a                   ; Control-a
    #\meta-b                ; Meta-b
    #\c-s-m-h-a             ; Control-Meta-Super-Hyper-A

The following character-names are supported, shown here with their ASCII equivalents:

    Character Name          ASCII Name
    --------------          ----------

    altmode                 ESC
    backnext                US
    backspace               BS
    call                    SUB
    linefeed                LF
    page                    FF
    return                  CR
    rubout                  DEL
    space
    tab                     HT

In addition, `#\newline` is the same as `#\linefeed` (but this may change in the future, so you should not depend on it). All of the standard ASCII names for non-printing characters are supported:

    NUL     SOH     STX     ETX     EOT     ENQ     ACK     BEL
    BS      HT      LF      VT      FF      CR      SO      SI
    DLE     DC1     DC2     DC3     DC4     NAK     SYN     ETB
    CAN     EM      SUB     ESC     FS      GS      RS      US
    DEL

procedure: char->name char [slashify?]
  Returns a string corresponding to the printed representation of char. This is the character or character-name component of the external representation, combined with the appropriate bucky bit prefixes.

      (char->name #\a)                        =>  "a"
      (char->name #\space)                    =>  "Space"
      (char->name #\c-a)                      =>  "C-a"
      (char->name #\control-a)                =>  "C-a"

  Slashify?, if specified and true, says to insert the necessary backslash characters in the result so that `read` will parse it correctly. In other words, the following generates the external representation of char:

      (string-append "#\\" (char->name char #t))

  If slashify? is not specified, it defaults to `#f`.

procedure: name->char string
  Converts a string that names a character into the character specified. If string does not name any character, `name->char` signals an error.

      (name->char "a")                        =>  #\a
      (name->char "space")                    =>  #\Space
      (name->char "c-a")                      =>  #\C-a
      (name->char "control-a")                =>  #\C-a

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

## 5.2 Comparison of Characters

procedure: char=? char1 char2
procedure: char<? char1 char2
procedure: char>? char1 char2
procedure: char<=? char1 char2
procedure: char>=? char1 char2
procedure: char-ci=? char1 char2
procedure: char-ci<? char1 char2
procedure: char-ci>? char1 char2
procedure: char-ci<=? char1 char2
procedure: char-ci>=? char1 char2
  Returns `#t` if the specified characters are have the appropriate order relationship to one another; otherwise returns `#f`. The `-ci` procedures don't distinguish uppercase and lowercase letters.

  Character ordering follows these rules:

  - The digits are in order; for example, `(char<? #\0 #\9)` returns `#t`.

  - The uppercase characters are in order; for example, `(char<? #\A #\B)` returns `#t`.

  - The lowercase characters are in order; for example, `(char<? #\a #\b)` returns `#t`.

  In addition, MIT Scheme orders those characters that satisfy `char-standard?` the same way that ISO-8859-1 does.

  Characters are ordered by first comparing their bucky bits part and then their code part. In particular, characters without bucky bits come before characters with bucky bits.

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

## 5.3 Miscellaneous Character Operations

procedure: char? object
  Returns `#t` if object is a character; otherwise returns `#f`.

procedure: char-upcase char
procedure: char-downcase char
  Returns the uppercase or lowercase equivalent of char if char is a letter; otherwise returns char. These procedures return a character char2 such that `(char-ci=? char char2)`.

procedure: char->digit char [radix]
  If char is a character representing a digit in the given radix, returns the corresponding integer value. If you specify radix (which must be an exact integer between 2 and 36 inclusive), the conversion is done in that base, otherwise it is done in base 10. If char doesn't represent a digit in base radix, `char->digit` returns `#f`.

  Note that this procedure is insensitive to the alphabetic case of char.

      (char->digit #\8)                       =>  8
      (char->digit #\e 16)                    =>  14
      (char->digit #\e)                       =>  #f

procedure: digit->char digit [radix]
  Returns a character that represents digit in the radix given by radix. Radix must be an exact integer between 2 and 36 (inclusive), and defaults to 10. Digit, which must be an exact non-negative integer, should be less than radix; if digit is greater than or equal to radix, `digit->char` returns `#f`.

      (digit->char 8)                         =>  #\8
      (digit->char 14 16)                     =>  #\E

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

## 5.4 Internal Representation of Characters

An MIT Scheme character consists of a code part and a bucky bits part. The MIT Scheme set of characters can represent more characters than ASCII can; it includes characters with Super, Hyper, and Top bucky bits, as well as Control and Meta. Every ASCII character corresponds to some MIT Scheme character, but not vice versa.(6)

MIT Scheme uses a 16-bit character code with 5 bucky bits. Normally, Scheme uses the least significant 8 bits of the character code to contain the ISO-8859-1 representation for the character. The representation is expanded in order to allow for the use of UTF-16 in the future.

procedure: make-char code bucky-bits
  Builds a character from code and bucky-bits. Both code and bucky-bits must be exact non-negative integers in the appropriate range. Use `char-code` and `char-bits` to extract the code and bucky bits from the character. If `0` is specified for bucky-bits, `make-char` produces an ordinary character; otherwise, the appropriate bits are turned on as follows:

      1               Meta
      2               Control
      4               Super
      8               Hyper
      16              Top

  For example,

      (make-char 97 0)                        =>  #\a
      (make-char 97 1)                        =>  #\M-a
      (make-char 97 2)                        =>  #\C-a
      (make-char 97 3)                        =>  #\C-M-a

procedure: char-bits char
  Returns the exact integer representation of char's bucky bits. For example,

      (char-bits #\a)                         =>  0
      (char-bits #\m-a)                       =>  1
      (char-bits #\c-a)                       =>  2
      (char-bits #\c-m-a)                     =>  3

procedure: char-code char
  Returns the character code of char, an exact integer. For example,

      (char-code #\a)                         =>  97
      (char-code #\c-a)                       =>  97

variable: char-code-limit
variable: char-bits-limit
  These variables define the (exclusive) upper limits for the character code and bucky bits (respectively). The character code and bucky bits are always exact non-negative integers, and are strictly less than the value of their respective limit variable.

procedure: char->integer char
procedure: integer->char k
  `char->integer` returns the character code representation for char. `integer->char` returns the character whose character code representation is k.

  In MIT Scheme, if `(char-ascii? char)` is true, then

      (eqv? (char->ascii char) (char->integer char))

  However, this behavior is not required by the Scheme standard, and code that depends on it is not portable to other implementations.

  These procedures implement order isomorphisms between the set of characters under the `char<=?` ordering and some subset of the integers under the `<=` ordering. That is, if

      (char<=? a b)  =>  #t    and    (<= x y)  =>  #t

  and `x` and `y` are in the range of `char->integer`, then

      (<= (char->integer a)
          (char->integer b))                  =>  #t
      (char<=? (integer->char x)
               (integer->char y))             =>  #t

  Note: If the argument to `char->integer` or `integer->char` is a constant, the compiler will constant-fold the call, replacing it with the corresponding result. This is a very useful way to denote unusual character constants or ASCII codes.

variable: char-integer-limit
  The range of `char->integer` is defined to be the exact non-negative integers that are less than the value of this variable (exclusive).

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

## 5.5 ISO-8859-1 Characters

MIT Scheme internally uses ISO-8859-1 codes for I/O, and stores character objects in a fashion that makes it convenient to convert between ISO-8859-1 codes and characters. Also, character strings are implemented as byte vectors whose elements are ISO-8859-1 codes; these codes are converted to character objects when accessed. For these reasons it is sometimes desirable to be able to convert between ISO-8859-1 codes and characters.

Not all characters can be represented as ISO-8859-1 codes. A character that has an equivalent ISO-8859-1 representation is called an ISO-8859-1 character.

For historical reasons, the procedures that manipulate ISO-8859-1 characters use the word "ASCII" rather than "ISO-8859-1".

procedure: char-ascii? char
  Returns the ISO-8859-1 code for char if char has an ISO-8859-1 representation; otherwise returns `#f`.

  In the current implementation, the characters that satisfy this predicate are those in which the bucky bits are turned off, and for which the character code is less than 256.

procedure: char->ascii char
  Returns the ISO-8859-1 code for char. An error `condition-type:bad-range-argument` is signalled if char doesn't have an ISO-8859-1 representation.

procedure: ascii->char code
  Code must be the exact integer representation of an ISO-8859-1 code. This procedure returns the character corresponding to code.

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

## 5.6 Character Sets

MIT Scheme's character-set abstraction is used to represent groups of characters, such as the letters or digits. Character sets may contain only ISO-8859-1 characters; in the future this may be changed to allow the full range of characters.

There is no meaningful external representation for character sets; use `char-set-members` to examine their contents. There is (at present) no specific equivalence predicate for character sets; use `equal?` for this purpose.

procedure: char-set? object
  Returns `#t` if object is a character set; otherwise returns `#f`.

variable: char-set:upper-case
variable: char-set:lower-case
variable: char-set:alphabetic
variable: char-set:numeric
variable: char-set:alphanumeric
variable: char-set:whitespace
variable: char-set:not-whitespace
variable: char-set:graphic
variable: char-set:not-graphic
variable: char-set:standard
  These variables contain predefined character sets. To see the contents of one of these sets, use `char-set-members`.

  Alphabetic characters are the 52 upper and lower case letters. Numeric characters are the 10 decimal digits. Alphanumeric characters are those in the union of these two sets. Whitespace characters are `#\space`, `#\tab`, `#\page`, `#\linefeed`, and `#\return`. Graphic characters are the printing characters and `#\space`. Standard characters are the printing characters, `#\space`, and `#\newline`. These are the printing characters:

      ! " # $ % & ' ( ) * + , - . /
      0 1 2 3 4 5 6 7 8 9
      : ; < = > ? @
      A B C D E F G H I J K L M N O P Q R S T U V W X Y Z
      [ \ ] ^ _ `
      a b c d e f g h i j k l m n o p q r s t u v w x y z
      { | } ~

procedure: char-upper-case? char
procedure: char-lower-case? char
procedure: char-alphabetic? char
procedure: char-numeric? char
procedure: char-alphanumeric? char
procedure: char-whitespace? char
procedure: char-graphic? char
procedure: char-standard? object
  These predicates are defined in terms of the respective character sets defined above.

procedure: char-set-members char-set
  Returns a newly allocated list of the characters in char-set.

procedure: char-set-member? char-set char
  Returns `#t` if char is in char-set; otherwise returns `#f`.

procedure: char-set char ...
  Returns a character set consisting of the specified ISO-8859-1 characters. With no arguments, `char-set` returns an empty character set.

procedure: chars->char-set chars
  Returns a character set consisting of chars, which must be a list of ISO-8859-1 characters. This is equivalent to `(apply char-set chars)`.

procedure: string->char-set string
  Returns a character set consisting of all the characters that occur in string.

procedure: ascii-range->char-set lower upper
  Lower and upper must be exact non-negative integers representing ISO-8859-1 character codes, and lower must be less than or equal to upper. This procedure creates and returns a new character set consisting of the characters whose ISO-8859-1 codes are between lower (inclusive) and upper (exclusive).

  For historical reasons, the name of this procedure refers to "ASCII" rather than "ISO-8859-1".

procedure: predicate->char-set predicate
  Predicate must be a procedure of one argument. `predicate->char-set` creates and returns a character set consisting of the ISO-8859-1 characters for which predicate is true.

procedure: char-set-difference char-set1 char-set2
  Returns a character set consisting of the characters that are in char-set1 but aren't in char-set2.

procedure: char-set-intersection char-set ...
  Returns a character set consisting of the characters that are in all of the char-sets.

procedure: char-set-union char-set ...
  Returns a character set consisting of the characters that are in at least one o the char-sets.

procedure: char-set-invert char-set
  Returns a character set consisting of the ISO-8859-1 characters that are not in char-set.

[ < ] | [ > ] |  | [ << ] | [ Up ] | [ >> ] |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

## 5.7 Unicode

MIT Scheme provides rudimentary support for Unicode characters. In an ideal world, Unicode would be the base character set for MIT Scheme, but this implementation predates the invention of Unicode. And converting an application of this size is a considerable undertaking. So for the time being, the base character set is ISO-8859-1 and Unicode support is grafted on.

This Unicode support was implemented as a part of the XML parser (see section 14.12 XML Parser) implementation. XML uses Unicode as its base character set, and any XML implementation must support Unicode.

The Unicode implementation consists of two parts: I/O procedures that read and write UTF-8 characters, and an alphabet abstraction, which is an efficient implementation of sets of Unicode code points (similar to the `char-set` abstraction).

The basic unit in a Unicode implementation is the code point.

procedure: unicode-code-point? object
  Returns `#t` if object is a Unicode code point. Code points are implemented as exact non-negative integers. Code points are further limited, by the Unicode standard, to be strictly less than `#x80000000`.

The next few procedures do I/O on code points.

procedure: read-utf8-code-point port
  Reads and returns a UTF-8-encoded code point from port. Returns an end-of-file object if there are no more characters available from port. Signals an error if the input stream isn't a valid UTF-8 encoding.

procedure: write-utf8-code-point code-point port
  Writes code-point to port in the UTF-8 encoding.

procedure: utf8-string->code-point string
  Reads and returns a UTF-8-encoded code point from string. Equivalent to

      (read-utf8-code-point (string->input-port string))

procedure: code-point->utf8-string code-point
  Returns a newly-allocated string containing the UTF-8 encoding of code-point. Equivalent to

      (with-string-output-port
       (lambda (port)
         (write-utf8-code-point code-point port)))

Applications often need to manipulate sets of characters, such as the set of alphabetic characters or the set of whitespace characters. The alphabet abstraction provides an efficient implementation of sets of Unicode code points.

procedure: alphabet? object
  Returns `#t` if object is a Unicode alphabet, otherwise returns `#f`.

procedure: code-points->alphabet items
  Returns a Unicode alphabet containing the code points described by items. Items must satisfy `well-formed-code-points-list?`.

procedure: alphabet->code-points alphabet
  Returns a well-formed code-points list that describes the code points represented by alphabet.

procedure: well-formed-code-points-list? object
  Returns `#t` if object is a well-formed code-points list, otherwise returns `#f`. A well-formed code-points list is a proper list, each element of which is either a code point or a pair of code points. A pair of code points represents a contiguous range of code points. The CAR of the pair is the lower limit, and the CDR is the upper limit. Both limits are inclusive, and the lower limit must be strictly less than the upper limit.

procedure: code-point-in-alphabet? code-point alphabet
  Returns `#t` if code-point is a member of alphabet, otherwise returns `#f`.

procedure: char-in-alphabet? char alphabet
  Returns `#t` if char is a member of alphabet, otherwise returns `#f`. Equivalent to

      (code-point-in-alphabet? (char-code char) alphabet)

Character sets and alphabets can be converted to one another, provided that the alphabet contains only 8-bit code points. This is true because 8-bit code points in Unicode map directly to ISO-8859-1 characters, which is what character sets contain.

procedure: char-set->alphabet char-set
  Returns a Unicode alphabet containing the code points that correspond to characters that are members of char-set.

procedure: alphabet->char-set alphabet
  Returns a character set containing the characters that correspond to 8-bit code points that are members of alphabet. (Code points outside the 8-bit range are ignored.)

procedure: string->alphabet string
  Returns a Unicode alphabet containing the code points corresponding to the characters in string. Equivalent to

      (char-set->alphabet (string->char-set string))

procedure: alphabet->string alphabet
  Returns a newly-allocated string containing the characters corresponding to the 8-bit code points in alphabet. (Code points outside the 8-bit range are ignored.)

procedure: 8-bit-alphabet? alphabet
  Returns `#t` if alphabet contains only 8-bit code points, otherwise returns `#f`.

procedure: alphabet+ alphabet ...
  Returns a Unicode alphabet that contains each code point that is a member of any of the alphabet arguments.

procedure: alphabet- alphabet1 alphabet2
  Returns a Unicode alphabet that contains each code point that is a member of alphabet1 and is not a member of alphabet2.

[ << ] | [ >> ] |  |  |  |  |  | [Top] | [Contents] | [Index] | [ ? ]

This document was generated by Chris Hanson on June, 17 2002 using texi2html