2.2 Lexical Elements, Separators, and Delimiters

{text of a program} The text of a program consists of the texts of one or more compilations. {lexical element} {token: See lexical element} The text of each compilation is a sequence of separate lexical elements. Each lexical element is formed from a sequence of characters, and is either a delimiter, an identifier, a reserved word, a numeric_literal, a character_literal, a string_literal, or a comment. The meaning of a program depends only on the particular sequences of lexical elements that form its compilations, excluding comments.

{AI95-00285-01} The text of a compilation is divided into {line} lines. {end of a line} In general, the representation for an end of line is implementation defined. However, a sequence of one or more format_effectors other than the character whose code position is 16#09# (CHARACTER TABULATION) character tabulation (HT) signifies at least one end of line. 

{AI95-00285-01} {separator} [In some cases an explicit separator is required to separate adjacent lexical elements.] A separator is any of a separator_space space character, a format_effector format effector, or the end of a line, as follows: 

One or more separators are allowed between any two adjacent lexical elements, before the first of each compilation, or after the last. At least one separator is required between an identifier, a reserved word, or a numeric_literal and an adjacent identifier, reserved word, or numeric_literal.

{AI95-00285-01} {delimiter} A delimiter is either one of the following special characters: 

{compound delimiter} or one of the following compound delimiters each composed of two adjacent special characters 

Each of the special characters listed for single character delimiters is a single delimiter except if this character is used as a character of a compound delimiter, or as a character of a comment, string_literal, character_literal, or numeric_literal.

The following names are used when referring to compound delimiters:

An implementation shall support lines of at least 200 characters in length, not counting any characters used to signify the end of a line. An implementation shall support lexical elements of at least 200 characters in length. The maximum supported line length and lexical element length are implementation defined.