Enhanced C#
Language of your choice: library documentation

Documentation moved to ecsharp.net

GitHub doesn't support HTTP redirects, so you'll be redirected in 3 seconds.

 All Classes Namespaces Functions Variables Enumerations Enumerator Properties Events Pages
Public fields | Public Member Functions | Static Public Member Functions | Protected Member Functions | Protected fields | List of all members
Loyc.Syntax.Les.LesLexer Class Reference

Lexer for EC# source code. More...


Source files:
Inheritance diagram for Loyc.Syntax.Les.LesLexer:
Loyc.Syntax.Lexing.ILexer< Token > Loyc.ICloneable< out T > Loyc.Syntax.IIndexToLine

Remarks

Lexer for EC# source code.

See also
ILexer{Token}, TokensToTree

Public fields

bool AllowNestedComments = true
 
bool SkipValueParsing = false
 Used for syntax highlighting, which doesn't care about token values. This option causes the Token.Value to be set to a default, like '\0' for single-quoted strings and 0 for numbers. Operator names are still parsed. More...
 

Public Member Functions

 LesLexer (UString text, IMessageSink errorSink)
 
 LesLexer (ICharSource text, string fileName, IMessageSink sink, int startPosition=0)
 
override void Reset (ICharSource source, string fileName="", int inputPosition=0, bool newSourceFile=true)
 
LesLexer Clone ()
 
override Maybe< TokenNextToken ()
 Scans the next token and returns information about it. More...
 
bool TDQStringLine ()
 
bool TSQStringLine ()
 
bool MLCommentLine (ref int nested)
 
- Public Member Functions inherited from Loyc.Syntax.IIndexToLine
SourcePos IndexToLine (int index)
 Returns the position in a source file of the specified index. More...
 

Static Public Member Functions

static string UnescapeQuotedString (ref UString sourceText, Action< int, string > onError, UString indentation=default(UString))
 Parses a normal or triple-quoted string that still includes the quotes. Supports quote types '\'', '"' and '`'. More...
 
static void UnescapeQuotedString (ref UString sourceText, Action< int, string > onError, StringBuilder sb, UString indentation=default(UString))
 Parses a normal or triple-quoted string that still includes the quotes (see documentation of the first overload) into a StringBuilder. More...
 
static bool UnescapeString (ref UString sourceText, char quoteType, bool isTripleQuoted, Action< int, string > onError, StringBuilder sb, UString indentation=default(UString))
 Parses a normal or triple-quoted string whose starting quotes have been stripped out. If triple-quote parsing was requested, stops parsing at three quote marks; otherwise, stops parsing at a single end-quote or newline. More...
 
static string ParseIdentifier (ref UString source, Action< int, string > onError, out bool checkForNamedLiteral)
 Parses an LES-style identifier such as foo, , <tt>foo or &ndash;punctuation–. More...
 
static object ParseNumberCore (UString source, bool isNegative, int numberBase, bool isFloat, Symbol typeSuffix, out string error)
 Parses the digits of a literal (integer or floating-point), not including the radix prefix (0x, 0b) or type suffix (F, D, L, etc.) More...
 

Protected Member Functions

override void Error (int lookaheadIndex, string message, params object[] args)
 
sealed override void AfterNewline ()
 
override bool SupportDotIndents ()
 

Protected fields

Dictionary< UString, Symbol_idCache = new Dictionary<UString,Symbol>()
 

Additional Inherited Members

- Properties inherited from Loyc.Syntax.Lexing.ILexer< Token >
ISourceFile SourceFile [get]
 The file being lexed. More...
 
IMessageSink ErrorSink [get, set]
 Event handler for errors. More...
 
int IndentLevel [get]
 Indentation level of the current line. This is updated after scanning the first whitespaces on a new line, and may be reset to zero when NextToken() returns a newline. More...
 
UString IndentString [get]
 Gets a string slice that holds the spaces or tabs that were used to indent the current line. More...
 
int LineNumber [get]
 Current line number (1 for the first line). More...
 
int InputPosition [get]
 Current input position (an index into SourceFile.Text). More...
 

Member Function Documentation

override Maybe<Token> Loyc.Syntax.Les.LesLexer.NextToken ( )
inline

Scans the next token and returns information about it.

Returns
The next token, or null at the end of the source file.

Implements Loyc.Syntax.Lexing.ILexer< Token >.

References Loyc.Syntax.Lexing.Operator, and Loyc.Syntax.Lexing.Spaces.

static string Loyc.Syntax.Les.LesLexer.ParseIdentifier ( ref UString  source,
Action< int, string >  onError,
out bool  checkForNamedLiteral 
)
inlinestatic

Parses an LES-style identifier such as foo, , <tt>foo or &ndash;punctuation–.

Parameters
sourceText to parse. On return, the range has been decreased by the length of the token; this method also stops if this range becomes empty.
onErrorA method to call on error
checkForNamedLiteralThis is set to true when the input starts with @ but doesn't use backquotes, which could indicate that it is an LES named literal such as or .
Returns
The parsed version of the identifier.
static object Loyc.Syntax.Les.LesLexer.ParseNumberCore ( UString  source,
bool  isNegative,
int  numberBase,
bool  isFloat,
Symbol  typeSuffix,
out string  error 
)
inlinestatic

Parses the digits of a literal (integer or floating-point), not including the radix prefix (0x, 0b) or type suffix (F, D, L, etc.)

Parameters
sourceDigits of the number (not including radix prefix or type suffix)
isFloatWhether the number is floating-point
numberBaseRadix. Must be 2 (binary), 10 (decimal) or 16 (hexadecimal).
typeSuffixType suffix: F, D, M, U, L, UL, or null.
errorSet to an error message in case of error.
Returns
Boxed value of the literal, null if total failure (result is not null in case of overflow), or CodeSymbols.Sub (-) if isNegative is true but the type suffix is unsigned or the number is larger than long.MaxValue.
static string Loyc.Syntax.Les.LesLexer.UnescapeQuotedString ( ref UString  sourceText,
Action< int, string >  onError,
UString  indentation = default(UString) 
)
inlinestatic

Parses a normal or triple-quoted string that still includes the quotes. Supports quote types '\'', '"' and '`'.

Parameters
sourceTextinput text
onErrorCalled in case of parsing error (unknown escape sequence or missing end quotes)
indentationInside a triple-quoted string, any text following a newline is ignored as long as it matches this string. For example, if the text following a newline is "\t\t Foo" and this string is "\t\t\t", the tabs are ignored and " Foo" is kept.
Returns
The decoded string

This method recognizes LES and EC#-style string syntax. Firstly, it recognizes triple-quoted strings (''' """ ```). These strings enjoy special newline handling: the newline is always interpreted as
regardless of the actual kind of newline ( and
newlines come out as
), and indentation following the newline can be stripped out. Triple-quoted strings can have escape sequences that use both kinds of slash, like so:
/ / \'/ "/ \0/
. However, there are no unicode escapes (/ is NOT supported).

Secondly, it recognizes normal strings (' " `). These strings stop parsing (with an error) at a newline, and can contain C-style escape sequences:
\' " \0
etc. C#-style verbatim strings are NOT supported.

static void Loyc.Syntax.Les.LesLexer.UnescapeQuotedString ( ref UString  sourceText,
Action< int, string >  onError,
StringBuilder  sb,
UString  indentation = default(UString) 
)
inlinestatic

Parses a normal or triple-quoted string that still includes the quotes (see documentation of the first overload) into a StringBuilder.

References Loyc.Localize.Localized().

static bool Loyc.Syntax.Les.LesLexer.UnescapeString ( ref UString  sourceText,
char  quoteType,
bool  isTripleQuoted,
Action< int, string >  onError,
StringBuilder  sb,
UString  indentation = default(UString) 
)
inlinestatic

Parses a normal or triple-quoted string whose starting quotes have been stripped out. If triple-quote parsing was requested, stops parsing at three quote marks; otherwise, stops parsing at a single end-quote or newline.

Returns
true if parsing stopped at one or three quote marks, or false if parsing stopped at the end of the input string or at a newline (in a string that is not triple-quoted).

This method recognizes LES and EC#-style string syntax.

References Loyc.Localize.Localized().

Member Data Documentation

bool Loyc.Syntax.Les.LesLexer.SkipValueParsing = false

Used for syntax highlighting, which doesn't care about token values. This option causes the Token.Value to be set to a default, like '\0' for single-quoted strings and 0 for numbers. Operator names are still parsed.