Main Page   Namespace List   Class Hierarchy   Compound List   File List   Compound Members   File Members  

Lex Class Reference

#include <token.h>

Collaboration diagram for Lex:

Collaboration graph
[legend]
List of all members.

Public Methods

 Lex (Program *)
int GetToken (Token &)
int LookAhead (int)
int LookAhead (int, Token &)
char * Save ()
void Restore (char *)
void GetOnlyClosingBracket (Token &)
Ptree * GetComments ()
Ptree * GetComments2 ()
uint LineNumber (char *, char *&, int &)

Static Public Methods

bool RecordKeyword (char *, int)
bool Reify (Ptree *, unsigned int &)
bool Reify (Ptree *t, char *&)

Private Methods

uint Tokenp ()
int TokenLen ()
char * TokenPosition ()
char Ref (uint i)
void Rewind (char *)
int ReadToken (char *&, int &)
void SkipAttributeToken ()
int SkipExtensionToken (char *&, int &)
char GetNextNonWhiteChar ()
int ReadLine ()
bool ReadCharConst (uint top)
bool ReadStrConst (uint top)
int ReadNumber (char c, uint top)
int ReadFloat (uint top)
bool ReadLineDirective ()
int ReadIdentifier (uint top)
int Screening (char *identifier, int len)
int ReadSeparator (char c, uint top)
int SingleCharOp (unsigned char c)
int ReadComment (char c, uint top)

Private Attributes

Program * file
TokenFifo fifo
uint tokenp
int token_len
int last_token

Static Private Attributes

HashTable * user_keywords = 0
Ptree * comments = 0

Friends

class TokenFifo

Constructor & Destructor Documentation

Lex::Lex (  Program *    ) 
 

Definition at line 73 of file token.cc.

References file, InitializeOtherKeywords(), last_token, Program::Rewind(), token_len, and tokenp.

Member Function Documentation

Ptree * Lex::GetComments (    ) 
 

Definition at line 1029 of file token.cc.

References comments.

Referenced by Parser::rClassBody(), Parser::rClassSpec(), Parser::rCompoundStatement(), and Parser::rDefinition().

Ptree * Lex::GetComments2 (    ) 
 

Definition at line 1035 of file token.cc.

References comments.

Referenced by Parser::rClassMember(), and Parser::rDefinition().

char Lex::GetNextNonWhiteChar (    )  [private]
 

Definition at line 510 of file token.cc.

References file, Program::Get(), is_blank, and Program::Unget().

Referenced by ReadLine().

void Lex::GetOnlyClosingBracket (  Token &    ) 
 

Definition at line 104 of file token.cc.

References Token::ptr, and Restore().

Referenced by Parser::rTemplateArgs().

int Lex::GetToken (  Token &    ) 
 

Definition at line 114 of file token.cc.

References fifo, Token::kind, Token::len, Lex::TokenFifo::Pop(), and Token::ptr.

Referenced by Parser::optCvQualify(), Parser::optIntegralTypeOrClassSpec(), Parser::optMemberSpec(), Parser::optPtrOperator(), Parser::optStorageSpec(), Parser::optThrowDecl(), Parser::rAccessDecl(), Parser::rAdditiveExpr(), Parser::rAllocateExpr(), Parser::rAllocateInitializer(), Parser::rAllocateType(), Parser::rAndExpr(), Parser::rArgDeclaration(), Parser::rArgDeclList(), Parser::rBaseSpecifiers(), Parser::rCastExpr(), Parser::rClassBody(), Parser::rClassMember(), Parser::rClassSpec(), Parser::rCommaExpression(), Parser::rCompoundStatement(), Parser::rConditionalExpr(), Parser::rConstDeclaration(), Parser::rConstructorDecl(), Parser::rDeclarator2(), Parser::rDeclarators(), Parser::rDeclaratorWithInit(), Parser::rDoStatement(), Parser::rEnumBody(), Parser::rEnumSpec(), Parser::rEqualityExpr(), Parser::rExclusiveOrExpr(), Parser::rExpression(), Parser::rExprStatement(), Parser::rExternTemplateDecl(), Parser::rForStatement(), Parser::rFunctionArguments(), Parser::rIfStatement(), Parser::rInclusiveOrExpr(), Parser::rInitializeExpr(), Parser::rIntegralDeclaration(), Parser::rIntegralDeclStatement(), Parser::rLinkageBody(), Parser::rLinkageSpec(), Parser::rLogicalAndExpr(), Parser::rLogicalOrExpr(), Parser::rMemberInit(), Parser::rMemberInitializers(), Parser::rMetaArguments(), Parser::rMetaclassDecl(), Parser::rMultiplyExpr(), Parser::rName(), Parser::rNamespaceSpec(), Parser::rNewDeclarator(), Parser::rNullDeclaration(), Parser::rOperatorName(), Parser::rOtherDeclaration(), Parser::rOtherDeclStatement(), Parser::rPmExpr(), Parser::rPostfixExpr(), Parser::rPrimaryExpr(), Parser::rProgram(), Parser::rPtrToMember(), Parser::rRelationalExpr(), Parser::rShiftExpr(), Parser::rSizeofExpr(), Parser::rStatement(), Parser::rSwitchStatement(), Parser::rTempArgDeclaration(), Parser::rTempArgList(), Parser::rTemplateArgs(), Parser::rTemplateDecl2(), Parser::rThrowExpr(), Parser::rTryStatement(), Parser::rTypedef(), Parser::rUnaryExpr(), Parser::rUserAccessSpec(), Parser::rUserdefKeyword(), Parser::rUserdefStatement(), Parser::rUsing(), Parser::rVarNameCore(), Parser::rWhileStatement(), and Parser::SkipTo().

uint Lex::LineNumber (  char *   ,
char *&   ,
int &   
) 
 

Definition at line 109 of file token.cc.

References file, Program::LineNumber(), and uint.

Referenced by Parser::LineNumber().

int Lex::LookAhead (  int   ,
Token &   
) 
 

Definition at line 125 of file token.cc.

References fifo, Token::kind, Token::len, Lex::TokenFifo::Peek(), and Token::ptr.

int Lex::LookAhead (  int    ) 
 

Definition at line 120 of file token.cc.

References fifo, and Lex::TokenFifo::Peek().

Referenced by Parser::isAllocateExpr(), Parser::isConstructorDecl(), Parser::isPtrToMember(), Parser::isTemplateArgs(), Parser::isTypeSpecifier(), Parser::moreVarName(), Parser::optCvQualify(), Parser::optIntegralTypeOrClassSpec(), Parser::optMemberSpec(), Parser::optPtrOperator(), Parser::optStorageSpec(), Parser::optThrowDecl(), Parser::rAdditiveExpr(), Parser::rAllocateExpr(), Parser::rAllocateInitializer(), Parser::rAllocateType(), Parser::rAndExpr(), Parser::rArgDeclaration(), Parser::rArgDeclList(), Parser::rArgDeclListOrInit(), Parser::rBaseSpecifiers(), Parser::rCastExpr(), Parser::rClassBody(), Parser::rClassMember(), Parser::rClassSpec(), Parser::rCommaExpression(), Parser::rCompoundStatement(), Parser::rConditionalExpr(), Parser::rConstDeclaration(), Parser::rConstructorDecl(), Parser::rDeclaration(), Parser::rDeclarationStatement(), Parser::rDeclarator2(), Parser::rDeclarators(), Parser::rDeclaratorWithInit(), Parser::rDefinition(), Parser::rEnumBody(), Parser::rEnumSpec(), Parser::rEqualityExpr(), Parser::rExclusiveOrExpr(), Parser::rExpression(), Parser::rExprStatement(), Parser::rForStatement(), Parser::rFunctionArguments(), Parser::rIfStatement(), Parser::rInclusiveOrExpr(), Parser::rInitializeExpr(), Parser::rIntegralDeclaration(), Parser::rIntegralDeclStatement(), Parser::rLinkageBody(), Parser::rLinkageSpec(), Parser::rLogicalAndExpr(), Parser::rLogicalOrExpr(), Parser::rMemberInitializers(), Parser::rMetaArguments(), Parser::rMultiplyExpr(), Parser::rName(), Parser::rNamespaceSpec(), Parser::rNewDeclarator(), Parser::rOperatorName(), Parser::rOtherDeclaration(), Parser::rPmExpr(), Parser::rPostfixExpr(), Parser::rPrimaryExpr(), Parser::rProgram(), Parser::rPtrToMember(), Parser::rRelationalExpr(), Parser::rShiftExpr(), Parser::rSizeofExpr(), Parser::rStatement(), Parser::rTempArgDeclaration(), Parser::rTempArgList(), Parser::rTemplateArgs(), Parser::rTemplateDecl2(), Parser::rThrowExpr(), Parser::rTryStatement(), Parser::rTypeSpecifier(), Parser::rUnaryExpr(), Parser::rUserdefKeyword(), Parser::rUserdefStatement(), Parser::rUsing(), Parser::rVarNameCore(), Parser::SkipTo(), and Parser::SyntaxError().

bool Lex::ReadCharConst (  uint    top )  [private]
 

Definition at line 586 of file token.cc.

References FALSE, file, Program::Get(), Program::GetCurPos(), token_len, TRUE, and uint.

Referenced by ReadLine().

int Lex::ReadComment (  char    c,
uint    top
)  [private]
 

Definition at line 1002 of file token.cc.

References comments, file, Program::Get(), Program::GetCurPos(), Ignore, Program::Read(), Ptree::Snoc(), token_len, uint, and Program::Unget().

Referenced by ReadSeparator().

int Lex::ReadFloat (  uint    top )  [private]
 

Definition at line 679 of file token.cc.

References Constant, file, Program::Get(), Program::GetCurPos(), is_digit(), is_eletter(), is_float_suffix(), Program::Rewind(), token, token_len, uint, and Program::Unget().

Referenced by ReadLine(), and ReadNumber().

int Lex::ReadIdentifier (  uint    top )  [private]
 

Definition at line 732 of file token.cc.

References file, Program::Get(), Program::GetCurPos(), is_digit(), is_letter(), Program::Read(), Screening(), token_len, uint, and Program::Unget().

Referenced by ReadLine().

int Lex::ReadLine (    )  [private]
 

Definition at line 532 of file token.cc.

References CharConst, file, Program::Get(), Program::GetCurPos(), GetNextNonWhiteChar(), is_digit(), is_letter(), last_token, ReadCharConst(), ReadFloat(), ReadIdentifier(), ReadLineDirective(), ReadNumber(), ReadSeparator(), ReadStrConst(), Program::Rewind(), SingleCharOp(), StringL, token, token_len, tokenp, uint, and Program::Unget().

Referenced by ReadToken().

bool Lex::ReadLineDirective (    )  [private]
 

Definition at line 722 of file token.cc.

References file, Program::Get(), and TRUE.

Referenced by ReadLine().

int Lex::ReadNumber (  char    c,
uint    top
)  [private]
 

Definition at line 644 of file token.cc.

References Constant, file, Program::Get(), Program::GetCurPos(), is_digit(), is_eletter(), is_hexdigit(), is_int_suffix(), is_xletter(), ReadFloat(), token, token_len, uint, and Program::Unget().

Referenced by ReadLine().

int Lex::ReadSeparator (  char    c,
uint    top
)  [private]
 

Definition at line 899 of file token.cc.

References ArrowOp, AssignOp, BadToken, Ellipsis, EqualOp, file, Program::Get(), IncOp, LogAndOp, LogOrOp, PmOp, ReadComment(), RelOp, Scope, ShiftOp, SingleCharOp(), token, token_len, uint, and Program::Unget().

Referenced by ReadLine().

bool Lex::ReadStrConst (  uint    top )  [private]
 

Definition at line 611 of file token.cc.

References FALSE, file, Program::Get(), Program::GetCurPos(), is_blank, Program::Rewind(), token_len, TRUE, and uint.

Referenced by ReadLine().

int Lex::ReadToken (  char *&   ,
int &   
)  [private]
 

Definition at line 331 of file token.cc.

References ASM, ATTRIBUTE, DECLSPEC, EXTENSION, Ignore, last_token, ReadLine(), SkipAttributeToken(), SkipExtensionToken(), TokenLen(), and TokenPosition().

Referenced by Lex::TokenFifo::Peek2(), and Lex::TokenFifo::Pop().

bool Lex::RecordKeyword (  char *   ,
int   
)  [static]
 

Definition at line 146 of file token.cc.

References HashTable::AddEntry(), FALSE, HashValue, HashTable::Peek(), token, TRUE, and user_keywords.

Referenced by InitializeOtherKeywords(), Class::RegisterMetaclass(), Class::RegisterNewAccessSpecifier(), Class::RegisterNewClosureStatement(), Class::RegisterNewForStatement(), Class::RegisterNewMemberModifier(), Class::RegisterNewModifier(), and Class::RegisterNewWhileStatement().

char Lex::Ref (  uint    i )  [private]
 

Definition at line 136 of file token.cc.

References file, Program::Ref(), and uint.

bool Lex::Reify (  Ptree *    t,
char *&   
)  [static]
 

Definition at line 210 of file token.cc.

References FALSE, Ptree::GetLength(), Ptree::GetPosition(), Ptree::IsLeaf(), and TRUE.

bool Lex::Reify (  Ptree *   ,
unsigned int &   
)  [static]
 

Definition at line 166 of file token.cc.

References FALSE, Ptree::GetLength(), Ptree::GetPosition(), is_digit(), is_int_suffix(), is_xletter(), Ptree::IsLeaf(), and TRUE.

Referenced by Ptree::Reify().

void Lex::Restore (  char *    ) 
 

Definition at line 93 of file token.cc.

References Lex::TokenFifo::Clear(), fifo, last_token, Rewind(), token_len, and tokenp.

Referenced by GetOnlyClosingBracket(), Parser::rAllocateType(), Parser::rArgDeclListOrInit(), Parser::rCastExpr(), Parser::rClassMember(), Parser::rExprStatement(), Parser::rSizeofExpr(), and Parser::rTemplateArgs().

void Lex::Rewind (  char *    )  [private]
 

Definition at line 141 of file token.cc.

References file, Program::Read(), and Program::Rewind().

Referenced by Restore().

char * Lex::Save (    ) 
 

Definition at line 84 of file token.cc.

References fifo, and Lex::TokenFifo::Peek().

Referenced by Parser::rAllocateType(), Parser::rArgDeclListOrInit(), Parser::rCastExpr(), Parser::rClassMember(), Parser::rExprStatement(), Parser::rSizeofExpr(), and Parser::rTemplateArgs().

int Lex::Screening (  char *    identifier,
int    len
)  [private]
 

Definition at line 870 of file token.cc.

References HashValue, Identifier, HashTable::Lookup(), rw_table::name, table, token, user_keywords, and rw_table::value.

Referenced by ReadIdentifier().

int Lex::SingleCharOp (  unsigned char    c )  [private]
 

Definition at line 987 of file token.cc.

References BadToken, and token.

Referenced by ReadLine(), and ReadSeparator().

void Lex::SkipAttributeToken (    )  [private]
 

Definition at line 377 of file token.cc.

References file, and Program::Get().

Referenced by ReadToken().

int Lex::SkipExtensionToken (  char *&   ,
int &   
)  [private]
 

Definition at line 399 of file token.cc.

References file, Program::Get(), Identifier, Ignore, is_blank, TokenLen(), TokenPosition(), and Program::Unget().

Referenced by ReadToken().

int Lex::TokenLen (    )  [inline, private]
 

Definition at line 105 of file token.h.

References token_len.

Referenced by ReadToken(), and SkipExtensionToken().

uint Lex::Tokenp (    )  [inline, private]
 

Definition at line 104 of file token.h.

References tokenp, and uint.

Referenced by TokenPosition().

char * Lex::TokenPosition (    )  [private]
 

Definition at line 131 of file token.cc.

References file, Program::Read(), and Tokenp().

Referenced by ReadToken(), and SkipExtensionToken().

Friends And Related Function Documentation

friend class TokenFifo [friend]
 

Definition at line 102 of file token.h.

Member Data Documentation

Ptree * Lex::comments = 0 [static, private]
 

Definition at line 71 of file token.cc.

Referenced by GetComments(), GetComments2(), and ReadComment().

TokenFifo Lex::fifo [private]
 

Definition at line 134 of file token.h.

Referenced by GetToken(), LookAhead(), Restore(), and Save().

Program* Lex::file [private]
 

Definition at line 133 of file token.h.

Referenced by GetNextNonWhiteChar(), Lex(), LineNumber(), ReadCharConst(), ReadComment(), ReadFloat(), ReadIdentifier(), ReadLine(), ReadLineDirective(), ReadNumber(), ReadSeparator(), ReadStrConst(), Ref(), Rewind(), SkipAttributeToken(), SkipExtensionToken(), and TokenPosition().

int Lex::last_token [private]
 

Definition at line 137 of file token.h.

Referenced by Lex(), ReadLine(), ReadToken(), and Restore().

int Lex::token_len [private]
 

Definition at line 136 of file token.h.

Referenced by Lex(), ReadCharConst(), ReadComment(), ReadFloat(), ReadIdentifier(), ReadLine(), ReadNumber(), ReadSeparator(), ReadStrConst(), Restore(), and TokenLen().

uint Lex::tokenp [private]
 

Definition at line 135 of file token.h.

Referenced by Lex(), ReadLine(), Restore(), and Tokenp().

HashTable * Lex::user_keywords = 0 [static, private]
 

Definition at line 70 of file token.cc.

Referenced by RecordKeyword(), and Screening().

The documentation for this class was generated from the following files:
Generated on Mon Feb 10 17:35:51 2003 for VFiasco Semantics Compiler by doxygen1.2.15