RDoc::TokenStream

A TokenStream is a list of tokens, gathered during the parse of some entity (say a method). Entities populate these streams by being registered with the lexer. Any class can collect tokens by including TokenStream. From the outside, you use such an object by calling the #start_collecting_tokens method, followed by calls to #add_token and pop_token.

Public Instance Methods

add_token(*tokens) click to toggle source

Alias for: add_tokens

add_tokens(*tokens) click to toggle source

Adds tokens to the collected tokens

 
               # File rdoc/token_stream.rb, line 13
def add_tokens(*tokens)
  tokens.flatten.each { |token| @token_stream << token }
end

Also aliased as: add_token

collect_tokens() click to toggle source

Starts collecting tokens

 
               # File rdoc/token_stream.rb, line 22
def collect_tokens
  @token_stream = []
end

Also aliased as: start_collecting_tokens

pop_token() click to toggle source

Remove the last token from the collected tokens

 
               # File rdoc/token_stream.rb, line 31
def pop_token
  @token_stream.pop
end

start_collecting_tokens() click to toggle source

Alias for: collect_tokens

token_stream() click to toggle source

Current token stream

 
               # File rdoc/token_stream.rb, line 38
def token_stream
  @token_stream
end

tokens_to_s() click to toggle source

Returns a string representation of the token stream

 
               # File rdoc/token_stream.rb, line 45
def tokens_to_s
  token_stream.map { |token| token.text }.join ''
end

Home Classes Methods

In Files

Methods

Class/Module Index

RDoc::TokenStream

Public Instance Methods