252
edits
JWE Project Ideas: Difference between revisions
→Location info in the parse tree
| (7 intermediate revisions by the same user not shown) | |||
| Line 7: | Line 7: | ||
Refactor comment handling in lexer and parser. | Refactor comment handling in lexer and parser. | ||
* Gather and attach all comments to tokens in the lexer, never in the parser. This change will allow us to simplify the grammar by eliminating the stash_comment nonterminal symbol in the parser. | * [<span style="color:DarkGreen">Done</span>] Gather and attach all comments to tokens in the lexer, never in the parser. This change will allow us to simplify the grammar by eliminating the stash_comment nonterminal symbol in the parser. | ||
* Store tokens (with location info and comments attached) in the parse tree instead of imprecise data like "leading_comment" or "trailing_comment". This change will allow better location info for error reporting and easier and more accurate access to comments so that we can more easily find documentation strings or test and demo blocks (some of that work is already done). | * [<span style="color:DarkOrange">In Progress - still need to handle some separator tokens</span>] Store tokens (with location info and comments attached) in the parse tree instead of imprecise data like "leading_comment" or "trailing_comment". This change will allow better location info for error reporting and easier and more accurate access to comments so that we can more easily find documentation strings or test and demo blocks (some of that work is already done). | ||
* Recognize and tag comments that look like test or demo blocks. Skip those when looking for doc strings. | |||
* Update the parse tree classes to provide access to the new information stored in them. | |||
* Update the tree_print_code class to use the new info to provide better output. Allow comments to be omitted from the output. | |||
* Combine and simplify start_function, finish_function, and recover_from_parsing_function functions in the parser into a single make_function function. (This job is somewhat separate from the comment handling changes but now seems like a good time to do it.) | |||
* Modify Octave's demo function to find demo comment blocks that are associated with classdef methods. | |||
== [<span style="color:DarkGreen">Done</span>] Location info in the parse tree == | |||
Once all tokens are stored in the parse tree, we can eliminate the separate storage of line and column info and eliminate two arguments from nearly every tree_* class constructor. Any location info that is needed later for error messages, debugging, or code generation can be obtained from individual tokens. | |||
== MException object == | == MException object == | ||