Created by: natefaubion
I tried to touch the parser as little as possible, so scanComments/skipComments
still calls addComments
which in turn adds it to extra.comments
. readToken
checks the diff on the length and adds the most recent comments to the token. If you'd like me to factor this out in a different way that doesn't rely on extra.comments
, I can, but that might extend into more parts of the parser.
I also went ahead and changed the return type of read
to just return the tree now, since returning comments also seemed superfluous. This propagated to all the tests.
There are a couple points to consider before a merge:
- Comments are only printed in the final source if the token survives expansion, so you can get into a situation where you might comment a statement that starts with a macro invocation. The macro name won't survive and the comment will be lost. Should we copy over the comment on the macro name to the first token in the expanded code, or just defer to macro writers to implement this if they want it?
- I'm not sure what you want to do with
takeLine
. I added code to copy over comments because otherwise no comments where propagated to the final tree, but I don't know exactly wheretakeLine
is called and if it could result in duplicated comments in some instances.