Skip to content

Consolidate transient-keyword lexing: context-blind strip can corrupt string/comment text in the AST #70

Description

@apbendi

Background

Because solang-parser (even at latest upstream) cannot parse the transient keyword, scopelint handles it in two independent layers:

  1. src/parser.rs — on parse failure, a regex blanks every \btransient\b with same-length whitespace and re-parses. This regex is context-blind: it also blanks occurrences inside string literals and comments.
  2. src/check/validators/variable_names.rs — the validator re-scans the original source with a careful comment- and string-aware state machine to decide whether a state variable is transient.

Problem 1: silent corruption of string/comment text

When the fallback triggers (i.e. the file really uses transient storage), any transient token inside a string literal or comment is also blanked. The AST's string values and the parsed comment text then contain nine spaces where the word was. Downstream consumers that read that text — the EIP-712 typehash validator reads string literals, and spec/inline-config read comment text — would see corrupted content. This requires a file that both uses transient storage and mentions the word in a load-bearing string or comment, so it is unlikely in practice, but the failure would be silent when it hits.

Problem 2: three lexers for one job

  • declaration_start_before_name and contains_keyword_before_name in variable_names.rs are ~150 nearly identical lines of comment/string-aware scanning.
  • The parser.rs regex is a third, cruder implementation of "find transient tokens".

(The existing CommentState helper in comments.rs is not string-aware, so it cannot replace them as-is.)

Proposed fix

Do one careful, string/comment-aware lexing pass in sanitize() (in parser.rs) that:

  1. blanks only real keyword occurrences (skipping strings and comments), fixing Problem 1; and
  2. records the byte offsets of the blanked tokens into Parsed (e.g. transient_token_offsets: Vec<usize>).

The validator then determines "is this state variable transient" with a simple range query — is there a recorded offset between the declaration's start and the variable name — deleting the duplicated state machines in variable_names.rs (roughly half the code added in #64) and removing the per-variable O(file size) rescan as a bonus.

Existing unit tests in variable_names.rs (comment/same-line/ignore-directive cases) already pin the expected behavior, so the refactor can be validated against them.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions