Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,15 @@ Note: Can be used with `oxsecurity/megalinter@beta` in your GitHub Action mega-l
- Fix `LINTER_RULES_PATH` not being used to resolve config files for linters using `active_only_if_file_found` (e.g. `REPOSITORY_LS_LINT`, `SPELL_PROSELINT`, `SPELL_VALE`), fixes [#8416](https://github.com/oxsecurity/megalinter/issues/8416)
- Honor `EXCLUDED_DIRECTORIES` and `ADDITIONAL_EXCLUDED_DIRECTORIES` in changed-files mode (`VALIDATE_ALL_CODEBASE: false`), so files inside excluded directories are pruned from the `git diff` file list the same way they are during full-codebase validation ([#8360](https://github.com/oxsecurity/megalinter/issues/8360))
- Declare `COPYPASTE_JSCPD` as `linux/amd64` only: since jscpd v5 the tool is a Rust binary shipped through per-platform npm packages, and upstream publishes no `linux-arm64-musl` target, so it exits with `Unsupported platform linux/arm64` on the Alpine-based ARM images
- Fix sporadic `ENOENT` crashes when `COPYPASTE_JSCPD` and `REPOSITORY_SECRETLINT` run in parallel: jscpd now writes its report to a temporary folder outside the linted workspace, and MegaLinter no longer deletes report files while other linters are still scanning them, fixes [#3979](https://github.com/oxsecurity/megalinter/issues/3979). As jscpd no longer removes its report at the end of a clean run, a copy-paste report left by a previous run is now removed when jscpd starts, so the report folder never mixes results from two runs
- Fix `REPOSITORY_SECRETLINT` silently narrowing its scan when `REPOSITORY_SECRETLINT_ARGUMENTS` names a `--secretlintignore` file that does not exist: MegaLinter no longer falls back to `.secretlintignore` or `.gitignore` in its place, as those patterns often exclude the very files a secrets scanner must inspect
- Fix the `copy-paste` report folder being created unreadable to non-root users (such as a later artifact-upload step), as it inherited the private permissions of the temporary folder jscpd writes to
- Stop `REPOSITORY_SECRETLINT` from scanning MegaLinter's own report folder, which could raise false positives on secrets echoed into other linters' reports, using an ignore file generated inside the report folder so nothing is ever written to or deleted from the linted sources
- Always exclude `REPORT_OUTPUT_FOLDER` from linted directories, even when `EXCLUDED_DIRECTORIES` is overridden
- Report a linter that crashes before producing SARIF output as a failure showing its raw output, instead of a spurious single finding accompanied by a SARIF parsing error
- Stop reporting a linter that exits without parsable SARIF output as a clean success: its results could not be counted, so it is now reported as a warning rather than as zero findings. Applies to linters declaring `can_output_sarif` when SARIF output is enabled, including security scanners such as `REPOSITORY_SECRETLINT`, `REPOSITORY_SEMGREP` and `REPOSITORY_TRIVY`
- Fix `REPOSITORY_SECRETLINT` ignoring a `.secretlintignore` located in `LINTER_RULES_PATH` (e.g. the default `.github/linters`): only its base name was passed, so secretlint resolved it from the workspace root and applied either no patterns at all or those of a different file with the same name
- Keep applying `REPOSITORY_SECRETLINT` ignore patterns when `REPORT_OUTPUT_FOLDER` is disabled, by generating the merged ignore file in a temporary folder outside the linted sources instead of giving up on it

- Reporters

Expand Down
6 changes: 4 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1285,7 +1285,7 @@ description: List of common variables that you can use to customize MegaLinter b
| [**ENABLE_ERRORS_LINTERS**](https://github.com/oxsecurity/megalinter/tree/main/docs/config-activation.md) | <!-- --> | List of enabled and blocking linters keys [(more info)](https://github.com/oxsecurity/megalinter/tree/main/docs/config-activation.md) |
| [**ENABLE**](https://github.com/oxsecurity/megalinter/tree/main/docs/config-activation.md) | <!-- --> | List of enabled descriptors keys [(more info)](https://github.com/oxsecurity/megalinter/tree/main/docs/config-activation.md) |
| [**ENABLE_LINTERS**](https://github.com/oxsecurity/megalinter/tree/main/docs/config-activation.md) | <!-- --> | List of enabled linters keys [(more info)](https://github.com/oxsecurity/megalinter/tree/main/docs/config-activation.md) |
| **EXCLUDED_DIRECTORIES** | \[…many values…\] | List of excluded directory basenames. They're excluded at any nested level. |
| **EXCLUDED_DIRECTORIES** | \[…many values…\] | List of excluded directory basenames. They're excluded at any nested level. The value of **REPORT_OUTPUT_FOLDER** is always excluded in addition to this list. |
| **EXTENDS** | <!-- --> | Base `mega-linter.yml` config file(s) to extend local configuration from. Can be a single URL or a list of `.mega-linter.yml` config files URLs. Later files take precedence. |
| **FAIL_IF_MISSING_LINTER_IN_FLAVOR** | `false` | If set to `true`, MegaLinter fails if a linter is missing in the selected flavor |
| **FAIL_IF_UPDATED_SOURCES** | `false` | If set to `true`, MegaLinter fails if a linter or formatter has autofixed sources, even if there are no errors |
Expand Down Expand Up @@ -1415,6 +1415,8 @@ ADDITIONAL_EXCLUDED_DIRECTORIES:
- vendor
```

MegaLinter's own report folder (**REPORT_OUTPUT_FOLDER**, `megalinter-reports` by default) is always excluded from linting, even if you override **EXCLUDED_DIRECTORIES**. Linting MegaLinter's own output produces false positives, and the reporters write to that folder while linters are still running.

<!-- config-filtering-section-end -->
<!-- config-apply-fixes-section-start -->
<!-- markdown-headers
Expand Down Expand Up @@ -1685,7 +1687,7 @@ Allowing `file` or `list_of_files` to be overridden to `project` is mostly for w

Special considerations:

- Linters that are configured to use the `project` lint mode ignore variables like `FILTER_REGEX_INCLUDE` and `FILTER_REGEX_EXCLUDE`, as they are not passed a list of files to lint. For those linters, you must check their documentation to see if a linter can be configured to ignore specific files. For example, the [Secretlint](https://megalinter.io/latest/descriptors/repository_secretlint/) linter ignores files listed in `~/.secretlintignore` by default, or it can be configured to instead ignore files listed in `~/.gitignore` by setting `REPOSITORY_SECRETLINT_ARGUMENTS` to `--secretlintignore .gitignore.`
- Linters that are configured to use the `project` lint mode ignore variables like `FILTER_REGEX_INCLUDE` and `FILTER_REGEX_EXCLUDE`, as they are not passed a list of files to lint. For those linters, you must check their documentation to see if a linter can be configured to ignore specific files. For example, the [Secretlint](https://megalinter.io/latest/descriptors/repository_secretlint/) linter ignores files listed in `~/.secretlintignore` by default, or it can be configured to ignore files listed in `~/.gitignore` by setting `REPOSITORY_SECRETLINT_ARGUMENTS` to `--secretlintignore .gitignore.` MegaLinter merges the patterns of that file into a generated ignore file, so the file you name is honored wherever it lives. When the report folder sits inside the linted workspace, it is appended to that generated file so MegaLinter's own reports are never scanned.

<!-- config-cli-lint-mode-section-end -->
<!-- configuration-section-end -->
Expand Down
2 changes: 1 addition & 1 deletion mega-linter-runner/lib/megalinter-vars.json
Original file line number Diff line number Diff line change
Expand Up @@ -7849,7 +7849,7 @@
"name": "EXCLUDED_DIRECTORIES",
"type": "array",
"title": "Excluded directories",
"description": "List of excluded directory basenames replacing MegaLinter default excluded directories. They are excluded at any nested level.",
"description": "List of excluded directory basenames replacing MegaLinter default excluded directories. They are excluded at any nested level. The value of REPORT_OUTPUT_FOLDER is always excluded in addition to this list.",
"examples": [
[
".github",
Expand Down
44 changes: 36 additions & 8 deletions megalinter/Linter.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,9 @@ def __init__(self, params=None, linter_config=None):
self.linter_speed = 3
self.can_output_sarif = False
self.output_sarif = False
# Sticky flag: set when SARIF output could not be parsed, so a linter that
# exits 0 with unparsable SARIF is not reported as clean (see get_sarif_result_count)
self.sarif_parse_failed = False
# ex: https://eslint.org/
self.linter_url = (
"Field 'linter_url' must be overridden at custom linter class level"
Expand Down Expand Up @@ -940,6 +943,18 @@ def run(
if self.cli_lint_mode == "list_of_files":
self.update_files_lint_results(self.files, None, None, None, None, None)

# A linter that exited without parsable SARIF has not been measured, so
# counting zero results would report it as clean. Every reporter still shows
# it with zero errors and zero warnings, hence the log: the status alone
# cannot tell an operator that the results are missing rather than empty.
if self.status == "success" and self.sarif_parse_failed is True:
self.status = "warning"
logging.warning(
f"[{self.linter_name}] results could not be counted, as its SARIF "
"output was not parsable: reporting this linter as a warning rather "
"than as a clean success"
)

# Set return code to 0 if failures in this linter must not make the MegaLinter run fail
if self.return_code != 0:
# Disable errors: no failure, just warning
Expand Down Expand Up @@ -1693,8 +1708,20 @@ def get_sarif_result_count(self, stdout: str, level: str):
sarif_output = yaml.safe_load(sarif_file)
# SARIF is in stdout
else:
# SARIF is in stdout
sarif_output = yaml.safe_load(stdout)
# SARIF is in stdout. Require it to actually have SARIF shape:
# find_json_in_stdout returns "" unless the payload parses as JSON
# containing a "runs" key. A linter that crashed before producing
# SARIF leaves a stack trace here, which must surface as a failure
# rather than be parsed as results
sarif_stdout = utils.find_json_in_stdout(stdout)
if sarif_stdout == "":
logging.error(
f"[{self.linter_name}] exited without producing parsable SARIF "
f"output while counting {level}s.\nLinter output: {stdout}"
)
self.sarif_parse_failed = True
return 0
sarif_output = json.loads(sarif_stdout)

for run in sarif_output["runs"]:
rule_default_level_map = {}
Expand Down Expand Up @@ -1737,14 +1764,15 @@ def get_sarif_result_count(self, stdout: str, level: str):

return total_result
except Exception as e:
total_result = 1
# Return 0 rather than inventing a finding: get_total_number_errors
# already reports 1 error for any non-success linter, so a broken linter
# still fails the run without being attributed a phantom result
logging.error(
f"Error while getting total {level}s from SARIF output.\nError:"
+ str(e)
+ "\nstdout: "
+ stdout
f"[{self.linter_name}] unable to compute total {level}s from SARIF "
f"output.\nError: {str(e)}\nLinter output: {stdout}"
)
return total_result
self.sarif_parse_failed = True
return 0

# Build the CLI command to get linter version (can be overridden if --version is not the way to get the version)
def build_version_command(self):
Expand Down
18 changes: 12 additions & 6 deletions megalinter/descriptors/repository.megalinter-descriptor.yml
Original file line number Diff line number Diff line change
Expand Up @@ -633,6 +633,15 @@ linters:
- **Team Collaboration**: Shared configuration files ensure consistent secret detection across development teams
- **Performance Optimized**: Fast scanning suitable for large repositories and automated workflows
- **Open Source**: Community-driven with transparent detection rules and continuous updates

**Ignore patterns in MegaLinter:**

MegaLinter does not pass your ignore file to secretlint directly. It generates a `.secretlintignore-megalinter` file inside **REPORT_OUTPUT_FOLDER** and passes that instead, because secretlint applies only one ignore file.

- The generated file merges the patterns of your own ignore file: the one named by `REPOSITORY_SECRETLINT_ARGUMENTS: --secretlintignore <file>` if you set it, otherwise your `.secretlintignore`, otherwise your `.gitignore`. Your file is honored wherever it lives, including in **LINTER_RULES_PATH**.
- MegaLinter's own report folder is appended last, so it is always excluded. Ignore rules are last match wins, so this cannot be undone by a negation pattern in your own file.
- Nothing else is added. MegaLinter's **EXCLUDED_DIRECTORIES** are deliberately not merged in: build artifacts such as `.terraform` or `.terragrunt-cache` can contain real credentials baked in at generation time, and a secrets scanner must still see them.
- When **REPORT_OUTPUT_FOLDER** is disabled, the generated file is written to a temporary folder outside your sources instead, so your patterns still apply.
linter_url: https://github.com/secretlint/secretlint
linter_repo: https://github.com/secretlint/secretlint
linter_banner_image_url: https://github.com/secretlint/secretlint/raw/master/docs/assets/SecretLintLP.png
Expand All @@ -652,12 +661,9 @@ linters:
- project
cli_lint_extra_args_after:
# secretlint v13 introduced a ripgrep-based file walker that respects
# .gitignore by default. --no-gitignore keeps the v12 scan breadth (scan
# everything, filter only via .secretlintignore). The .secretlintignore
# itself (which excludes .automation/test fixtures containing intentional
# fake secrets) is applied via SecretLintLinter.get_ignore_arguments,
# which passes it by base name because the v13 walker matches ignore
# files by base name and drops an absolute path.
# .gitignore by default. --no-gitignore keeps the v12 scan breadth: scan
# everything, filter only via the ignore file MegaLinter generates in
# SecretLintLinter.build_megalinter_ignore_file.
# See https://github.com/secretlint/secretlint/releases (v13)
- "--no-gitignore"
cli_lint_mode_project_extra_args_after:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9300,7 +9300,7 @@
},
"EXCLUDED_DIRECTORIES": {
"$id": "#/properties/EXCLUDED_DIRECTORIES",
"description": "List of excluded directory basenames replacing MegaLinter default excluded directories. They are excluded at any nested level.",
"description": "List of excluded directory basenames replacing MegaLinter default excluded directories. They are excluded at any nested level. The value of REPORT_OUTPUT_FOLDER is always excluded in addition to this list.",
"examples": [
[
".github",
Expand Down
Loading
Loading