Issues/252 megalinter

[nicain]

Fix #252

Checking out megalinter; slight hiccup using npx mega-linter-runner because of my sudo-protected docker:

What I tried: npx mega-linter-runner

What I got:

Pulling docker image oxsecurity/megalinter:v6 ... 
INFO: this operation can be long during the first use of mega-linter-runner
The next runs, it will be immediate (thanks to docker cache !)
WARNING: Error loading config file: /root/.docker/config.json: open /root/.docker/config.json: permission denied
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Post "http://%2Fvar%2Frun%2Fdocker.sock/v1.24/images/create?fromImage=oxsecurity%2Fmegalinter&tag=v6": dial unix /var/run/docker.sock: connect: permission denied

How I fixed it: https://stackoverflow.com/a/54504083
sudo setfacl --modify user:<user name or ID>:rw /var/run/docker.sock

[github-actions]

🦙 MegaLinter status: ❌ ERROR

Descriptor	Linter	Files	Fixed	Errors	Elapsed time
❌ ACTION	actionlint	12		5	0.15s
⚠️ BASH	bash-exec	21		2	0.04s
✅ BASH	shellcheck	21		0	0.43s
✅ BASH	shfmt	21	20	0	0.07s
❌ COPYPASTE	jscpd	yes		51	3.96s
✅ CSS	stylelint	2	0	0	1.4s
✅ DOCKERFILE	hadolint	7		0	0.29s
✅ ENV	dotenv-linter	2	2	0	0.07s
❌ HTML	djlint	1		2	0.62s
✅ HTML	htmlhint	1		0	0.28s
❌ JAVASCRIPT	eslint	28	1	1	2.88s
❌ JAVASCRIPT	standard	28	28	1	8.98s
✅ JSON	eslint-plugin-jsonc	13	0	0	2.16s
✅ JSON	jsonlint	12		0	0.22s
❌ JSON	npm-package-json-lint	yes		1	0.59s
✅ JSON	prettier	12	4	0	1.83s
✅ JSON	v8r	12		0	13.06s
✅ MARKDOWN	markdownlint	10	8	0	0.63s
❌ MARKDOWN	markdown-link-check	10		4	10.67s
✅ MARKDOWN	markdown-table-formatter	10	8	0	0.33s
❌ PYTHON	bandit	72		255	1.81s
✅ PYTHON	black	72	10	0	3.83s
✅ PYTHON	flake8	72		0	1.28s
✅ PYTHON	isort	72	11	0	0.39s
❌ PYTHON	mypy	72		1	0.51s
❌ PYTHON	pylint	72		189	8.62s
❌ PYTHON	pyright	72		261	9.92s
❌ PYTHON	ruff	72	11	31	0.13s
❌ REPOSITORY	checkov	yes		48	13.49s
❌ REPOSITORY	devskim	yes		50	0.84s
✅ REPOSITORY	dustilock	yes		no	3.98s
✅ REPOSITORY	gitleaks	yes		no	0.7s
❌ REPOSITORY	git_diff	yes		1	0.1s
✅ REPOSITORY	secretlint	yes		no	1.51s
✅ REPOSITORY	syft	yes		no	0.71s
❌ REPOSITORY	trivy	yes		1	7.17s
❌ SPELL	cspell	209		1377	106.57s
✅ SPELL	misspell	208	24	0	0.23s
❌ TERRAFORM	checkov	20		8	56.81s
❌ TERRAFORM	kics	20		10	881.1s
✅ TERRAFORM	terraform-fmt	20	14	0	1.63s
❌ TERRAFORM	terrascan	yes		1	9.54s
✅ TERRAFORM	tflint	20		0	2.24s
✅ YAML	prettier	18	18	0	1.0s
❌ YAML	v8r	18		1	22.71s
❌ YAML	yamllint	18		1	0.5s

See detailed report in MegaLinter reports

You could have same capabilities but better runtime performances if you request a new MegaLinter flavor.

• Click here to request the new flavor

MegaLinter is graciously provided by

[koverholt]

Looks like a net positive change to me: smaller Docker images, parallel runs, easy links to errors via a comment, and other improvements over super-linter. 👍

My only question is on the total runtime, it seems like super-linter took 13-15 mins to run, whereas megalinter is taking 15-18 mins to run, specifically because of the terraform_kics tests. What is taking so long in those tests, is it spinning up infra (or mock infra)?

I wonder if it would be worthwhile to disable terraform_kics tests, then the linting would take ~70 seconds (based on the next-slowest-running test). Then you could configure a "nightly" (or similar infrequent) run of the tests that includes compliance with terraform_kics.

[koverholt]

I don't think this repo has a master branch, correct?

[nvuillam]

@koverholt most repositories using MegaLinter disable some linters / rules / errors etc... for a lot of reasons

I have no idea why KICS spends so much time on terraform files, but with all other linters checking terraform it would feel safe to disable it by adding the following in .mega-linter.yml config file

DISABLE_LINTERS:
  - TERRAFORM_KICS

PYTHON_PYRIGHT and REPOSITORY_DEVSKIM are also known to raise lots of not so relevant issues... I usually disable them too

About cspell, in the report artifacts zip you have an updated .cspell config that you can paste at the root of your repo, and by viewing the diff you'll see which are real words that needs to be accept and which are typos that should be corrected

You can also decide to use MegaLinter gradually by defining some linters are not blocking using variable DISABLE_ERRORS_LINTERS

More info here: https://megalinter.io/latest/configuration/#activation-and-deactivation

[nvuillam]

About eslint issue, you could add the following in .mega-linter.yml config file

PRE_COMMANDS:
  - command: npm install eslint-plugin-json

[nicain]

I have no idea why KICS spends so much time on terraform files, but with all other linters checking terraform it would feel safe to disable it by adding the following in .mega-linter.yml config file

@nvuillam Wow what a coincidence, I was just doing exactly this! TY for reaching out!!

[nicain]

@nvuillam
One thing about my setup; I run docker explicitly with sudo; when I ran megalinter with APPLY_FIXES: all, it changed the owner and group of every shell script in my repo to root/root (presumably through a docker mount). Is this a known issue? Any way to avoid this? Should I file a bug?

[nvuillam]

@nicain the planets are aligned ;)

About other linter issues, you can decide if you keep them enabled or not, blocking or not

If you want to keep them enabled but disable some errors , each linter doc on MegaLinter has a direct link to the related linter page containing the doc explaining how to disable stuff ^^

Note: there is an awful bug in latest version, if you see a 404 when clicking to the doc, juste remove the version number of the url to keep "latest"

[nvuillam]

@nvuillam One thing about my setup; I run docker explicitly with sudo; when I ran megalinter with APPLY_FIXES: all, it changed the owner and group of every shell script in my repo to root/root (presumably through a docker mount). Is this a known issue? Any way to avoid this? Should I file a bug?

There is a current issue oxsecurity/megalinter#1975 and a PR oxsecurity/megalinter#1985 , handled by @Kurt-von-Laven

It has testing CI jobs failing... if you are a docker expert maybe you could help by providing an example of docker run command that would work on your config, and we could add an option to mega-linter-runner to activate it !

[nvuillam]

If it can help, the docker run command is built here -> https://github.com/oxsecurity/megalinter/blob/0234f93c064f6e5da9414dea7b764b640ede3613/mega-linter-runner/lib/runner.js#L123

You can also see it as it is run in the logs

[nvuillam]

Other tip: if you're bored to see all those config files at the root of the repo, you can put them in ./github/linters and it will work the same :) ( except for those that you use with IDE extensions, in that case you need to let them at the root so they are found by the extensions )

[nicain]

@koverholt I think the best way to move this forward is to:

• Add the GH Actions yaml config for this linter
• Start small with specific Linters that we need, fixing up any failures as we go.
• Migrate the config files to the .github/linters folder
• When the functionality subsumes super-linter, we can deactivate it and switch the megalinter flow to the blocker.

If we use the same config files for both linters, we should have parity between the two functionalities, so we can ease a transition one piece at a time.

WDYT?

[koverholt]

That sounds good to me! I'd be in favor of enabling the linters that are passing in this PR, then as a next pass (in another PR) enabling the linters that have < 5 errors and fixing them, then switching to megalinter as you mentioned.

[nvuillam]

If I may, I suggest you to do the following:

• disable linters you know you will never want, using DISABLE_LINTERS property
• keep active linters with errors but make them not blocking for PRs, using DISABLE_ERRORS_LINTERS
  • for those ones, you'll still know what to fix in future PRs ( example: dead links detected by MARKDOWN_MARKDOWN_LINK_CHECK )

[nvuillam]

FYI, url bug leading to 404 is fixed :)

[koverholt]

Going to close this PR for now and revisit in the future while I fix current CI tests in #383.