Ignore the generated directories of the common ecosystems

CHANGELOG.md

@@ -61,6 +61,9 @@
 - [#2119](https://github.com/bbatsov/projectile/pull/2119): The PHP types are now registered after the JavaScript ones, so a PHP application isn't detected as a Node project because of the `package.json` its asset pipeline ships.
 - [#2119](https://github.com/bbatsov/projectile/pull/2119): The supported project types table in the manual is now generated from the registry, which had drifted well behind it.
 - [#2121](https://github.com/bbatsov/projectile/pull/2121): `go.mod` is now registered as the Go type's project file, so it can anchor the root of a Go project outside version control and mark a module inside a larger repository. The type itself is still detected by predicate, since a directory of `.go` files is a Go project with or without modules.
+- [#2122](https://github.com/bbatsov/projectile/pull/2122): `projectile-globally-ignored-directories` now also covers the dependency and build output directories of the common ecosystems - `node_modules`, `target`, `_build`, `__pycache__`, `.venv`, `.next`, `.terraform` and a couple of dozen more.
+  - This mostly shows up under `native` and `hybrid` indexing, and in projects that aren't under version control; under `alien` the VCS was already ignoring those directories in nearly every project.
+  - Directories that projects do sometimes commit (`vendor`, `dist`, `public`, `build`) are deliberately left out. If one of the new entries hides something you want, add it to `projectile-globally-unignored-directories`, or put a `!` line in that project's `.projectile`.
 - [#2114](https://github.com/bbatsov/projectile/pull/2114): In `projectile-dispatch`, "display buffer" moved from `B` to `C-o` (mirroring its `s-p 4 C-o` binding), so `B` could become the bookmark prefix.
 - [#2110](https://github.com/bbatsov/projectile/pull/2110): The grep/ag search integration and the ignore predicates now derive their exclusions from the same gitignore patterns indexing uses, instead of expanding the dirconfig into absolute paths on their own.
   - `projectile-ignored-file-p` and `projectile-ignored-directory-p` now take an optional project root instead of a pre-computed list of ignored paths, and answer exactly what indexing would (ensure entries included, a file under an ignored directory counted as ignored).

doc/modules/ROOT/pages/ignoring.adoc

@@ -236,6 +236,18 @@ gitignore patterns described xref:pattern-language[above], and they take effect
 with every indexing method; under `alien` they are pushed down into the external
 indexing tool.
 
+Out of the box `projectile-globally-ignored-directories` covers the editor and
+version control directories plus the dependency and build output directories of
+the common ecosystems - `node_modules`, `target`, `__pycache__`, `.venv`,
+`.next`, `.terraform` and so on. Under `alien` your VCS is usually ignoring
+those anyway; the defaults matter for `native` and `hybrid` indexing, and for
+projects that aren't under version control at all.
+
+Directories that projects do sometimes commit - `vendor`, `dist`, `public`,
+`build` - are deliberately left out. If one of the defaults gets in your way,
+add it to `projectile-globally-unignored-directories`, or put a `!` line in
+that project's `.projectile` to get it back for that project alone.
+
 [source,elisp]
 ----
 ;; Ignore files by name, at any depth

projectile.el

@@ -624,24 +624,58 @@ were the gitignore pattern `*SUFFIX' (e.g. \".elc\" ignores every
   :package-version '(projectile . "0.12.0"))
 
 (defcustom projectile-globally-ignored-directories
-  '(".idea"
+  '(;; editors
+    ".idea"
     ".vscode"
+    ".ccls-cache"
+    ".cache"
+    ".clangd"
+    ;; version control
     ".git"
     ".hg"
     ".fslckout"
     "_FOSSIL_"
     ".bzr"
     "_darcs"
     ".pijul"
-    ".tox"
     ".svn"
-    ".stack-work"
-    ".ccls-cache"
-    ".cache"
-    ".clangd"
     ".sl"
     ".jj"
-    ".osc")
+    ".osc"
+    ;; dependencies and build output
+    ;;
+    ;; These are all directories a tool generates and a project doesn't
+    ;; commit.  Under `alien' they're usually excluded by the VCS anyway;
+    ;; the point of listing them is `native' and `hybrid' indexing, and
+    ;; projects that aren't under version control at all.  Names that some
+    ;; projects do commit - `vendor', `build', `dist', `public' - are
+    ;; deliberately not here.
+    "node_modules"
+    "target"
+    "_build"
+    ".gradle"
+    ".stack-work"
+    ".build"
+    "elm-stuff"
+    ".dart_tool"
+    ".zig-cache"
+    "zig-out"
+    "__pycache__"
+    "*.egg-info"
+    ".venv"
+    ".tox"
+    ".mypy_cache"
+    ".pytest_cache"
+    ".ruff_cache"
+    ".next"
+    ".nuxt"
+    ".svelte-kit"
+    ".astro"
+    ".turbo"
+    ".parcel-cache"
+    "_site"
+    ".terraform"
+    ".direnv")
   "A list of directories globally ignored by projectile.
 
 Entries are gitignore patterns, matched against paths relative to the
@@ -659,11 +693,18 @@ Matching is case-sensitive.  Note that a leading `*' is a plain
 wildcard - it used to be a marker meaning \"at any depth\", which is
 now the default for every slashless pattern.
 
+Besides the editor and version control directories, the default value
+covers the dependency and build output directories of the common
+ecosystems - `node_modules', `target', `__pycache__' and so on.  Add
+an entry to `projectile-globally-unignored-directories' to get one of
+them back, or a `!' line to a project's `.projectile' to get it back
+for that project only.
+
 See also `projectile-global-ignore-file-patterns'."
   :safe (lambda (x) (not (remq t (mapcar #'stringp x))))
   :group 'projectile
   :type '(repeat string)
-  :package-version '(projectile . "3.1.0"))
+  :package-version '(projectile . "3.3.0"))
 
 (defcustom projectile-globally-unignored-directories nil
   "A list of directories globally unignored by projectile.

test/projectile-ignore-test.el

@@ -83,6 +83,32 @@
                           files)
               :to-equal (projectile-remove-ignored files)))))
 
+(describe "projectile-globally-ignored-directories"
+  (it "keeps the generated directories of the common ecosystems out of the listing"
+    (let ((files '("src/main.js"
+                   "node_modules/left-pad/index.js"
+                   "target/debug/build.rs"
+                   "app/__pycache__/views.cpython-312.pyc"
+                   "demo.egg-info/PKG-INFO"
+                   ".venv/lib/python3.12/site-packages/x.py"
+                   "web/.next/server/pages.js"
+                   "infra/.terraform/providers/registry.tf")))
+      (expect (projectile-remove-ignored files) :to-equal '("src/main.js"))))
+
+  (it "leaves the directories projects do commit alone"
+    ;; Go vendors into `vendor', libraries commit `dist' and `public',
+    ;; and `build' is as often a source directory as an output one.
+    (let ((files '("vendor/github.com/pkg/errors/errors.go"
+                   "dist/bundle.js"
+                   "public/index.html"
+                   "build/ci.sh")))
+      (expect (projectile-remove-ignored files) :to-equal files)))
+
+  (it "can be overridden per entry by the unignored directories"
+    (let ((projectile-globally-unignored-directories '("node_modules")))
+      (expect (projectile-remove-ignored '("node_modules/left-pad/index.js"))
+              :to-equal '("node_modules/left-pad/index.js")))))
+
 (describe "projectile-globally-ignored-files"
   (it "includes TAGS file by default"
     (expect (member projectile-tags-file-name projectile-globally-ignored-files) :to-be-truthy))