Repository navigation
memoise_bioc_available(): query all Bioconductor repositories, not just BioCsoft - #402
Draft
aclark02-arcus wants to merge 2 commits into
Draft
aclark02-arcus wants to merge 2 commits into
aclark02-arcus wants to merge 2 commits into
Conversation
Previously memoise_bioc_available() hard-coded a single URL: https://bioconductor.org/packages/release/bioc/src/contrib/PACKAGES which corresponds to BioCsoft (the software repository) only. That meant packages hosted in BioCann, BioCexp, BioCworkflows and BioCbooks - ~1,400 additional packages in Bioconductor 3.22 - were invisible to pkg_bioc() and pkg_ref(), so risk assessments couldn't be produced for them. Query every BioC* repository advertised by BiocManager::repositories() via utils::available.packages(), combine the results, and record the actual repository each package resolves from. pkg_bioc() now uses that repository URL instead of hard-coding release/bioc, and assess_dependencies.pkg_bioc_remote() likewise queries all BioC repos instead of only the first entry of BiocManager::repositories(). Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR expands Bioconductor package discovery so riskmetric can recognize and assess packages hosted outside the BioCsoft (“software”) repo (e.g., annotation/experiment/workflows/books), by querying all BioC* repositories advertised by BiocManager::repositories().
Changes:
- Update
memoise_bioc_available()to query multiple Bioconductor repositories viautils::available.packages()and add a newbioc_repositories()helper. - Update
pkg_bioc()to derive its repository URL from the matched package’s repository entry (instead of hard-coding BioCsoft). - Update
assess_dependencies.pkg_bioc_remote()to query dependencies across all BioC repositories and add targeted tests + NEWS entry.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
R/utils_memoised.R |
Reworks Bioconductor availability lookup and introduces bioc_repositories() helper. |
R/pkg_ref_class.R |
Adjusts pkg_bioc() to record the repository a package resolves from. |
R/assess_dependencies.R |
Expands dependency lookup to search across all Bioconductor repositories. |
tests/testthat/test_memoise_bioc_available.R |
Adds unit tests for repo filtering and multi-repo availability behavior. |
NEWS.md |
Documents the expanded Bioconductor repository coverage. |
Comments suppressed due to low confidence (1)
tests/testthat/test_memoise_bioc_available.R:81
- Same as above:
.localwill be non-NULL under the test harness, so this call will try to read a local Bioc fixture (and currently errors because the fixture file is missing) rather than exercising the intended no-network path. Pass.local = NULLto force the code path under test.
repositories = function(...) character(0),
.package = "BiocManager",
{
out <- memoise_bioc_available()
expect_s3_class(out, "data.frame")
expect_equal(nrow(out), 0L)
expect_true(all(c("Package", "Version", "Repository") %in% names(out)))
}
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| available.packages = function(repos = NULL, ...) fake_ap, | ||
| .package = "utils", | ||
| { | ||
| out <- memoise_bioc_available() |
Comment on lines
+64
to
+73
| if (!is.null(.local)) { | ||
| db <- read.csv( | ||
| file.path(.local, "test_webmocks", "data", "bioc_packages.csv"), | ||
| stringsAsFactors = FALSE) | ||
| if (!"Repository" %in% names(db)) { | ||
| db[["Repository"]] <- paste0( | ||
| "https://bioconductor.org/packages/release/bioc", "/src/contrib") | ||
| } | ||
| return(db) | ||
| } |
Comment on lines
+65
to
+67
| bioc_repos <- bioc_repositories() | ||
| if (length(bioc_repos) == 0L) bioc_repos <- BiocManager::repositories()[1] | ||
| get_package_dependencies(x$name, repo = bioc_repos) |
Comment on lines
+213
to
+217
| repo <- if ("Repository" %in% colnames(info) && nrow(info) > 0L) { | ||
| sub("/src/contrib$", "", info[, "Repository"][1]) | ||
| } else { | ||
| "https://bioconductor.org/packages/release/bioc" | ||
| } |
aclark02-arcus
added a commit
to pharmaR/val.pipeline
that referenced
this pull request
Jul 30, 2026
Released riskmetric::pkg_bioc() builds pkg_bioc_remote with a hard-coded repo = "https://bioconductor.org/packages/release/bioc" discarding the Repository column returned by memoise_bioc_available(). Every downstream metric that follows x$repo_base_url -- pkg_ref_cache.web_url.pkg_bioc_remote() among others -- then scrapes bioconductor.org via httr::GET and fails on air-gapped hosts with <pkg_metric_error in curl::curl_fetch_memory(...): Failed to connect to bioconductor.org port 443: Connection refused> for has_news, remote_checks, news_current, has_vignettes, has_maintainer, bugs_status, has_source_control, has_bug_reports_url, license, and friends. Extend configure_riskmetric_offline() with Shim 4: replace pkg_bioc() with a version that pulls the Repository column from the shimmed memoise_bioc_available() (the internal PPM BioC URL) so x$repo_base_url points at the mirror and all derived scrapes hit the mirror too. Falls back to bioc_repos_from_config()[[1]] when the package is unknown. Mirrors the upstream fix at pharmaR/riskmetric#402. Standalone snippet dev/riskmetric-bioc-available-shim.txt updated to install Shim 4 as well. (#81) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…ix is_available_cran() ordering
Two related follow-ups to the memoise_bioc_available / pkg_bioc fix,
surfaced when field-testing on air-gapped Posit Package Manager mirrors.
## bioc_repositories() falls through to options('repos') / accepts override
The previous implementation only recognized BiocManager::repositories()
entries whose names start with 'BioC'. On a fresh session against an
internal PPM, BiocManager::repositories() often returns nothing usable
because the mirror URLs are supplied through options('repos') instead --
so bioc_repositories() returned character(0) and memoise_bioc_available()
still fell back to whatever was in the public BiocManager default,
defeating the whole point of the fix.
Widen the resolver, in order of specificity:
1. options('riskmetric.bioc_repos') -- explicit override, named char
or single URL.
2. Sys.getenv('RISKMETRIC_BIOC_REPOS') -- comma-separated URLs.
3. BiocManager::repositories() entries whose names begin 'BioC'
(existing behaviour).
4. options('repos') entries whose name begins 'BioC' *or* whose URL
contains 'bioconductor' / '/bioc/' (case-insensitive).
Public-network users are unaffected: BiocManager::repositories() still
returns the five BioC* entries and step 3 short-circuits before step 4.
## is_available_cran() no longer wins on BioC-known packages
verify_pkg_source() checks is_available_cran() *before*
is_available_bioc(). When options('repos') advertises both a CRAN and a
BioC entry (standard PPM setup, and equally common on-network with
options(repos = BiocManager::repositories()) sessions),
available.packages() over the full repo list finds a BioC package like
BiocGenerics and returns TRUE from is_available_cran() first --
classification stops at pkg_cran_remote and never reaches
is_available_bioc(). Downstream BioC-specific cache methods
(pkg_ref_cache.examples, pkg_ref_cache.remote_checks.pkg_bioc_remote,
pkg_ref_cache.web_url.pkg_bioc_remote, ...) then either error with 'no
applicable method' or scrape the wrong HTML endpoint.
Teach is_available_cran() to short-circuit FALSE when the package is
also present in memoise_bioc_available(). Dispatch then falls through
to is_available_bioc() as intended, and the ref is built as
pkg_bioc_remote with the correct repo_base_url from the pkg_bioc()
fix already in this branch.
## Tests
Four new tests in tests/testthat/test_memoise_bioc_available.R cover:
- the options('riskmetric.bioc_repos') override,
- the RISKMETRIC_BIOC_REPOS env var override,
- options('repos') URL-pattern fallthrough when BiocManager errors,
- is_available_cran() vetoing a BioC-known package while
memoise_available_packages() still contains it.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Motivation
Two related bugs bite
pkg_ref()/pkg_bioc()/pkg_assess()for Bioconductor packages, both surfaced by field-testing on internal Posit Package Manager (PPM) mirrors but with knock-on effects on the public-network path too.1.
memoise_bioc_available()only sees the software repoThe current implementation hard-codes:
url("https://bioconductor.org/packages/release/bioc/src/contrib/PACKAGES")That gives it only the
BioCsoftsnapshot, so annotation, experiment, workflow and books packages (~1,400 additional in Bioc 3.22) are invisible topkg_bioc()/pkg_ref(). On air-gapped hosts the URL is unreachable entirely, and every BioC assessment fails with:2.
pkg_bioc()throws away the resolved repositoryEven after
memoise_bioc_available()starts returning the correct Repository column,pkg_bioc()builds thepkg_bioc_remotereference with:hard-coded. So
x$repo_base_urlis alwaysbioconductor.org, and every downstream metric that scrapesx$web_html(has_news,remote_checks,news_current,has_vignettes,has_maintainer,bugs_status,has_source_control,has_bug_reports_url,license, ...) hitsbioconductor.orgtoo:Changes
memoise_bioc_available()rewritten to query every Bioconductor repository frombioc_repositories()viautils::available.packages(), dedup byPackage(software repo wins), preserve theRepositorycolumn, and stay memoised. Public-network callers still gethttps://bioconductor.org/...in Repository; air-gapped callers get the internal mirror URL — automatically.pkg_bioc()now uses theRepositorycolumn, stripping the trailing/src/contrib.x$repo_base_urltherefore points at the actual mirror, notbioconductor.org.bioc_repositories()(new helper) resolves the repo set in order of specificity:options("riskmetric.bioc_repos")— explicit override.Sys.getenv("RISKMETRIC_BIOC_REPOS")— comma-separated URLs.BiocManager::repositories()entries whose names beginBioC(existing behaviour; public network unchanged).options("repos")entries whose name beginsBioCor whose URL containsbioconductor//bioc/. This covers PPM setups where the BioC snapshot is only surfaced viaoptions("repos")(e.g.c(CRAN = "<ppm>/cran/latest", BioC = "<ppm>/bioconductor-3.22/latest")), not throughBiocManager::repositories().is_available_cran()now short-circuits FALSE when the package is also present inmemoise_bioc_available().verify_pkg_source()dispatches CRAN-before-BioC, so before this change any BioC package advertised throughoptions("repos")alongside a CRAN repo was classified aspkg_cran_remote— which then brokepkg_ref_cache.examples(nopkg_cran_remotemethod) and routedremote_checks/web_urlthrough the CRAN scrape paths against a Bioconductor package. This affects on-network sessions as much as air-gapped ones wheneveroptions(repos)contains both a CRAN and a BioC entry.assess_dependencies.pkg_bioc_remote()now queries all Bioconductor repos (previously only the first entry ofBiocManager::repositories()), with a fallback to preserve current behaviour ifbioc_repositories()is empty.Compatibility
BiocManager::repositories()path (BioCsoft, BioCann, BioCexp, BioCworkflows, BioCbooks). The two new fall-throughs (options(repos)and explicit override) only apply when the earlier tiers return nothing.Tests
New / expanded tests in
tests/testthat/test_memoise_bioc_available.R:memoise_bioc_available()querying all five BioC repos andpkg_bioc()picking up theRepositorycolumn.bioc_repositories()honoursoptions("riskmetric.bioc_repos").bioc_repositories()honoursSys.getenv("RISKMETRIC_BIOC_REPOS").bioc_repositories()falls through tooptions("repos")and matches on URL pattern whenBiocManager::repositories()errors.is_available_cran()vetoes packages known tomemoise_bioc_available()even whenmemoise_available_packages()still reports them.Related
Companion draft PR: #401 replaces
devtools::revdep(bioconductor = TRUE)insideassess_reverse_dependencies.default()(same air-gapped failure class, hits.../VIEWSinstead of.../PACKAGES).Opened as a draft for maintainer discussion.