Skip to content

Allow DataciteDoi shard configuration - #1563

Open
jrhoads wants to merge 3 commits into
masterfrom
shards-datacite-dois
Open

Allow DataciteDoi shard configuration#1563
jrhoads wants to merge 3 commits into
masterfrom
shards-datacite-dois

Conversation

@jrhoads

@jrhoads jrhoads commented Jul 21, 2026

Copy link
Copy Markdown
Contributor

Purpose

Separate Opensearch index template matching and shard configuration so that DataciteDoi receives its own shard setting without affecting Doi or OtherDoi indices.

Approach

Separate template handling for DataciteDoi/Doi and OtherDoi, use explicit index patterns for the shared DOI indices, and introduce a configurable shard count specifically for DataciteDoi.

Key Modifications

  • Updated Indexable#create_template to:
    • Avoid the broad dois* pattern, which also matches dois-other*.
    • Use explicit patterns for Doi and DataciteDoi:
      • dois
      • dois_v1
      • dois_v2
    • Keep wildcard matching for OtherDoi using its own alias prefix.
    • Apply DataciteDoi settings only to DataciteDoi templates.
  • Added NUMBER_OF_SHARDS_DATACITE_DOI configuration with a default value of 1.
    • This default number is also what modern opensearch defaults to. The elasticsearch-models gem has an older default value of 5. Too much for many environments, and still not enough for production.
  • Added a DataciteDoi-specific Elasticsearch setting for number_of_shards.
  • Added specs covering:
    • Default shard configuration.
    • Isolation of shard settings from Doi and OtherDoi.
    • Correct template patterns and settings for both DOI types.

Important Technical Details

  • DataciteDoi.settings inherits the base Doi index settings and overrides number_of_shards using ENV["NUMBER_OF_SHARDS_DATACITE_DOI"].
  • The environment variable is converted to an integer when applied to Elasticsearch settings.
  • OtherDoi continues to use Doi.settings, ensuring it does not inherit the Datacite-specific shard configuration.
  • Explicit index patterns prevent OtherDoi indices from being unintentionally matched by the main DOI template.

Types of changes

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)

Reviewer, please remember our guidelines:

  • Be humble in the language and feedback you give, ask don't tell.
  • Consider using positive language as opposed to neutral. This is to avoid the negative bias that can occur with neutral language appearing negative.
  • Offer suggestions on how to improve code, e.g. simplification or expanding clarity.
  • Ensure you give reasons for the changes you are proposing.

@jrhoads jrhoads changed the title Fix Elasticsearch DOI template matching and DataciteDoi shard configuration Allow DataciteDoi shard configuration Jul 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant