using file-based metadata - #102

Draft

pkdash wants to merge 99 commits into

use-file-based-metadata

Commits on Jul 22, 2026

Adding schema files for core metadata, dataset, and data variable based on schema org (later these should be part of hsmodels)
pkdash
committed
cd569c0
Adding adapter module for resource metadata conversion between the new schema org based pydnatic models and the legacy pydantic models
pkdash
committed
d82d0d0
Adding load_json helper function to handle JSON metadata loading into schema models
pkdash
committed
6e98d72
Initial S3 integration for file-based json metadata handling
pkdash
committed
b285662
Additional S3 integration in resource and aggregation operations/functionalities
pkdash
committed
0a00f32
test: fxing hydroshare fixture to use s3 url path
pkdash
committed
01460fa
Refactor DefinedTerm classes to use Literal types for name fields
pkdash
committed
3ef09a6
fixing 'relation' field of CoreMetadata - deleting the comma
pkdash
committed
dd4f085
Adding metadata adapters and resource models for legacy resource metadata and schema.org based resource metadata convertion
pkdash
committed
2d9ba24
using metadata adapter for resource metadata conversion
pkdash
committed
1844420
Using the new metadata adapter in the helper load_json function
pkdash
committed
d49b00b
Adding tests for legacy and schema.org resource metadata adapters
pkdash
committed
d1835af
Adding tests for resource metadata editing using hsclient
pkdash
committed
36f336b
Adding s3fs to requirements for working with s3 buckets
pkdash
committed
9390d90
Fix type hint to use Union[str, None]
pkdash
committed
86b6533
refactor: Add AggregationTypeAdapter for converting between schema and legacy aggregation types
pkdash
committed
d611328
refactor: Update aggregation type handling to use aggregation type adapter
pkdash
committed
b649a10
refactor: associated media items (data files) retrieval for FileSetAggregation using file_manifest.json
pkdash
committed
2023bb7
refactor: exclude file_manifest.json and has_parts.json from S3 file retrieval when colllecting resource aggregations
pkdash
committed
784a501
refactor: rolling back folder rename logic to use API endpoint instead of S3 client
pkdash
committed
db32647
refactor: update file aggregation (create aggregation) logic ton use s3 protocol - replacing hsydroshare api endpoint use
pkdash
committed
0f5f1cb
fix: handling of associated media items (aggregation data files) as part of aggregation move functionality
pkdash
committed
53db523
fix: handling of aggregation media items (data files) as part of aggregation delete functionality
pkdash
committed
ca056d3
feat: add geographic validation for bounding box in SpatialReference model
pkdash
committed
d80d893
fix: add Private status to creativeWorkStatus field in CoreMetadata model
pkdash
committed
3a43485
feat: add SINGLE_FILE and FILE_SET types to AdditionalType enum
pkdash
committed
bb06a9b
fix: make variableMeasured and dimensions fields optional in ScientificDataset model
pkdash
committed
4e1a29b
refactor: simplify load_json function that loads json metadata to pydantic models
pkdash
committed
08d745e
refactot: pydantic models reorganization for core metadata, dataset (aggregation) and resource meta models
pkdash
committed
349d13d
fix: remove duplicate model field
pkdash
committed
b96dab0
feat: initial implementation of a raster metadata adpater for translating raster metadata between schema.org and legacy models
pkdash
committed
b60b8c9
fix: add missing entries to .gitignore for build and __pycache__ directories
pkdash
committed
e41f726
refactor: add optional fields for URL and license in CoreMetadata model
pkdash
committed
da1d058
refactor: add legacy raster models for now for the adapter to work - later update hsmodels
pkdash
committed
e6705bd
feat: metadata adapter to support geographic raster metadata conversion
pkdash
committed
32f9c7f
fix: add .DS_Store to .gitignore to prevent tracking of macOS system files
pkdash
committed
560d1cc

Commits on Jul 25, 2026

Commits on Aug 5, 2026

refactor: removing the aggregation from the resource aggregation list to avoid a full refesh of resource
pkdash
committed
650590e

Commits on Aug 6, 2026

Commits on Aug 7, 2026

Commits on Aug 8, 2026

Commits on Aug 9, 2026

Commits on Aug 11, 2026

Commits on Aug 13, 2026

Commits on Aug 14, 2026

Commits on Aug 15, 2026

Commits on Aug 20, 2026

add TODOs for updating HydroShare API endpoints related to file-based metadata handling
pkdash
committed
b930df2