Skip to content
Merged
Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
8681bac
Add out-of-core DataLoader for minibatch variational inference
YichengYang-Ethan Jun 23, 2026
f1bb776
Fix ruff lint in streaming tests: escape regex metacharacter in match…
YichengYang-Ethan Jul 5, 2026
f8c50e5
Merge remote-tracking branch 'origin/main' into streaming-dataset
YichengYang-Ethan Jul 19, 2026
69f9deb
Rename streaming.py to dataloader.py (per review)
YichengYang-Ethan Jul 19, 2026
9e2c2a1
Address DataLoader review: auto default, less indirection, casual docs
YichengYang-Ethan Jul 19, 2026
30c3c9c
Set copyright year to 2026 on the new files (per review)
YichengYang-Ethan Jul 20, 2026
9a11709
Trim the module docstring and add a DataLoader example (per review)
YichengYang-Ethan Jul 20, 2026
f13e0c1
Share one Parquet schema check and flatten the source normalizer
YichengYang-Ethan Jul 22, 2026
0b0d218
Consolidate the DataLoader tests with parametrize
YichengYang-Ethan Jul 22, 2026
fba3767
Drop the batch counters and give the loader one iteration path
YichengYang-Ethan Jul 22, 2026
7febeee
Keep the parts of two messages that said what to do
YichengYang-Ethan Jul 22, 2026
4d89081
Cut the __len__ docstring down to the one thing it has to say
YichengYang-Ethan Jul 22, 2026
7ae9324
Fold the three shuffle=True tests into one, and pin the copy
YichengYang-Ethan Jul 22, 2026
fed3c30
Copy source blocks before the source is pulled again
YichengYang-Ethan Aug 1, 2026
e925a01
Pin the DataLoader's conservation and equivalence invariants
YichengYang-Ethan Aug 1, 2026
088805a
Take the sample shape from the blocks instead of a constructor argument
YichengYang-Ethan Aug 2, 2026
3e285ef
Inline _prepare and drop IterableDataset
zaxtax Aug 6, 2026
80e69a2
Drop all size validation — do what PyTorch does
zaxtax Aug 6, 2026
d2d7265
Remove comments that add nothing
zaxtax Aug 6, 2026
945ef7c
Drop meandering comment in _auto_total_size
zaxtax Aug 6, 2026
153114e
Trim meandering docstrings
zaxtax Aug 6, 2026
08f9d5e
Remove _rebatch: source must yield exact batch_size blocks
zaxtax Aug 6, 2026
e386f59
Match torch: len(DataLoader) returns batch count, not N
zaxtax Aug 6, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
26 changes: 26 additions & 0 deletions pymc_extras/variational/__init__.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
# Copyright 2026 - present The PyMC Developers
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
from pymc_extras.variational.dataloader import (
DataLoader,
IterableDataset,
parquet_source,
shuffle_buffer,
)

__all__ = [
"DataLoader",
"IterableDataset",
"parquet_source",
"shuffle_buffer",
]
Loading