Skip to content

Ownership for self-bordering blocks #1445

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
c-prather wants to merge 1 commit into
base: develop
Choose a base branch
from
Draft

Ownership for self-bordering blocks #1445

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions src/mesh/forest/block_ownership.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ namespace parthenon {
block_ownership_t
DetermineOwnership(const LogicalLocation &main_block,
const std::vector<forest::NeighborLocation> &allowed_neighbors,
const std::array<bool, 3> self_border_block,

@lroberts36 lroberts36 Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to actually pass this in or can we just check if the main_block logical location is in allowed_neighbors?

const std::unordered_set<LogicalLocation> &newly_refined) {
block_ownership_t main_owns;

Expand All @@ -63,6 +64,19 @@ DetermineOwnership(const LogicalLocation &main_block,
return a.morton() < b.morton();
};

// Does this block border itself over this face, and are we on the left side?
// Then Morton number would say we own the face/edge/corner, but we don't
// (the right side does).
auto self_border_left = [self_border_block](int ox1, int ox2, int ox3) {
// Can take care of each direction independently, corners follow
if ((self_border_block[0] && ox1 < 0) || (self_border_block[1] && ox2 < 0) ||
(self_border_block[2] && ox3 < 0)) {
return true;
} else {
return false;
}
};

for (int ox1 : {-1, 0, 1}) {
for (int ox2 : {-1, 0, 1}) {
for (int ox3 : {-1, 0, 1}) {
Expand All @@ -72,6 +86,10 @@ DetermineOwnership(const LogicalLocation &main_block,
main_block.IsNeighborOfTE(n.origin_loc, {ox1, ox2, ox3})) {
main_owns(ox1, ox2, ox3) = false;
break;
} else if (main_block.level() == 0 && main_block == n.global_loc &&
self_border_left(ox1, ox2, ox3)) {
main_owns(ox1, ox2, ox3) = false;
break;
Comment on lines +89 to +92

@lroberts36 lroberts36 Sep 2, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, I see you are making the comparison here. So why do we need the self_border_block information at all? Shouldn't we be able to just have a tie breaker for ownership based on the offsets for the two sides in their own frames?

@lroberts36 lroberts36 Sep 2, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The main thing I am thinking here is that in a general forest, there can be all sorts of connectivity between a block and itself beyond what a periodic mesh would give.

@c-prather c-prather Sep 2, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, yeah I get what you mean.
I was treating ownership_less_than as always comparing blocks, but we can just make it direction-aware. Testing a much smaller patch that just does that now

}
}
}
Expand Down
1 change: 1 addition & 0 deletions src/mesh/forest/block_ownership.hpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,7 @@ namespace parthenon {
block_ownership_t
DetermineOwnership(const LogicalLocation &main_block,
const std::vector<forest::NeighborLocation> &allowed_neighbors,
const std::array<bool, 3> self_border_block = {false, false, false},
const std::unordered_set<LogicalLocation> &newly_refined = {});

// Given a topological element, ownership array of the sending block, and offset indices
Expand Down
19 changes: 17 additions & 2 deletions src/mesh/mesh-gmg.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -62,6 +62,21 @@ void SetMeshBlockNeighbors(Mesh *pmesh, GridIdentifier grid_id, BlockList_t &blo
{ndim > 2 ? -1 : 0, ndim > 2 ? 1 : 0});
BufferID buffer_id(ndim, multilevel);

// If we're periodic with just 1 block in any direction,
// we must treat ownership specially
auto block_size = pmesh->GetDefaultBlockSize();
std::array<bool, 3> self_border_block{
!pmesh->mesh_size.symmetry(X1DIR) &&
pmesh->mesh_size.nx(X1DIR) / block_size.nx(X1DIR) == 1 &&
pmesh->mesh_bcs[BoundaryFace::inner_x1] == BoundaryFlag::periodic,
!pmesh->mesh_size.symmetry(X2DIR) &&
pmesh->mesh_size.nx(X2DIR) / block_size.nx(X2DIR) == 1 &&
pmesh->mesh_bcs[BoundaryFace::inner_x2] == BoundaryFlag::periodic,
!pmesh->mesh_size.symmetry(X3DIR) &&
pmesh->mesh_size.nx(X3DIR) / block_size.nx(X3DIR) == 1 &&
pmesh->mesh_bcs[BoundaryFace::inner_x3] == BoundaryFlag::periodic,
};

for (auto &pmb : block_list) {
std::vector<NeighborBlock> all_neighbors;
const auto &loc = pmb->loc;
Expand Down Expand Up @@ -90,8 +105,8 @@ void SetMeshBlockNeighbors(Mesh *pmesh, GridIdentifier grid_id, BlockList_t &blo
auto &nb = all_neighbors.back();
auto neighbor_neighbors = forest.FindNeighbors(nloc.global_loc, grid_id);

nb.ownership =
DetermineOwnership(nloc.global_loc, neighbor_neighbors, newly_refined);
nb.ownership = DetermineOwnership(nloc.global_loc, neighbor_neighbors,
self_border_block, newly_refined);
nb.ownership.initialized = true;

// Set logical coordinate transformation from this block to the neighbor
Expand Down
Loading