Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/seeded-sweep-draws.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@hashintel/petrinaut": patch
---

Range sweeps rotate each axis's low-discrepancy parameter draws by a seed-derived shift (Cranley–Patterson), so draws are unbiased over the seed and experiments with different seeds explore different value sequences — prefix stability and the selection cache are unaffected.
2 changes: 1 addition & 1 deletion libs/@hashintel/petrinaut/docs/experiments.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ A sweep computes **what you have selected**. The results drawer grows a **Parame

Move a slider and compute immediately restarts on the new selection, like a raytracer dropping its rays when the camera moves. Every position you have visited keeps its results: narrowing a range, collapsing to a point, or sliding back to an earlier value restores its runs and distributions instantly, and refinement resumes where it left off.

Every selection uses the same seed sequence (common random numbers), and a run's parameter draw depends only on its position in the sequence, so differences you see between selections come from the parameters, not from sampling luck.
Every selection uses the same seed sequence (common random numbers), and a run's parameter draw depends only on the experiment's seed and the run's position in the sequence, so differences you see between selections come from the parameters, not from sampling luck — while experiments with different seeds explore their own value sequences.

#### The surface view

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -201,31 +201,57 @@ describe("selections and regions", () => {
describe("sweepRunFraction", () => {
it("is prefix-stable: a run's draw never depends on how many runs exist", () => {
const first = Array.from({ length: 8 }, (_, index) =>
sweepRunFraction(index, 0),
sweepRunFraction(7, index, 0),
);
const extended = Array.from({ length: 25 }, (_, index) =>
sweepRunFraction(index, 0),
sweepRunFraction(7, index, 0),
);
expect(extended.slice(0, 8)).toEqual(first);
});

it("spreads early runs across the unit interval", () => {
const fractions = Array.from({ length: 8 }, (_, index) =>
sweepRunFraction(index, 0),
);
expect(Math.min(...fractions)).toBeLessThan(0.2);
expect(Math.max(...fractions)).toBeGreaterThan(0.8);
it("spreads early runs across the unit interval, whatever the seed", () => {
for (const seed of [0, 7, 123_456, 2 ** 31 - 1]) {
const fractions = Array.from({ length: 8 }, (_, index) =>
sweepRunFraction(seed, index, 0),
);
expect(Math.min(...fractions)).toBeLessThan(0.2);
expect(Math.max(...fractions)).toBeGreaterThan(0.8);
for (const fraction of fractions) {
expect(fraction).toBeGreaterThanOrEqual(0);
expect(fraction).toBeLessThan(1);
}
}
});

it("draws different axes from different sequences", () => {
const xDraws = Array.from({ length: 6 }, (_, index) =>
sweepRunFraction(index, 0),
sweepRunFraction(7, index, 0),
);
const yDraws = Array.from({ length: 6 }, (_, index) =>
sweepRunFraction(index, 1),
sweepRunFraction(7, index, 1),
);
expect(xDraws).not.toEqual(yDraws);
});

it("reproduces exactly for the same seed", () => {
const first = Array.from({ length: 12 }, (_, index) =>
sweepRunFraction(42, index, 0),
);
const again = Array.from({ length: 12 }, (_, index) =>
sweepRunFraction(42, index, 0),
);
expect(again).toEqual(first);
});

it("draws a different value sequence per experiment seed", () => {
const seedA = Array.from({ length: 6 }, (_, index) =>
sweepRunFraction(1, index, 0),
);
const seedB = Array.from({ length: 6 }, (_, index) =>
sweepRunFraction(2, index, 0),
);
expect(seedA).not.toEqual(seedB);
});
});

describe("getNextRunTarget", () => {
Expand Down
37 changes: 30 additions & 7 deletions libs/@hashintel/petrinaut/src/react/experiments/parameter-grid.ts
Original file line number Diff line number Diff line change
Expand Up @@ -235,21 +235,44 @@ function radicalInverse(index: number, base: number): number {
return result;
}

/**
* A seed-derived fraction in [0, 1): each axis's Cranley–Patterson shift.
* `Math.imul` keeps every product in 32 bits — plain `*` would round through
* f64 and diverge across engines.
*/
/* eslint-disable no-bitwise -- a 32-bit hash is bit manipulation by definition */
function axisShift(seed: number, axisIndex: number): number {
let hash =
(Math.imul(seed | 0, 0x9e3779b1) ^ Math.imul(axisIndex + 1, 0x85ebca6b)) >>>
0;
hash = Math.imul(hash ^ (hash >>> 15), 0x2c1b3c6d) >>> 0;
hash = Math.imul(hash ^ (hash >>> 13), 0x297a2d39) >>> 0;
hash ^= hash >>> 16;
return (hash >>> 0) / 4_294_967_296;
}
/* eslint-enable no-bitwise */

/**
* Where run `globalRunIndex` falls along ranged axis `axisIndex`, in [0, 1):
* a per-axis low-discrepancy sequence (radical inverse in a distinct prime
* base per axis), so any prefix of runs covers every range near-uniformly and
* jointly. Prefix-stable in the run index — a ladder batch extends the exact
* sequence earlier batches drew from, so cached runs never go stale.
* base per axis), rotated by a seed-derived shift (Cranley–Patterson). Any
* prefix of runs covers every range near-uniformly and jointly; the rotation
* makes the draws unbiased over the seed and gives every experiment seed its
* own value sequence. Prefix-stable — a draw depends only on the seed, the
* axis, and the run index — so a ladder batch extends the exact sequence
* earlier batches drew from, and cached runs never go stale.
*/
export function sweepRunFraction(
seed: number,
globalRunIndex: number,
axisIndex: number,
): number {
return radicalInverse(
globalRunIndex + 1,
HALTON_BASES[axisIndex % HALTON_BASES.length]!,
);
const fraction =
radicalInverse(
globalRunIndex + 1,
HALTON_BASES[axisIndex % HALTON_BASES.length]!,
) + axisShift(seed, axisIndex);
return fraction >= 1 ? fraction - 1 : fraction;
}

function mergeDistributionBins(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -212,9 +212,11 @@ export function sweepSelectionMidValues(
* indices `[from, target)`, or undefined when every axis is a point. Each
* ranged axis draws continuously inside its selected value interval — the
* quantized positions bound the interval, they do not grid it — via the
* axis's own low-discrepancy sequence, prefix-stable in the run index.
* axis's own seed-shifted low-discrepancy sequence, prefix-stable in the
* run index.
*/
export function sweepRangeRuns(
seed: number,
axes: readonly ExperimentParameterAxis[],
selection: SweepSelection,
from: number,
Expand Down Expand Up @@ -246,7 +248,8 @@ export function sweepRangeRuns(
const globalIndex = from + localIndex;
const parameterValues: Record<string, string> = {};
for (const { axis, axisIndex, low, high } of ranged) {
const raw = low + sweepRunFraction(globalIndex, axisIndex) * (high - low);
const raw =
low + sweepRunFraction(seed, globalIndex, axisIndex) * (high - low);
parameterValues[axis.identifier] = String(
axis.integer ? Math.round(raw) : Number(raw.toPrecision(12)),
);
Expand Down Expand Up @@ -332,6 +335,7 @@ export function createSweepSession(
};

const runs = sweepRangeRuns(
seed,
axes,
selection,
snapshot.runsCompleted,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -21,8 +21,11 @@ precomputed for cameras nobody is looking through.
A **point** selection runs the experiment at that value. A **range**
selection runs one stochastic experiment over the ranges: each run carries
its own drawn value per ranged parameter (`sweepRangeRuns`), low-discrepancy
across the selected value interval and prefix-stable in the global run index
— the quantized positions bound the interval, they do not grid it. One
across the selected value interval, rotated by a seed-derived shift
(Cranley–Patterson, so draws are unbiased over the seed and each experiment
seed explores its own sequence) and prefix-stable in the seed, axis, and
global run index — the quantized positions bound the interval, they do not
grid it. One
selection is one experiment: full worker-pool parallelism, and the metric
stream is the experiment's own, so the distribution over the region streams
from the first frames and sharpens as runs land.
Expand Down
Loading