feat(abtest): hypothesis metadata for pre-registration (Change Set A)

.sampo/changesets/abtest-hypothesis.md

@@ -0,0 +1,5 @@
+---
+npm/@listmonk-ops/abtest: minor (Added)
+---
+
+Add hypothesis metadata for A/B test pre-registration: structured objective, primary metric, expected lift, owner, and experiment scope with canonical checksum locking. AbTest gains optional hypothesis and assignmentProvenance fields.

.sampo/changesets/abtest-stratification.md

@@ -0,0 +1,5 @@
+---
+npm/@listmonk-ops/abtest: minor (Added)
+---
+
+Add recipient-domain stratification for A/B test assignment: classify subscribers into provider strata and compute a constrained quota matrix where row sums match stratum sizes and column sums match exact variant/holdout counts. Includes the DEFAULT_STRATIFICATION_POLICY, normalizeDomain, classifyStratum, and a paired-swap computeStratifiedQuotas solver.

packages/abtest/README.md

@@ -661,3 +661,148 @@ assignment and chunked bulk list membership:
 ## License
 
 MIT License - see LICENSE file for details.
+
+## Hypothesis pre-registration (advanced experimentation)
+
+An A/B test can carry a **pre-registered hypothesis** that is locked
+(checksummed) before recipient assignment. After locking, the hypothesis
+cannot change without discarding the assignment manifest and provisioning.
+This prevents post-hoc hypothesis adjustment (p-hacking) and gives reports a
+stable reference.
+
+### Locking and verification
+
+```typescript
+import {
+  lockHypothesis,
+  verifyHypothesisChecksum,
+  validateHypothesisMetadata,
+} from "@listmonk-ops/abtest";
+
+const metadata = {
+  objective: "Increase click-through rate on the welcome email",
+  hypothesis: "A shorter subject line will increase CTR by 10%",
+  primaryMetric: { type: "click_rate", direction: "maximize" },
+  expectedLift: { kind: "relative", value: 0.1 },
+  owner: { id: "user-1", displayName: "Test User" },
+  experimentScope: {
+    channel: "email",
+    experimentFamilyKey: "onboarding.welcome.subject",
+    attributionWindowHours: 72,
+    exclusionWindowHours: 168,
+  },
+  createdAt: new Date().toISOString(),
+};
+
+validateHypothesisMetadata(metadata, true); // strict launch check
+const locked = lockHypothesis(metadata);
+verifyHypothesisChecksum(locked); // true
+```
+
+The checksum recursively canonicalizes nested fields
+(`primaryMetric`, `expectedLift`, `owner`, `experimentScope`, `createdAt`),
+so any change after locking invalidates it.
+
+### Wiring through creation
+
+Pass a `hypothesis` field to `createAbTest`. The service locks it before
+provisioning, so the assignment manifest is always bound to a frozen
+hypothesis:
+
+```typescript
+const test = await abTestExecutors.createAbTest({
+  name: "Subject Line Test",
+  variants: [/* ... */],
+  lists: [1, 2],
+  hypothesis: {
+    objective: "Increase CTR",
+    hypothesis: "Shorter subject lifts CTR",
+    primary_metric: { type: "click_rate", direction: "maximize" },
+    expected_lift: { kind: "relative", value: 0.1 },
+    owner: { id: "user-1" },
+    experiment_scope: {
+      channel: "email",
+      experiment_family_key: "onboarding.welcome",
+      attribution_window_hours: 72,
+      exclusion_window_hours: 168,
+    },
+  },
+});
+```
+
+### Validation rules
+
+- `createdAt` and `lockedAt` must be strict ISO 8601 timestamps (the year-zero
+  string `"0"`, localized formats like `"01/02/03"`, and overflowed dates like
+  `"2026-02-30"` are rejected).
+- `primaryMetric.type` ∈ `click_rate | conversion_rate | revenue_per_recipient`.
+- `primaryMetric.direction` ∈ `maximize | minimize`.
+- `expectedLift.kind` ∈ `relative | absolute`. Absolute lifts require a `unit`
+  ∈ `percentage_point | currency_per_recipient`.
+- Metric/unit coupling: `revenue_per_recipient` requires
+  `currency_per_recipient` absolute lift; `click_rate`/`conversion_rate`
+  require `percentage_point`. Relative lift is unit-agnostic.
+- `experimentScope.experimentFamilyKey` must be dotted alphanumeric segments
+  (`onboarding.activation.day1`); `.`, `foo.`, and `foo..bar` are rejected.
+
+### 가설 사전 등록 (Korean)
+
+A/B 테스트에 **사전 등록된 가설**을 설정하면 수신자 할당 전에 잠금(체크섬)
+처리됩니다. 잠금 후에는 할당 매니페스트와 프로비저닝을 폐기하지 않는 한
+가설을 변경할 수 없습니다. 이는 사후 가설 조정(p-hacking)을 방지하고
+보고서에 안정적인 기준점을 제공합니다.
+
+- `createAbTest`에 `hypothesis` 필드를 전달하면 서비스가 프로비저닝 전에
+  잠금 처리합니다.
+- 체크섬은 중첩 필드(`primaryMetric`, `expectedLift`, `owner`,
+  `experimentScope`, `createdAt`)를 재귀적으로 정규화하므로 잠금 후 어떤
+  변경도 무효화됩니다.
+- `createdAt`/`lockedAt`은 엄격한 ISO 8601이어야 합니다.
+- `experimentFamilyKey`는 점으로 구분된 영숫자 세그먼트여야 합니다.
+
+## Recipient-domain stratification (advanced experimentation)
+
+Stratification classifies subscribers by email-domain provider and computes a
+**constrained quota matrix** so each provider stratum gets a proportional share
+of every variant/holdout group. This prevents a single large provider (e.g.
+Gmail) from dominating one variant and skewing results.
+
+```typescript
+import {
+  classifyStratum,
+  computeStratifiedQuotas,
+  DEFAULT_STRATIFICATION_POLICY,
+} from "@listmonk-ops/abtest";
+
+const policy = { ...DEFAULT_STRATIFICATION_POLICY, enabled: true };
+const stratum = classifyStratum("user@gmail.com", policy); // "gmail"
+
+const result = computeStratifiedQuotas({
+  stratumSizes: { gmail: 600, naver: 300, other: 100 },
+  groupExactCounts: { "variant:A": 500, "variant:B": 500 },
+  groupOrder: ["variant:A", "variant:B"],
+  totalAudience: 1000,
+});
+```
+
+The solver uses the largest-remainder method per stratum row, then a paired-swap
+column correction that preserves row sums while matching exact group column
+counts. Configured domains in the provider map are normalized with the same
+rules applied to subscriber emails, so mixed-case entries like `"GMAIL.COM"`
+match correctly.
+
+During holdout provisioning, when a stratification policy is enabled and the
+resolved audience carries emails, the quota matrix is computed and stored on
+the `AbTest.stratification` field for reporting and validation.
+
+### 수신자 도메인 층화 (Korean)
+
+층화는 구독자를 이메일 도메인 제공자별로 분류하고, 각 제공자 층(stratum)이
+모든 변형/홀드아웃 그룹의 비례 배분을 받도록 **제약된 할당량 행렬**을
+계산합니다. 단일 대형 제공자(예: Gmail)가 하나의 변형을 독점하여 결과를
+왜곡하는 것을 방지합니다.
+
+- `classifyStratum`으로 구독자를 분류하고, `computeStratifiedQuotas`로
+  할당량 행렬을 계산합니다.
+- 홀드아웃 프로비저닝 시 층화 정책이 활성화되어 있으면 할당량 행렬이
+  `AbTest.stratification`에 저장되어 보고/검증에 사용됩니다.

packages/abtest/src/abtest-service.ts

@@ -11,6 +11,7 @@ import {
 	DEFAULT_STATISTICAL_POLICY,
 	fixedHorizonGate,
 } from "./statistics";
+import { lockHypothesis } from "./hypothesis";
 import type {
 	AbTest,
 	AbTestConfig,
@@ -178,6 +179,13 @@ export class AbTestService {
 			autoDeployWinner,
 			campaignMappings: [],
 			testListMappings: [],
+			// Lock the pre-registration hypothesis before any provisioning so the
+			// assignment manifest is bound to a frozen, checksummed hypothesis.
+			hypothesis: config.hypothesis
+				? config.hypothesis.lockedAt
+					? config.hypothesis
+					: lockHypothesis(config.hypothesis)
+				: undefined,
 		};
 
 		// Create Listmonk campaigns if integration is available
@@ -224,6 +232,14 @@ export class AbTestService {
 					abTest.assignmentSeed = segmentationResult.assignmentSeed;
 					abTest.audienceSnapshot = segmentationResult.audienceSnapshot;
 					abTest.assignmentManifest = segmentationResult.assignmentManifest;
+					// Record that recipients were assigned through a deterministic
+					// manifest, so consumers can distinguish it from legacy splits.
+					abTest.assignmentProvenance = "manifest_v1";
+					// Capture the stratified quota matrix when a stratification
+					// policy produced one, so reports can show per-provider shares.
+					if (segmentationResult.stratification) {
+						abTest.stratification = segmentationResult.stratification;
+					}
 				} else {
 					// Use full-split methodology (legacy)
 					testListMappings = await this.listmonkIntegration.segmentSubscribers(
@@ -238,6 +254,8 @@ export class AbTestService {
 						);
 					testGroupSize = totalSubscribers;
 					holdoutGroupSize = 0;
+					// Full-split provisioning predates deterministic manifests.
+					abTest.assignmentProvenance = "legacy_unavailable";
 				}
 				provisionedResources = {
 					...provisionedResources,

packages/abtest/src/audience.ts

@@ -30,6 +30,9 @@ export interface AudienceMember {
 	subscriberId: number;
 	/** Stable UUID used for identity, dedupe, checksum, and deterministic assignment. */
 	subscriberUuid: string;
+	/** Subscriber email, used for recipient-domain stratification. Optional
+	 * because legacy resolvers and test fixtures may not populate it. */
+	email?: string;
 }
 
 export interface AudienceSnapshot {
@@ -228,6 +231,7 @@ export function createListmonkAudienceResolver(
 							collected.push({
 								subscriberId: numericId,
 								subscriberUuid: uuid,
+								email: subscriber.email,
 							});
 						}
 					}

packages/abtest/src/basic.ts

@@ -48,6 +48,41 @@ export class CreateAbTestCommand {
 			ignoreStatisticalWarnings: input.ignore_sample_size_warnings || false,
 			durationHours: input.duration_hours,
 			launchAt: input.launch_at,
+			hypothesis: input.hypothesis
+				? {
+						objective: input.hypothesis.objective,
+						hypothesis: input.hypothesis.hypothesis,
+						primaryMetric: {
+							type: input.hypothesis.primary_metric.type,
+							direction: input.hypothesis.primary_metric.direction,
+						},
+						expectedLift:
+							input.hypothesis.expected_lift.kind === "relative"
+								? {
+										kind: "relative",
+										value: input.hypothesis.expected_lift.value,
+									}
+								: {
+										kind: "absolute",
+										value: input.hypothesis.expected_lift.value,
+										unit: input.hypothesis.expected_lift.unit,
+									},
+						owner: {
+							id: input.hypothesis.owner.id,
+							displayName: input.hypothesis.owner.display_name,
+						},
+						experimentScope: {
+							channel: input.hypothesis.experiment_scope.channel,
+							experimentFamilyKey:
+								input.hypothesis.experiment_scope.experiment_family_key,
+							attributionWindowHours:
+								input.hypothesis.experiment_scope.attribution_window_hours,
+							exclusionWindowHours:
+								input.hypothesis.experiment_scope.exclusion_window_hours,
+						},
+						createdAt: new Date().toISOString(),
+					}
+				: undefined,
 		};
 
 		return await this.abTestService.createTest(config);