Skip to content

[Internal] Complete Age Signals telemetry and release validation - #16377

Open
JorgeMucientes wants to merge 4 commits into
issue/woomob-3767-age-signals-sdk-004from
issue/woomob-3768-age-signals-telemetry
Open

[Internal] Complete Age Signals telemetry and release validation#16377
JorgeMucientes wants to merge 4 commits into
issue/woomob-3767-age-signals-sdk-004from
issue/woomob-3768-age-signals-telemetry

Conversation

@JorgeMucientes

@JorgeMucientes JorgeMucientes commented Aug 6, 2026

Copy link
Copy Markdown
Contributor

Fixes WOOMOB-3768

⚠️ Do not merge. I'll merge all 4 stacked PRs together as a final working solution.

Description

Completes the Age Signals 0.0.4 migration with privacy-bounded telemetry, legacy cleanup, release notes, and an executable release-validation checklist.

  • Extends account_age_restriction_checked with bounded request stage, access status, age band, significant-change status, final decision, restriction reason, SDK error, retry count, and recovery properties.
  • Adds account_age_verification_action for opening Play, manual retry, and return-from-Play retry; retry actions are emitted only after a check is admitted.
  • Uses evaluator-produced range outcomes and sends only below_13, 13_15, 16_17, 18_plus, eligible, or ambiguous categories—never raw bounds, install ID, approval date, exception text, or user IDs.
  • Keeps the legacy Boolean read-only at the app boundary and clears it through a narrow migration operation when an authoritative eligible result clears the typed restriction.
  • Removes the final v0.0.3 supervised-approval restriction compatibility value.
  • Adds a 25.4 release note and docs/age-signals-sdk-004-qa.md with policy, Play Console, Play-installed QA, accessibility, privacy, rollback, monitoring, and evidence gates.
  • Adds exact-payload tests, prohibited-field assertions, canonical and nonstandard eligible ranges, every significant-change state, actions, errors, recovery context, retry contention and cancellation, and recreated-checker coverage for durable migration clearing.

Automated validation passed on the exact rewritten stack: the complete age-eligibility unit-test package and repository-wide detektAll. GitHub CI will separately rerun its full build and checks on the pushed commits.

Stack: #16374#16375#16376 ← this PR. Merge bottom-up only after #16376 is approved.

Test Steps

The manual-test patch replaces the production manager provided by Hilt with Google's SDK 0.0.4 FakeAgeSignalsManager. It is based on commit c3dee55541c169268b7e332b8f1f19f0ef8ddf80, is only for local review, and must not be committed.

  1. Start from a clean checkout of this PR, then download and apply the patch:

    patch_file=/tmp/pr-16377-fake-age-signals.patch
    gh api repos/woocommerce/woocommerce-android/issues/comments/5344468672 --jq .body > "$patch_file"
    git apply --check "$patch_file"
    git apply "$patch_file"
  2. Modify the value for ACTIVE_MANUAL_AGE_SIGNALS_SCENARIO in ManualAgeSignalsTestScenario.kt with the below values then install the Wasabi debug build, and cold-start both logged-in and logged-out flows:

    ./gradlew :WooCommerce:installWasabiDebug
  3. Verify the final decision and account_age_restriction_checked payload for each group:

    Scenarios Expected decision and bounded telemetry
    NOT_SHARED Allow unless already restricted; access status not_shared; no age band and no second-stage result.
    VERIFICATION_REQUIRED Keep the recoverable gate; access status verification_required; final decision verification_required; no logout.
    VERIFICATION_REQUIRED_THEN_SHARED_18_PLUS Track open_play_store and return_from_play_retry; recovery check ends allowed with age band 18_plus and dismisses the dialog. Use Retry separately to verify manual_retry.
    SHARED_BELOW_13 or SHARED_MISSING_LOWER_BELOW_13 Restrict with reason below_minimum_age, range outcome below_13, and log out.
    SHARED_13_TO_15, SHARED_16_TO_17, SHARED_18_PLUS Allow with range outcome 13_15, 16_17, or 18_plus, respectively.
    SHARED_13_PLUS_OPEN_ENDED Allow authoritatively with range outcome eligible; do not report it as ambiguous.
    SHARED_MISSING_BOUNDS, SHARED_CROSSING_BOUNDS Non-authoritative result with age band ambiguous; preserve a prior restriction or otherwise allow.
    SIGNIFICANT_CHANGE_PENDING, SIGNIFICANT_CHANGE_APPROVED, SIGNIFICANT_CHANGE_DECLINED Emit only the bounded significant-change category; do not block the whole app.
    UNEXPECTED_ACCESS_STATUS Emit the unexpected/unspecified bounded access category and preserve a prior restriction or otherwise allow.
    Access/check network errors Terminal result with the bounded SDK error code; preserve a prior restriction or otherwise allow.
    Access/check transient errors Three total attempts across the request, retry only the failing stage, then preserve a prior restriction or otherwise allow.
  4. Inspect the emitted events with the normal Tracks debugging workflow. Confirm the payload contains only the documented categorical properties and never raw age bounds, installId, approval date, exception text, or exact user identifiers.

  5. To verify persistence, first run SHARED_BELOW_13; then switch to NOT_SHARED, an ambiguous range, and an error without clearing app data. The restriction must remain. Switch to SHARED_18_PLUS; it must clear.

  6. Disable AGE_ELIGIBILITY_CHECKS and confirm the check is bypassed without erasing the persisted restriction. Re-enable it before continuing.

  7. Restore the scenario to the patch default (VERIFICATION_REQUIRED_THEN_SHARED_18_PLUS), then remove the manual patch:

    git apply --check --reverse "$patch_file"
    git apply --reverse "$patch_file"
    git status --short
  8. Separately complete docs/age-signals-sdk-004-qa.md with a Play-installed internal-testing build on a Play-certified Android 6.0+ device. The fake patch does not validate Play ownership, account configuration, or the real Play UI. Attach that evidence to WOOMOB-3726 and resolve the policy/Play Console gates before rollout.

The scenarios use the access, response, significant-change, and exception setters documented in Google's Age Signals testing guide. The verification-to-shared scenario composes two fake managers so recovery and its analytics can be verified in one app run.

Images/gif

No new UI in this layer; #16375 contains the verification screen and preview coverage.

  • I have considered if this change warrants release notes and added a 25.4 entry plus a release-validation checklist.

@JorgeMucientes JorgeMucientes added this to the 25.4 milestone Aug 6, 2026
@JorgeMucientes JorgeMucientes added type: task An internally driven task. status: do not merge Dependent on another PR, ready for review but not ready for merge. category: tracks Related to analytics, including Tracks Events. priority: high Affects lots of customers substantially, but not critically. category: unit tests Related to unit testing. status: feature-flagged Behind a feature flag. Milestone is not strongly held. org-team: Kiwi Kiwi team issues labels Aug 6, 2026
@dangermattic

dangermattic commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator
1 Error
🚫 This PR is tagged with status: do not merge label(s).
1 Warning
⚠️ This PR is larger than 300 lines of changes. Please consider splitting it into smaller PRs for easier and faster reviews.
1 Message
📖

This PR contains changes to Tracks-related logic. Please ensure (author and reviewer) the following are completed:

  • The tracks events must be validated in the Tracks system.
  • Verify the internal Tracks spreadsheet has also been updated.
  • Please consider registering any new events.
  • The PR must be assigned the category: tracks label.

Generated by 🚫 Danger

@wpmobilebot

wpmobilebot commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

App Icon📲 You can test the changes from this Pull Request in WooCommerce Android by scanning the QR code below to install the corresponding build.

App NameWooCommerce Android
Platform📱 Mobile
FlavorJalapeno
Build TypeDebug
Build Number776
Version25.3-rc-1
Application IDcom.woocommerce.android.prealpha
Commitc3dee55
Installation URL0iao4u6a7ap8o
Automatticians: You can use our internal self-serve MC tool to give yourself access to those builds if needed.

@wpmobilebot wpmobilebot modified the milestones: 25.4, 25.5 Aug 7, 2026
@wpmobilebot

Copy link
Copy Markdown
Collaborator

Version 25.4 has now entered code-freeze, so the milestone of this PR has been updated to 25.5.

@JorgeMucientes
JorgeMucientes force-pushed the issue/woomob-3767-age-signals-sdk-004 branch from 3d0aacf to 8e9c3ce Compare August 19, 2026 10:50
@JorgeMucientes
JorgeMucientes force-pushed the issue/woomob-3768-age-signals-telemetry branch 2 times, most recently from 38e1a66 to cdb5be6 Compare August 19, 2026 11:12
@JorgeMucientes
JorgeMucientes force-pushed the issue/woomob-3767-age-signals-sdk-004 branch from 8e9c3ce to 722ba97 Compare August 19, 2026 11:12
@JorgeMucientes JorgeMucientes modified the milestones: 25.5, 25.6 Aug 19, 2026
@JorgeMucientes
JorgeMucientes marked this pull request as ready for review August 19, 2026 15:12

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

AI Code Review - No issues found. The changes look good.

The durability fix is sound: userAgeRestrictionReason is now the authoritative source in readPersistedRestriction, and clearLegacyAgeRestriction() resets the legacy isUserAgeEligibleForAppUse flag (default true) so a stale legacy false can no longer re-trigger LEGACY_RESTRICTION_UNKNOWN_REASON after an authoritative allowed result. Making the wrapper property read-only is safe — the checker was its only writer. The removed SUPERVISED_APPROVAL_DENIED enum value has no remaining references. The new AgeSignalsAnalyticsTracker keeps all reported values categorical, and the tests explicitly assert raw bounds/IDs/exception data are never emitted.

PR housekeeping
  • Applied repo AGENTS.md/CLAUDE.md guidance (store-app analytics + testing conventions) while reviewing.
  • The added docs/age-signals-sdk-004-qa.md provides a detailed release-validation checklist covering the behaviour-changing paths.

Automatic review · claude-opus-4-8 · Workflow run

How to reply to a finding

Reply on this review (or inline at the line the finding refers to) with one of:

  • @claude addressed - I made the change. Bot verifies against the next diff before marking resolved.
  • @claude rejected: <reason> - Will not fix; reason gets quoted on the next review.
  • @claude not-applicable - Finding does not apply (wrong file, already covered elsewhere, etc.).

The bot honours these on the next review pass.

@JorgeMucientes

JorgeMucientes commented Aug 19, 2026

Copy link
Copy Markdown
Contributor Author

diff --git a/WooCommerce/src/main/kotlin/com/woocommerce/android/ui/ageeligibility/AgeSignalsModule.kt b/WooCommerce/src/main/kotlin/com/woocommerce/android/ui/ageeligibility/AgeSignalsModule.kt
index e014c01037b..1bd02860860 100644
--- a/WooCommerce/src/main/kotlin/com/woocommerce/android/ui/ageeligibility/AgeSignalsModule.kt
+++ b/WooCommerce/src/main/kotlin/com/woocommerce/android/ui/ageeligibility/AgeSignalsModule.kt
@@ -2,7 +2,6 @@ package com.woocommerce.android.ui.ageeligibility

import android.content.Context
import com.google.android.play.agesignals.AgeSignalsManager
-import com.google.android.play.agesignals.AgeSignalsManagerFactory
import dagger.Binds
import dagger.Module
import dagger.Provides
@@ -22,6 +21,6 @@ abstract class AgeSignalsModule {
@provides
@singleton
fun provideAgeSignalsManager(@ApplicationContext context: Context): AgeSignalsManager =

  •        AgeSignalsManagerFactory.create(context)
    
  •        createManualAgeSignalsManager()
    
    }
    }
    diff --git a/WooCommerce/src/main/kotlin/com/woocommerce/android/ui/ageeligibility/ManualAgeSignalsTestScenario.kt b/WooCommerce/src/main/kotlin/com/woocommerce/android/ui/ageeligibility/ManualAgeSignalsTestScenario.kt
    new file mode 100644
    index 00000000000..d2b5a504b46
    --- /dev/null
    +++ b/WooCommerce/src/main/kotlin/com/woocommerce/android/ui/ageeligibility/ManualAgeSignalsTestScenario.kt
    @@ -0,0 +1,175 @@
    +package com.woocommerce.android.ui.ageeligibility

+import com.google.android.gms.tasks.Task
+import com.google.android.play.agesignals.AgeSignalsAccessRequest
+import com.google.android.play.agesignals.AgeSignalsAccessResult
+import com.google.android.play.agesignals.AgeSignalsException
+import com.google.android.play.agesignals.AgeSignalsManager
+import com.google.android.play.agesignals.AgeSignalsRequest
+import com.google.android.play.agesignals.AgeSignalsResult
+import com.google.android.play.agesignals.model.AgeRangeSource
+import com.google.android.play.agesignals.model.AgeSignalsErrorCode
+import com.google.android.play.agesignals.model.AgeSignalsStatus
+import com.google.android.play.agesignals.model.SignificantChangeStatus
+import com.google.android.play.agesignals.testing.FakeAgeSignalsManager
+import java.util.Date
+import java.util.concurrent.atomic.AtomicInteger
+
+internal val ACTIVE_MANUAL_AGE_SIGNALS_SCENARIO =

  • ManualAgeSignalsTestScenario.VERIFICATION_REQUIRED_THEN_SHARED_18_PLUS

+internal enum class ManualAgeSignalsTestScenario {

  • NOT_SHARED,
  • VERIFICATION_REQUIRED,
  • VERIFICATION_REQUIRED_THEN_SHARED_18_PLUS,
  • SHARED_BELOW_13,
  • SHARED_MISSING_LOWER_BELOW_13,
  • SHARED_13_TO_15,
  • SHARED_16_TO_17,
  • SHARED_18_PLUS,
  • SHARED_13_PLUS_OPEN_ENDED,
  • SHARED_MISSING_BOUNDS,
  • SHARED_CROSSING_BOUNDS,
  • SIGNIFICANT_CHANGE_PENDING,
  • SIGNIFICANT_CHANGE_APPROVED,
  • SIGNIFICANT_CHANGE_DECLINED,
  • UNEXPECTED_ACCESS_STATUS,
  • ACCESS_NETWORK_ERROR,
  • ACCESS_TRANSIENT_ERROR,
  • CHECK_NETWORK_ERROR,
  • CHECK_TRANSIENT_ERROR
    +}

+internal fun createManualAgeSignalsManager(): AgeSignalsManager = when (ACTIVE_MANUAL_AGE_SIGNALS_SCENARIO) {

  • ManualAgeSignalsTestScenario.NOT_SHARED -> fakeManager(AgeSignalsStatus.NOT_SHARED)
  • ManualAgeSignalsTestScenario.VERIFICATION_REQUIRED -> fakeManager(AgeSignalsStatus.VERIFICATION_REQUIRED)
  • ManualAgeSignalsTestScenario.VERIFICATION_REQUIRED_THEN_SHARED_18_PLUS ->
  •    VerificationThenSharedAgeSignalsManager()
    
  • ManualAgeSignalsTestScenario.SHARED_BELOW_13 -> fakeManager(
  •    ageSignalsResult = result(ageLower = 0, ageUpper = 12)
    
  • )
  • ManualAgeSignalsTestScenario.SHARED_MISSING_LOWER_BELOW_13 -> fakeManager(
  •    ageSignalsResult = result(ageUpper = 12)
    
  • )
  • ManualAgeSignalsTestScenario.SHARED_13_TO_15 -> fakeManager(
  •    ageSignalsResult = result(ageLower = 13, ageUpper = 15)
    
  • )
  • ManualAgeSignalsTestScenario.SHARED_16_TO_17 -> fakeManager(
  •    ageSignalsResult = result(ageLower = 16, ageUpper = 17)
    
  • )
  • ManualAgeSignalsTestScenario.SHARED_18_PLUS -> fakeManager(
  •    ageSignalsResult = result(ageLower = 18, source = AgeRangeSource.TIER_D)
    
  • )
  • ManualAgeSignalsTestScenario.SHARED_13_PLUS_OPEN_ENDED -> fakeManager(
  •    ageSignalsResult = result(ageLower = 13)
    
  • )
  • ManualAgeSignalsTestScenario.SHARED_MISSING_BOUNDS -> fakeManager(
  •    ageSignalsResult = result(source = AgeRangeSource.TIER_A)
    
  • )
  • ManualAgeSignalsTestScenario.SHARED_CROSSING_BOUNDS -> fakeManager(
  •    ageSignalsResult = result(ageLower = 16, ageUpper = 15)
    
  • )
  • ManualAgeSignalsTestScenario.SIGNIFICANT_CHANGE_PENDING -> fakeManager(
  •    ageSignalsResult = result(
    
  •        ageLower = 13,
    
  •        ageUpper = 15,
    
  •        significantChangeStatus = SignificantChangeStatus.PENDING
    
  •    )
    
  • )
  • ManualAgeSignalsTestScenario.SIGNIFICANT_CHANGE_APPROVED -> fakeManager(
  •    ageSignalsResult = result(
    
  •        ageLower = 13,
    
  •        ageUpper = 15,
    
  •        significantChangeStatus = SignificantChangeStatus.APPROVED,
    
  •        approvalDate = Date(1777939200000L)
    
  •    )
    
  • )
  • ManualAgeSignalsTestScenario.SIGNIFICANT_CHANGE_DECLINED -> fakeManager(
  •    ageSignalsResult = result(
    
  •        ageLower = 13,
    
  •        ageUpper = 15,
    
  •        significantChangeStatus = SignificantChangeStatus.DECLINED
    
  •    )
    
  • )
  • ManualAgeSignalsTestScenario.UNEXPECTED_ACCESS_STATUS -> fakeManager(Int.MAX_VALUE)
  • ManualAgeSignalsTestScenario.ACCESS_NETWORK_ERROR -> fakeManager(
  •    accessError = AgeSignalsErrorCode.NETWORK_ERROR
    
  • )
  • ManualAgeSignalsTestScenario.ACCESS_TRANSIENT_ERROR -> fakeManager(
  •    accessError = AgeSignalsErrorCode.CLIENT_TRANSIENT_ERROR
    
  • )
  • ManualAgeSignalsTestScenario.CHECK_NETWORK_ERROR -> fakeManager(
  •    checkError = AgeSignalsErrorCode.NETWORK_ERROR
    
  • )
  • ManualAgeSignalsTestScenario.CHECK_TRANSIENT_ERROR -> fakeManager(
  •    checkError = AgeSignalsErrorCode.CLIENT_TRANSIENT_ERROR
    
  • )
    +}

+private fun fakeManager(

  • accessStatus: Int = AgeSignalsStatus.SHARED,
  • ageSignalsResult: AgeSignalsResult? = null,
  • accessError: Int? = null,
  • checkError: Int? = null
    +) = FakeAgeSignalsManager().apply {
  • if (accessError == null) {
  •    setNextAgeSignalsAccessResult(
    
  •        AgeSignalsAccessResult.builder().setAgeSignalsStatus(accessStatus).build()
    
  •    )
    
  • } else {
  •    setNextRequestAgeSignalsAccessException(AgeSignalsException(accessError))
    
  • }
  • when {
  •    checkError != null -> setNextAgeSignalsException(AgeSignalsException(checkError))
    
  •    ageSignalsResult != null -> setNextAgeSignalsResult(ageSignalsResult)
    
  • }
    +}

+private fun result(

  • ageLower: Int? = null,
  • ageUpper: Int? = null,
  • source: Int = AgeRangeSource.TIER_B,
  • significantChangeStatus: Int? = null,
  • approvalDate: Date? = null
    +): AgeSignalsResult {
  • val builder = AgeSignalsResult.builder().setAgeRangeSource(source)
  • ageLower?.let(builder::setAgeLower)
  • ageUpper?.let(builder::setAgeUpper)
  • significantChangeStatus?.let(builder::setSignificantChangeStatus)
  • approvalDate?.let(builder::setSignificantChangeApprovalDate)
  • return builder.build()
    +}

+private class VerificationThenSharedAgeSignalsManager : AgeSignalsManager {

  • private val accessRequestCount = AtomicInteger()
  • private val verificationRequired = fakeManager(AgeSignalsStatus.VERIFICATION_REQUIRED)
  • private val sharedAdult = fakeManager(
  •    ageSignalsResult = result(ageLower = 18, source = AgeRangeSource.TIER_D)
    
  • )
  • override fun requestAgeSignalsAccess(request: AgeSignalsAccessRequest): Task =
  •    if (accessRequestCount.getAndIncrement() == 0) {
    
  •        verificationRequired.requestAgeSignalsAccess(request)
    
  •    } else {
    
  •        sharedAdult.requestAgeSignalsAccess(request)
    
  •    }
    
  • override fun checkAgeSignals(request: AgeSignalsRequest): Task =
  •    sharedAdult.checkAgeSignals(request)
    

+}

@JorgeMucientes
JorgeMucientes force-pushed the issue/woomob-3767-age-signals-sdk-004 branch from 722ba97 to eaf575b Compare August 19, 2026 15:52
@JorgeMucientes
JorgeMucientes force-pushed the issue/woomob-3768-age-signals-telemetry branch from cdb5be6 to c3dee55 Compare August 19, 2026 15:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

category: tracks Related to analytics, including Tracks Events. category: unit tests Related to unit testing. org-team: Kiwi Kiwi team issues priority: high Affects lots of customers substantially, but not critically. status: do not merge Dependent on another PR, ready for review but not ready for merge. status: feature-flagged Behind a feature flag. Milestone is not strongly held. type: task An internally driven task.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants