!For emergencies

CSO

What Is a CSO?

A Critical Site Outage (CSO) is when a production issue is severely impacting users on adobe.com. The goal is to revert the offending change as fast as possible while keeping stage and main in sync.

Complete CSO Process — Step by Step

Step 1 — Identify the Offending PR

• Check the latest "Stage to Main" release PR on GitHub.
• Identify which PR in that batch caused the issue.
• Use logs, RUM data, or Splunk dashboards to confirm root cause.

Step 2 — Open a Blocker Jira Ticket

• Create a Blocker priority Jira ticket.
• Include: description of the outage, affected pages/features, and the suspected offending PR.

Step 3 — Raise Two PRs (Dual-PR Strategy)

Rule: You must raise two separate PRs — one targeting stage, one targeting main. They must contain identical changes.

• PR (A) → stage:
• Use the GitHub "Revert" button on the offending PR (creates a clean revert PR).
• If the revert button isn't available, manually create the revert branch off stage.
• PR (B) → main:
• Create a new branch off main.
• Cherry-pick only the revert commit from the stage revert PR.
• Do not merge the entire stage branch — this avoids dragging in untested stage code.

Step 4 — Get Approvals

PR	Required approvals	Notes
Stage PR (A)	2 approvals	At least 1 code owner recommended
Main PR (B)	3 approvals	At least 1 code owner required; only Admins can merge

Step 5 — Notify in #milo-dev
Post in #milo-dev tagging:

• @milo-eng (Milo core team)
• Milo Admins who can merge to main

Include:

• Short CSO description
• Jira CSO ticket link
• Links to both PRs (stage + main)
• Ask admins to be ready to merge main once approvals and QA sign-off are in

Step 6 — Submit the Slack Emergency Workflow (Mandatory)
Submit the "Milo Critical and Exceptional Releases" Slack workflow (see Slack Emergency Workflow for full details). This is a required tracking step.

You can find the Workflow inside of the #milo-dev channel under workflow tabs

• QA must respond in the workflow thread and sign off that stage is OK before the Milo team merges to main.

Step 7 — Merge Sequence

1. Merge PR (A) to stage using squash merge.
2. Validate on stage — QA and impacted teams confirm the fix works.
3. Milo Admin merges PR (B) to main.
4. Verify on production — impacted team, SOTs, and/or original reporter confirm.

CSO During an RCP Window

The process is the same as above, with one addition:

• VP approval is required. Fill in the VP approval field in the Slack workflow.
• Outside of RCP, VP approval is not required — admin + SOT + QA sign-off is sufficient.

---

Slack Emergency Workflow

"Milo Critical and Exceptional Releases" Workflow

This is a Slack Workflow Builder form used to track any Milo code deployment that happens outside the normal automated proccess. This includes:

• CSO/ emergency fixes
• RCP window exceptions
• Scheduled releases that can't go through normal automation

How to trigger It

1. Go to #milo-dev (or related Milo channels where the shortcut is available).
2. find the "Milo Critical and Exceptional Releases" workflow inside of thge "Workflow" tab.
3. Start the Workflow.
4. Fill in the form.

What Happens After Submission

1. The workflow posts a message in #milo-dev with the request summery and PR/Jira links.
2. @milo-eng (Milo-core Team) is automatically notified.
3. QA must respond in the workflow thread and sing off that stage is OK.
4. Once QA sings off, a Milo Admin performs the merge to main.

Key Points

• This workflow is mandatory for all CSO/emergency merges - it is the official tracking mechanism.
• It does not auto-trigger on label changes or pipeline events - it is always manually submitted by the requester.