From 3dc7b7d9df72faf3ab78e731bfb513ca7fc560f9 Mon Sep 17 00:00:00 2001 From: macedo Date: Sat, 19 Sep 2026 09:46:36 +0100 Subject: [PATCH 1/5] feat(skills): simplify research partner workflow --- README.md | 26 +++- TESTING.md | 17 ++- skills/_shared/research-partner-workflow.md | 114 +++++++++++++++ skills/collect-company-context/SKILL.md | 76 +++++++--- .../company-context-template.md | 57 -------- skills/daily-opportunities/SKILL.md | 137 +++++++++++------- skills/daily-outreach/SKILL.md | 105 ++++++++++---- 7 files changed, 359 insertions(+), 173 deletions(-) create mode 100644 skills/_shared/research-partner-workflow.md delete mode 100644 skills/collect-company-context/company-context-template.md diff --git a/README.md b/README.md index c10e42a..56fd213 100644 --- a/README.md +++ b/README.md @@ -21,11 +21,11 @@ Requires a Pro, Max, Team, or Enterprise plan with code execution enabled (Setti 1. Open **Customize → Plugins** and click **+ → Add marketplace**. 2. Sync from this GitHub repo: `noticedso/skills`. -3. From the synced marketplace, install **Networking Assistant**. All seven skills land in one shot. +3. From the synced marketplace, install **Networking Assistant**. All skills land in one shot. ## install (ChatGPT / Codex) -Add `noticedso/skills` as a plugin marketplace, then install **Networking Assistant**. The plugin keeps all seven capabilities together; do not upload them one by one. +Add `noticedso/skills` as a plugin marketplace, then install **Networking Assistant**. The plugin keeps the capabilities together; do not upload them one by one. ## the skills @@ -38,7 +38,9 @@ Add `noticedso/skills` as a plugin marketplace, then install **Networking Assist | **search-network** | Natural-language search across your network; returns a tight table; drills into anyone. | read-only | | **match-identities** | Triage the cross-source identity matches noticed proposes — confirm real matches, clear false positives, merge duplicates — and fill profile gaps by searching your network first. | yes | | **intro** | Find the warm path to a target, draft an intro between two people, write an intro request with a forwardable blurb, or write a backchannel vouch — all copy/paste. | only on confirmed intro tracking | -| **collect-company-context** | Gather onboarding context, propose goal-aligned target profiles and fill the Notion Company context template. | company context and Library updates | +| **collect-company-context** | Gather and verify the seven-section Company Brain while preserving sources in its Library. | Company Brain and Library updates | +| **daily-opportunities** | Prepare 25-person lists from observable segment rules, learn from Yes/No/Not sure reviews, and refine the next batch. | authorized lists and dashboard updates | +| **daily-outreach** | Develop attributed outreach for an approved person and reconcile confirmed sends, replies and meetings. | authorized drafts and dashboard updates; never sends automatically | ## how they fit together @@ -51,6 +53,17 @@ add-person → event-debrief `research-person` and `follow-up` support any contact at any time. `search-network` is the read path over everything you've captured. `intro` turns that network into warm paths and the messages that open them. +The research-partner loop is: + +``` +Company Brain → Goal → Segment rules → List → Review → Learning → Outreach +``` + +`collect-company-context` maintains the Company Brain. `daily-opportunities` +creates and refines reviewed lists. `daily-outreach` turns explicit Yeses into +attributed drafts and records confirmed outcomes. The routines share one workflow +contract so the three skills use the same schema and definitions. + ## shared conventions All skills follow the same rules, so they read as one system: @@ -90,4 +103,9 @@ reinstalling the plugin. ## Manual opportunity and outreach routines -`daily-opportunities` prepares or refines a fixed 25-person review list for one selected target profile. `daily-outreach` turns current human Yeses into Notion drafts and reconciles confirmed sends, replies and bookings. Run one requested step through Codex with noticed MCP and Notion. These skills do not schedule work or send messages. Customer context and iteration history remain in the private customer workspace. +`daily-opportunities` prepares or refines a 25-person review list for one +selected segment. `daily-outreach` turns current human Yeses into attributed +Notion drafts and reconciles confirmed sends, replies and bookings. Run one +requested step through Codex with noticed MCP and Notion. These skills do not +schedule work or send messages. Company facts, rules, lists and results remain in +the partner's private workspace. diff --git a/TESTING.md b/TESTING.md index 0956975..10b2b5e 100644 --- a/TESTING.md +++ b/TESTING.md @@ -63,18 +63,21 @@ Use synthetic company context and fixture-only tools; inspect proposed operation | Scenario | Expected behavior | |---|---| | Onboard a new research partner from a supplied directory with its own default template. Repeat with a matching partner already present. | Create the canonical record from the discovered template only when absent; otherwise reuse it. Save context there, never on a standalone staging page. | -| Save research and proposed profiles before the user validates the reference. | Saved content remains visibly draft/unvalidated; Last reviewed changes only after explicit reference validation. Profile selection alone is insufficient. | -| Request strategic investor/industry connectors in a city while the connector definition is absent, empty, a placeholder or substantive but unagreed. | Return to company-context/profile definition using the request as input; do not create an iteration, hypothesis or candidate list yet. | -| Run opportunities for a saved, explicitly agreed investor profile while another track is undefined and unrelated company context remains draft. | Proceed within the agreed profile and engagement goal without demanding full reference validation or treating investors as sales prospects. | -| Choose investors and partners for the direct engagement tracks of a company whose customers are hotels. Rename the context sections. | Preserve hotels as the stable customer ICP separately from both engagement tracks; follow sections by meaning without forcing customers into ICP #1. | -| Refine a hypothesis after a reviewed batch within an agreed track. | Use recorded learning within that track; preserve the existing batch, review-readiness and recovery safeguards. | -| Draft outreach for approved relationships in customer, investor, partner and hiring tracks, with each goal and track saved in context. | Inherit the relevant goal and track rather than treating every recipient as a buyer. Use the offer where relevant, preserve human approval and drafts, and leave sending to the user. | +| Save company research with two facts still needing confirmation. | Use the exact seven Company Brain headings, keep sources in Library, and mark only the unconfirmed facts red. Do not add target-profile prose to the Brain. | +| The company has buyer and investor goals with different names across old content. | Reuse one dashboard, choose one exact name for each goal, and use it consistently in Progress, Segments and Lists. Never combine their metrics. | +| Prepare a first list for a broad investor segment with Required, Prefer and Exclude rules. | Evaluate only public/enrichment signals, create one 25-person batch, configure the exact “This looks like a good fit…” review statement, and preserve the rules and evidence. | +| A list has 10 Yes, 12 No and 3 Not sure reviews with useful reasons. | Record the result, add one learning-log row, make the smallest supported rule refinement, and preserve the historic list and verdicts. | +| A list has only 10/25 reviews, including three Yeses. | Keep the list open for 15 reviews and allow those three approved people to move to Outreach without replacing the batch. | +| Create outreach for an approved person found in two lists for the same goal. | Reuse one Outreach entry, set the list that motivated the message as Source list, and keep other evidence in History & source. | +| A draft exists but no send is confirmed; later the user confirms a LinkedIn message was sent. | Keep To contact and Sent on empty before confirmation; then record Contacted, the channel and actual send date. Do not add follow-up-management fields. | +| A connector replies and offers an introduction, but the target has not replied and no meeting is accepted. | Preserve the evidence without counting a target reply or booked meeting. Attribute the activity through Source list → Goal. | ## what to watch for The behaviors the NYTW revisions were built to get right — confirm they hold across the tests above: -- **Capture skills preview before saving.** Company-context drafts may be saved incrementally; saving is not validation. +- **Company facts stay separate from targeting.** The Brain uses the seven canonical headings; observable rules and learning stay in Segments and Lists. +- **Outreach attribution is complete.** Every Outreach entry has a Source list, which resolves its Segment and Goal. - **No bare-name guessing.** A name with no URL/handle triggers a question, not a silent new record. - **The readback always appears** after a save, and reads like a person talking, not a field dump. - **Provenance tags never leak into chat** — you should never see `[from user]` or `[research, unverified]` in a message, only in the stored note. diff --git a/skills/_shared/research-partner-workflow.md b/skills/_shared/research-partner-workflow.md new file mode 100644 index 0000000..e4c5e92 --- /dev/null +++ b/skills/_shared/research-partner-workflow.md @@ -0,0 +1,114 @@ +# Research-partner workflow + +Use one canonical dashboard per company. A company may have one or more goals; +each goal has its own segments, lists, outreach, and metrics. Use the same exact +goal name everywhere and never aggregate unrelated funnels. + +The operating loop is: + +**Goal → Segment rules → List → Review → Learning → Refinement → Outreach** + +## Dashboard + +Keep these sections in this order: + +1. **Progress** — goal statements and per-goal Meetings, Outreach, and + Opportunities metrics; Network remains a company-level snapshot. +2. **Next** — a short to-do list naming the goal, owner, and concrete action. +3. **Segments** — the current observable targeting rules. +4. **Lists** — every generated batch and its result. +5. **Outreach** — approved people, drafts, and confirmed outcomes. +6. **Company** — link to the Company Brain. + +Unknown values display as “—”. Update the last-measurement date only for verified +data. Keep goal metrics separate. + +## Segments + +Database properties: **Segment** (title), **Goal**, **Priority**. + +The segment name is the short plain-language definition of who to find. There is +no separate “Who” field and no separate hypothesis page. A connector is simply a +segment with connector-specific rules. + +Every segment page uses a criteria table with **Category**, **Required**, +**Prefer**, and **Exclude**. Keep all categories present for consistency, leaving +unused cells empty: + +- Geography +- Sector / industry +- Company size +- Role +- Business model +- Offering / product +- Funding +- Hiring +- Headcount growth +- Market expansion +- Technology used +- Revenue +- Company age + +Write comma-separated tags or short observable rules, not profile prose. Required +means every populated category must match; comma-separated alternatives within a +cell mean “or.” Prefer improves ranking but is optional. Exclude disqualifies. +Rules must be testable through public search or data enrichment. Facts discoverable +only through contact belong in a collapsed contact-only note, not in targeting. + +Below the criteria, use **Learnings & refinements** with columns **Date**, +**Batch**, **Focus**, **Results · Yes / No / ?**, **Learned**, and +**Next refinement**. + +## Lists + +Database properties: **List** (title), **Goal**, **List link**, **Segment** +(relation), **Generated**, **Review**, **Results**. + +One row represents one generated batch. Preserve every batch so progress and rule +changes remain traceable. The list review tests the statement: **“This looks like +a good fit. We should reach out to them.”** Use explicit Yes, No, and Not sure. + +## Outreach + +Database properties: **Person** (title), **Source list** (relation to Lists), +**Status**, **Sender**, **Channel**, **Sent on**. + +Source list is required attribution. Its List row already supplies Segment and +Goal, so do not duplicate those properties in Outreach. + +The entry body contains **Message draft**, **Why reach out**, +**Introduction through**, and a collapsed **History & source** toggle. Outreach +is an evidence log, not a sales-management system. + +## Company Brain + +Use exactly these headings: + +1. Company & product +2. Pricing & business model +3. Market & positioning +4. Customer needs & examples +5. Business goals & baseline +6. Sales process & channels +7. Team & responsibilities + +Keep original source links in a collapsed **Library** toggle. Mark facts that +need human confirmation in red. The Brain describes the company; targeting rules +and batch learning live in Segments and Lists. + +## Counting and write safety + +Yes rate measures suggestion quality. Reply rate measures target responses. +Meetings are accepted meetings. Deduplicate people within each goal. A connector's +reply does not count as the target's reply unless the connector is itself the +target of that goal. + +Verify the selected company and destination data sources before writing. Reuse +existing rows before creating new ones and read back every write. After a timeout, +check whether the write landed before retrying. Preserve human edits, sources, +historic lists, verdicts, and results. + +Customer context, IDs, credentials, and run history stay outside this public +plugin. Existing authorization covers the agreed dashboard/list/draft updates; +it does not authorize sending messages, enabling schedules, or writing to another +company's workspace. diff --git a/skills/collect-company-context/SKILL.md b/skills/collect-company-context/SKILL.md index 8a4de3a..2e30386 100644 --- a/skills/collect-company-context/SKILL.md +++ b/skills/collect-company-context/SKILL.md @@ -1,28 +1,60 @@ --- name: collect-company-context -description: Research and discuss company context during customer onboarding, help the user choose target profiles, and save a validated company reference. +description: Gather, verify, and maintain the Company Brain for a research partner. Use during onboarding or when new company facts, goals, sources, or team information should update the shared Notion dashboard. --- # Collect company context -Own the canonical company reference: engagement goal and baseline, stable customer ICP, and selection and definition of three broad engagement target tracks—ICP #1, ICP #2 and Connectors. Help the user choose audiences most likely to advance the goal; hypotheses, candidates, review batches and iteration learning belong to `daily-opportunities`. - -Start from the customer page or research-partner directory linked in the conversation or workspace instructions. Resolve the customer and connected noticed team from available context; ask only when ambiguous. - -Before writing company context, resolve the canonical partner record. When a directory is available, check for a matching partner and reuse it; if absent, create it in that directory using its existing/default partner template, discovered from the directory. Never use a standalone/private staging page in its place. Keep this workspace setup out of the company reference. - -1. Read [company-context-template.md](company-context-template.md) and available company materials; research material gaps before interviewing. Establish the current strategy, available offer and engagement goal. Prioritize current, direct evidence and sources the user identifies as authoritative. Describe only currently available customer-facing offers and pricing. Resolve material conflicts with the user; present a provisional understanding without asking them to repeat known information. - -2. Ask one focused question at a time when the answer materially changes targeting, fit, approach or success measurement. Establish the meeting target, deadline and relevant business goal and baseline before validation. Prefer concise, grounded options labeled with numbers or letters; say whether one or several can be chosen and allow free-form corrections. Use examples when helpful or an open question when choices would be invented. After each answer, ask the next necessary question or present the finished draft for validation. Saving progress is not a stopping point; resolve material gaps in conversation rather than parking them under “To clarify.” - -3. Make target-profile selection the main discussion. Propose several distinct options, offering a real choice beyond the three final slots. Label each option with a number or letter. For each, briefly describe who they are, the situation that makes them relevant, why they could advance the goal, and the supporting evidence or assumptions. Include direct audiences and connectors; a connector option should explain whom they could reach. Let the user pick winners by label, combine or reject options, and optionally explain their choices. Ask for reasoning only when it would materially improve the profiles. Refine the options together instead of choosing on the user’s behalf. - -4. Confirm and save meaningful definitions for the three selected target profiles. ICP tracks may represent customers, investors or another audience relevant to the goal. The stable customer ICP describes the company’s durable best-fit customers; the engagement tracks are audiences being tested and need not put that ICP in ICP #1. If the user has not chosen all three, retain the confirmed choices and surface the remaining decision; do not invent agreement to fill a slot. Approval means worth pursuing, not proven fit. - -5. Fill the reference using the companion template. Research and provisional profiles may be saved incrementally on the canonical page, clearly marked draft/unvalidated until explicit reference validation. Distinguish capabilities from plans, results from targets, and facts from assumptions. Date baselines and time-sensitive claims. Aim for 600–800 words containing only information that changes targeting, fit, approach or success measurement. Capture the team’s existing approach and preferences; leave recommended outreach tactics, network selection and execution decisions to later routines. Do not pad missing information. - -6. Once material questions and conflicts are resolved—or the user explicitly accepts an unknown—save and present the actual draft for validation, especially the goal, current offer, stable ICP and selected profiles. Incorporate corrections and obtain explicit validation; profile selection alone does not validate the document. Set **Last reviewed** only on validation; unvalidated revisions remain drafts. Read back the saved content to verify it matches the agreed reference, then confirm completion with its link. - -For Notion, use **Knowledge → Company context**, with each section inside a toggle. Follow renamed sections by their meaning; preserve human edits and computed properties. Keep supporting materials and source links in **Library**, avoiding repeated links throughout the reference. The companion template defines the content independently of its destination. - -This is the current company reference, not a learning log. Keep customer-specific information outside this skill. +Maintain the concise company source of truth that research and outreach agents use. +Company facts live in the Company Brain; targeting rules and batch learning do not. + +Read the [research-partner workflow](../_shared/research-partner-workflow.md). +Start from the partner directory, dashboard, or company page supplied in the +conversation. Resolve the canonical partner record and connected noticed team. +When a directory is available, reuse a matching partner; if absent, create it from +that directory's default template. Do not use a separate staging page. + +## Gather the useful facts + +Read the existing dashboard, Company Brain, Library, call notes, and approved +public sources before asking the user to repeat information. Establish the +company's current offer, business goals, measurable baseline, deadline, existing +sales process, and the team members responsible. + +Ask one focused question at a time only when the answer materially changes +research, outreach, or success measurement. Prefer concise grounded options when +they help, and allow free-form corrections. Resolve material source conflicts +with the user. + +## Maintain the Company Brain + +Keep exactly these seven headings: + +1. Company & product +2. Pricing & business model +3. Market & positioning +4. Customer needs & examples +5. Business goals & baseline +6. Sales process & channels +7. Team & responsibilities + +Capture current facts that help an agent understand what the company does and how +it works. Include only currently available offers and pricing. Distinguish +capabilities from plans and actual results from targets. Date baselines and +time-sensitive claims. + +Keep original source links in the collapsed **Library** toggle and cite a source +beside a fact when its provenance materially affects confidence. Mark text that +still needs human confirmation in red. Do not pad empty sections or invent +missing facts. + +The Company Brain describes the company. Observable targeting rules belong in +Segments, and reviews belong in Lists and the segment's learning log. Do not add +ICP prose, target-profile definitions, or hypothesis pages here. A company may +have several goals; keep them on the same dashboard with consistent names rather +than creating duplicate company pages. + +Preserve useful existing facts, sources, and human edits. Read back the saved +content, then report the meaningful additions, conflicts, and facts still needing +confirmation. Keep partner-specific information in its workspace, never in this +skill. diff --git a/skills/collect-company-context/company-context-template.md b/skills/collect-company-context/company-context-template.md deleted file mode 100644 index 130e8f2..0000000 --- a/skills/collect-company-context/company-context-template.md +++ /dev/null @@ -1,57 +0,0 @@ -# Company context - -**Last reviewed:** [Date the user validated this reference, or “Not yet validated.” Mark subsequent unvalidated revisions as drafts.] - -Use these sections in the completed reference. In Notion, keep them as toggles inside **Knowledge → Company context** and store sources in **Library**. Date time-sensitive facts. Resolve material questions before validation; label any unknowns the user explicitly accepts. Omit unsupported detail and workspace setup. - -## Company - -- Name, website, what the company does, and whom it helps. -- Relevant stage, geography and operating context. - -## Product & offer - -- Problem solved, current offer, customer value and business model. -- Current customer-facing pricing and delivery capabilities when relevant; distinguish available features from plans. Exclude obsolete offers and internal pricing options. - -## Market & positioning - -- Market category, conditions and changes that affect the company. -- Competitors and alternatives, including existing workflows or doing nothing. -- Differentiation supported by evidence or explicitly stated as company positioning. - -## ICP & target profiles - -### Stable customer ICP - -- The company’s durable best-fit customers: organization, buyer/user and core need. -- Confirmed poor fits and actual examples, distinguishing customers from prospects or desired customers. - -### Selected target profiles - -Record the three audiences being tested for this engagement: **ICP track 1**, **ICP track 2**, and **Connector track**. Direct tracks can serve goals such as sales or fundraising; they do not redefine the stable customer ICP, which need not occupy ICP track 1. - -For each selected profile: - -- **Who:** role, organization characteristics and relevant situation; for connectors, whom they can reach. -- **Why:** how they could advance the goal and why the opportunity could matter to them. -- **Basis & boundaries:** supporting evidence, main exclusions and untested assumptions. - -Label unconfirmed choices as proposed. Keep the final reference focused on the winners; matching hypotheses, candidate lists and iteration history live in the opportunity routine. - -## Business goals & baseline - -- Priority outcome, meeting target and deadline, with a clear success measure. -- Dated starting point, current results and material constraints. -- Distinguish the company’s business goal from the engagement’s contribution to it. - -## Sales approach - -- How the team currently pursues the goal: channels, tactics and process. -- What works, what does not, and known bottlenecks. -- For other goals, describe the relevant fundraising, hiring or partnership approach. Record existing practices and preferences; leave proposed outreach tactics and network selection to later routines. - -## Team & responsibilities - -- User’s role and relevant team members, responsibilities and decision-makers. -- Link known profiles where useful. diff --git a/skills/daily-opportunities/SKILL.md b/skills/daily-opportunities/SKILL.md index bacc645..52cea3e 100644 --- a/skills/daily-opportunities/SKILL.md +++ b/skills/daily-opportunities/SKILL.md @@ -1,85 +1,116 @@ --- name: daily-opportunities -description: Prepare or refine 25 opportunities that advance the engagement goal within one agreed target profile, using a concrete hypothesis and human review feedback. +description: Find and refine research-partner opportunities in 25-person batches using observable segment rules, publish lists for human review, and record what the reviews teach. Use for preparing a first or later list, processing Yes/No/Not sure feedback, or updating opportunity metrics. For a one-off network lookup, use search-network instead. --- # Daily opportunities -Advance the engagement goal through relevant opportunities and meetings. Own hypotheses within an agreed target track, candidate selection, review batches, feedback and iteration learning. Improve the share receiving a human **Yes**, with booked meetings as a measurable outcome where applicable. +Help a partner discover who is worth contacting. The working loop is: -## Inputs and terms +**Goal → Segment rules → List → Review → Learning → Refinement** -Use the **customer page** and **selected target profile** supplied in the request or established in the conversation. The customer has three target profiles; each run focuses on one. If the selection is unclear, ask. +There is no separate hypothesis object. The current Required, Prefer, and Exclude +rules are the targeting hypothesis. The review tests one statement: +**“This looks like a good fit. We should reach out to them.”** -If starting from a research-partner directory linked in the conversation or workspace instructions, open the selected partner’s page and follow its links for setup and context. Confirm the customer and connected noticed team. Inspect the live Notion pages, fields and templates before writing; follow renamed fields by their meaning, preserve human edits and leave computed properties to Notion. Ask only for missing information needed for this step. +Read the [research-partner workflow](../_shared/research-partner-workflow.md), the +selected partner's dashboard, and the current segment before acting. -- **Target profile:** the broad audience selected for this run. -- **Hypothesis:** the specific characteristics and signals being tested to identify that audience. -- **Iteration:** one hypothesis, 25 suggested people, their reviews and recorded learning. +Confirm the company and connected noticed team. Inspect the live Notion fields and +templates before writing, preserve human edits, and follow renamed fields by +meaning. Confirm that noticed exposes the current list, enrichment, review setup, +and review-reading tools needed for the requested step. -Before candidate work, confirm that noticed exposes `configure_list_review` and `get_list_reviews`, and that the connected Notion account can read the customer page. Missing tools or access leave this run **Preparing**, with the exact blocker recorded; do not promise a working review flow. Once access is restored, resume the already requested step from saved evidence. +## Decide what advances -Establish the intended human reviewer from the customer page or conversation. `get_list_reviews` returns only the authenticated user's answers (`reviewScope: self`, `reviewerUserId`). Team-list access does not expose teammates' answers. If the connected identity is not the intended reviewer, report that mismatch instead of treating zero visible answers as customer non-response. +Work on one selected goal and segment at a time. Read the latest rules, Lists rows, +noticed list, review evidence, and learning log. -## Determine the next step +- **No list:** prepare the first batch from the current segment rules. +- **Preparing:** resume the same batch and repair missing work. +- **To review with unanswered people:** preserve the batch, report the remaining + count, and continue independent work for other segments if requested. +- **Reviewed:** record the result and learning before changing rules or starting + another batch. +- **Conflicting evidence:** reconcile it; do not infer completion from counters or + a filtered view. -Read **Knowledge → Company context**, retaining the relevant page, database and data-source IDs. If a response is truncated, fetch the relevant linked sections or query the scoped data source; partial content is not proof that a profile or iteration is absent. +A new day does not automatically create a new list. An explicit Yes can move to +Outreach before the other people are reviewed. -Before creating or refining a hypothesis, starting an iteration or selecting candidates, require a meaningful saved definition of the selected target profile and explicit user agreement to it. Empty, placeholder or “to define” content does not qualify; neither company positioning nor a new search instruction substitutes for that agreement. If missing or unagreed, return to `collect-company-context` to define and agree the profile first, using the new instruction as proposal input. Only the relevant profile needs agreement; unrelated context gaps or pending whole-reference validation do not block this handoff. +The review API may expose only the authenticated reviewer's answers. Verify that +identity against the intended human reviewer. If it differs, report the mismatch +rather than treating zero visible answers as non-response. -In **Hypothesis iterations** (formerly Target Profiles), query actual entries whose **Target profile** (formerly Profile) matches the selection and open the highest **Iteration**. Templates are not iteration rows. Read the saved list and review state before choosing the next step: +## Find and rank 25 people -- **No entry exists:** define and save the first hypothesis before selecting people. -- **The candidate list is unfinished:** continue building the same list to 25 people; preserve the hypothesis and work already saved. -- **25 people saved but reviews are not configured:** configure and verify reviews on this same list. -- **Reviews configured but the Notion handoff is incomplete:** repair the missing review link, evidence or tracker update; preserve the batch and any answers. -- **Some of the 25 people lack reviews, after readiness is verified:** update the selected target profile’s **What’s happening** row with the reviewed count, remaining count and review link. Stop this run. **Not sure** counts as reviewed. -- **All 25 are reviewed:** record the feedback and learning before defining the next hypothesis. +Evaluate people only with publicly observable or enrichment-supported facts. +Required rules must match, preferred signals improve rank, and exclusions +disqualify. Missing data is unknown, not a negative fact. -A new day does not start a new iteration. Continue only the step requested by the user. If list membership differs from the saved batch, report the discrepancy before changing membership, interpreting feedback or creating another iteration. +Search broadly, enrich eligible candidates, and read completed enrichment back +before using it. An asynchronous started/in-progress result is not evidence. +Select 25 defensible people unless the user requests another batch size. Rank by +rule fit, relevance now, and a credible relationship path. Deduplicate within the +batch and against previously approved, contacted, or dropped people for the same +goal. -## Define the hypothesis +For each opportunity preserve enough evidence for a quick decision: person and +role, company, sector, geography, size or stage when available, matched signals, +relationship path, concise fit reason, and source links. Connectors are ordinary +segments whose rules describe observable access or community signals; do not +assume willingness to introduce. -Using company context and recorded learning, write a concrete hypothesis: which characteristics and observable signals identify people who fit the selected target profile, and why. A hypothesis may experimentally refine an established profile; it cannot supply a missing broad track definition. +If fewer than 25 defensible people remain, keep the batch Preparing and report +the shortfall. Do not pad it or silently broaden the rules. -Prefer signals relevant to fit that can be observed in noticed. Distinguish direct evidence from proxies, and treat missing information as unknown. +## Publish and verify -Save the hypothesis in the iteration’s **Hypothesis** section. In **Basis**, summarize the supporting evidence. For subsequent iterations, use **What changed** to explain the refinement and which previous learning motivated it. +Create or reuse one noticed list for the batch. When the available contract uses +create_list, put the segment and fit criteria in its description, use the +selected team's organization ID, and keep uncontrolled AI suggestions disabled. +Add only the selected people and reuse the list when resuming. -## Select 25 people +Create or reuse one dashboard Lists row with its exact **Goal**, **Segment**, +**Generated** date, **Review** state, and **List link**. Keep membership stable +once review starts. -Search within the connected noticed team for around 50 candidates matching the hypothesis. Check the engagement's outreach records and relevant noticed actions to exclude people already being pursued; list membership alone does not mean active pursuit. A title such as founder is a proxy, not proof of the required product, company stage, sales responsibility or need. Keep those unknowns explicit and exclude known contradictions. +Configure noticed review with the exact statement **“This looks like a good fit. +We should reach out to them.”** Put short segment context in the description. +Read existing configuration before retrying and preserve a matching locked +question. -Enrich all eligible candidates with `enrich_person`, then select and rank the best 25. `started` and `in_progress` mean asynchronous work, not verified new facts. Read the person's record back, including enrichment state, before using refreshed evidence. Record skipped/ineligible outcomes. If selection depends on pending enrichment, use bounded checks, save the candidate IDs and remaining work, and leave the iteration **Preparing** rather than polling indefinitely or claiming completion. +Read back the full list and review configuration, following pagination rather +than relying on the first page. Verify the intended member IDs, total, question, +reviewer, access, and returned review URL. A list description or membership alone +does not configure review, and membership is not human approval. Do not use +acceptance tools to simulate reviews. -Create the list with `create_list`: put the hypothesis and fit criteria in `description`, use the selected team’s `organization_id`, and set `ai_enabled: false`. Add only the selected 25 through `add_to_list`; creation does not add members. Reuse the existing list when resuming. Save its ID on the iteration and record all 25 person IDs with rank, fit rationale, supporting observed facts versus proxies, enrichment outcome and remaining uncertainties. Preserve useful existing evidence when repairing an incomplete record. Use returned URLs; never invent routes. +Only after those checks set Review to **To review** and save the returned review +URL as the List link. Read the Notion row back before reporting success. -If fewer than 25 credible matches are available, keep the iteration **Preparing** and explain the shortfall. Do not weaken the hypothesis or fill the batch with known poor matches to reach 25. +When an approved person moves to Outreach, connect the Outreach entry to this +exact Lists row through **Source list**. -## Make the list ready for review +## Learn and refine -Use `configure_list_review` with `question` set to **“Does this person genuinely fit the target profile?”** and concise hypothesis context in `description`. Writing the question in Notion or the list description does not configure reviews. Read existing configuration first when resuming; preserve a matching question and any human edits. Identical setup is safe to retry, but a changed locked question must not be overwritten. +Use explicit Yes, No, and Not sure verdicts. Not sure counts as reviewed but not +Yes. Silence, disappearance, or suggestion rejection is unreviewed. Encourage a +brief reason for No or Not sure when supported, but preserve a valid verdict +without one. -Read back `get_list` and `get_list_reviews`: +After sufficient feedback: -- Compare all saved list member IDs and eligible review `memberPersonIds` with the intended 25 IDs. Use `member_total` and review `total`, not page lengths. Continue `get_list` with `page`/`page_size` while `member_has_more`; review member pages contain 10 IDs, so continue `page` while `memberHasMore`, even if `hasMore` for answers is false. -- Verify `setupState: configured`, the actual question/context, intended reviewer identity and list access. Caller access alone does not prove access for a different customer. Report missing/inaccessible members or reviewer mismatch; do not expand sharing to make a check pass. -- Save the returned **`review_url`** in the iteration's Target list field and the selected profile's **What’s happening** row. `canonical_url` opens the general goal, not its review controls. +- update the Lists row's Results summary; +- append one segment learning row with Date, Batch, Focus, results, Learned, and + Next refinement; +- add, remove, or change the smallest useful segment rule; +- keep the historic list, membership, verdicts, and prior rules intact. -Only after these checks and the candidate evidence are saved, set **To review** and read back both Notion destinations. If a write fails or its outcome is uncertain, inspect what landed and repair that step without recreating the list. Report partial completion until the handoff is consistent. - -Keep the hypothesis and candidate list stable during review. Membership and suggestion acceptance are not human fit approval. Link the ready review page, state the next action and wait for human feedback. - -## Record feedback and learning - -Use `get_list_reviews` without an answer filter and read every page while `hasMore`, including reasons. Match `relationshipId` on each current answer to a saved batch person ID. Verify the same intended reviewer throughout; corrections replace old answers and undone answers disappear. Silence, removal and suggestion rejection are not fit verdicts. Once all 25 distinct saved batch members have current human reviews, record the Yes / No / Not sure counts and **Yes rate = Yes ÷ 25** on the Notion iteration. **Not sure** is reviewed but not Yes; verify formula results rather than overwriting computed fields. - -In **What we learned**, compare the hypothesis, the people selected and the customer’s feedback. Explain which signals helped identify fit, where matches fell short, and whether the issue was the hypothesis or its application during selection. Keep uncertainties explicit. - -Mark the iteration **Complete**. Use the recorded learning to inform the next hypothesis when starting another iteration. - -## Close the run - -Update the selected target profile’s **What’s happening** row with the iteration number, current state and next action. - -Read back the saved list and changed Notion records before reporting success. After an uncertain write, check whether it landed before retrying. Briefly tell the user what was saved, link to the iteration and identify anything awaiting their input. +Do not overfit to one example or treat a small rate change as proof. The aim is +progressive inference from revealed preferences, not a perfect ICP upfront. + +Update opportunity metrics separately for the correct goal. Yes rate is explicit +Yes divided by all suggestions in the measured lists; use “—” when the denominator +is unknown. Finish with the review count, learning, rule change, and next +responsible person. diff --git a/skills/daily-outreach/SKILL.md b/skills/daily-outreach/SKILL.md index 63948ad..1706740 100644 --- a/skills/daily-outreach/SKILL.md +++ b/skills/daily-outreach/SKILL.md @@ -1,60 +1,105 @@ --- name: daily-outreach -description: Develop outreach interactively for human-approved relationships, using noticed context and learning the user's messaging preferences. +description: Develop attributed outreach for human-approved research-partner opportunities, save the chosen draft, and reconcile confirmed sends, replies, and meetings. Use for one relationship at a time, not automatic sending or task management. --- # Daily outreach -Help the customer advance the engagement goal through relevant meetings by developing outreach together for relationships they approved with a **Yes**. Use the agreed target track and relevant company and noticed context to recommend the approach and message. The user is the expert and may have additional context. +Turn an explicit human Yes into a message a human can send while keeping every +reply and meeting attributable to the goal that produced it. A draft never counts +as contact. -Work on one relationship at a time. Ask **one focused question at a time**, preferably with **numbered or lettered choices**. Recommend an option when the context supports it, use relevant examples when helpful, and allow free-text answers. Avoid asking about decisions already supported by the available context. +Read the [research-partner workflow](../_shared/research-partner-workflow.md), the +partner dashboard, the approved opportunity, and its Lists row. -## Choose the relationship +Confirm the company and connected noticed team. Inspect the live Notion fields and +templates before writing, preserve human edits, and follow renamed fields by +meaning. -Use the customer page and any relationship selected by the user. If none is selected, suggest one approved relationship that needs outreach or a follow-up, explain why, and ask whether to work on it. +## Choose and reconcile the relationship -If starting from a research-partner directory linked in the conversation or workspace instructions, open the selected partner’s page and follow its links for setup and context. Confirm the customer and connected noticed team. Inspect the live Notion pages, fields and templates before writing; follow renamed fields by their meaning and leave computed properties to Notion. Ask only for missing information needed now. +Use a relationship selected by the user. If none is selected, suggest one explicit +Yes that needs outreach, explain why, and ask whether to work on it. An individual +Yes is eligible even while the rest of its list remains unreviewed. -Use noticed MCP to read the relationship context and the customer's current approval. Read **Knowledge → Company context**, the approved opportunity's target track and hypothesis, saved **Outreach preferences**, and the existing Notion Outreach entry, including drafts and confirmed activity. Resolve the engagement goal and relevant agreed track from this context before proposing outreach; ask only if unclear. An individual Yes can move to outreach before the rest of its opportunity iteration is reviewed. +Read the noticed relationship, Company Brain, segment rules, source list, existing +Outreach entry, drafts, and confirmed activity. List membership alone is not +approval. If approval is missing, withdrawn, or unclear, resolve it before +drafting. -If approval is missing, withdrawn or unclear, resolve that before proceeding. Resume existing work; keep one Outreach entry per customer and relationship across target profiles. +Reuse an existing Outreach entry for the same person and goal. Set **Source list** +to the list that actually motivated this outreach; this relation is the reliable +path to Segment and Goal. If several lists contain the person, preserve other +useful evidence in History & source rather than making attribution ambiguous. -## Agree on the approach +Reconcile confirmed sends, target replies, bookings, and explicit drops before +deciding what to draft. Preserve actual dates and sources. Do not infer an event +from a plan or a stale status. -Use the engagement goal, target track, relationship history, shared connections and saved preferences, including the company's offer where relevant, to propose: +## Agree on the approach -- **Route:** direct outreach or an introduction. -- **Sender:** who is best placed to send it. -- **Channel:** where the conversation should happen. -- **Message approach:** what to lead with, why it is relevant to the recipient and how it advances the engagement goal. +Work one relationship at a time. Ask one focused question at a time, preferably +with numbered choices, only when the available context does not support a +decision. -Explain the recommendation briefly. Ask about the uncertainty that most affects it, using numbered choices where possible. Refine the approach with the user's answers. +Use the goal, company offer, segment evidence, relationship history, shared +connections, and sender preferences to recommend: -## Draft and refine +- direct outreach or an introduction; +- the sender; +- the channel; +- what to lead with and why it matters to the recipient. -Give the user **three short, direct message options**, each with a meaningfully different approach or style. Label them **1, 2 and 3**, with a brief description of the difference. +Explain the recommendation briefly. Do not invent familiarity, a connection, +sender, channel, or company fact. -Adapt to the actual relationship: established, dormant, through a mutual contact, or no prior relationship. Use familiarity only when supported by context; being recorded in noticed does not mean the sender knows the person. For cold outreach, briefly establish who the sender is and why the conversation is relevant. Use relevant shared context, give a clear reason to talk and make the next step easy. Keep messages short and direct; avoid forced personalization and unnecessary sales language. For an introduction, make clear who the message is addressed to. +Give three short message options with meaningfully different approaches, labeled +1, 2, and 3. For email, include a subject. For an introduction, identify the +intermediary and keep the request to them distinct from forwardable text. Let the +user choose, combine, or edit an option; preserve existing human edits. -Ask which option feels right; let the user choose, combine or edit. Refine using their feedback. Preserve existing human edits unless the user is revising that draft. +## Save the lean entry -## Preserve context and preferences +Use only these properties: -Save new **target-specific facts and context on the noticed relationship page**, preserving existing notes. +- Person +- Source list +- Status +- Sender +- Channel +- Sent on -Save **reusable outreach and messaging preferences** in an **Outreach preferences** toggle inside the customer's **Knowledge → Company context**. Create the toggle if missing. Attribute preferences to the relevant user or sender; distinguish a choice for this message from a general preference, and clarify when uncertain. +The page body contains **Message draft**, **Why reach out**, +**Introduction through**, and the collapsed **History & source** toggle. Save the +chosen copy in Message draft, the concise fit rationale in Why reach out, and the +evidenced route in Introduction through. -Update these records as the user supplies context or expresses preferences. Keep customer information outside this skill. +Do not add follow-up dates, next steps, activity-management fields, duplicate +Goal properties, or hidden reporting properties. The dashboard's simple Next +checklist holds current actions. -## Save the outcome +The human sends the message. If it was sent, record the confirmed sender, channel, +and actual **Sent on** date. If it was not sent, keep the entry at **To contact**. +Never mark a draft as contact. -Save the chosen message in the Notion Outreach entry's **Message draft**, the approach and brief rationale in **Context**, and the sender, channel and noticed relationship link in their properties. Preserve dated activity and update only the relevant fields. +## Record outcomes -The user sends the message. Ask what happened: +| Status | Evidence | +|---|---| +| To contact | Explicit Yes; no confirmed send. | +| Contacted | A direct message or introduction request was sent. | +| Replied | The target replied, regardless of sentiment. | +| Booked | The target accepted a meeting. | +| Dropped | There is an explicit decision to stop. | -1. **Sent** — record the confirmed send, asking for any missing sender, date or channel. -2. **Not sent yet** — keep the initial draft as **To contact**. An unsent follow-up does not reset an existing outreach status. +Record reply and meeting evidence in History & source or the configured activity +source. An intermediary's reply is not the target's reply. A proposed time is not +a booked meeting. -A saved draft never counts as contact. Record target replies and accepted calendar invitations when the user confirms them; an intermediary’s reply is not a target reply, and a proposed time is not a booking. Preserve earlier activity; keep an agreed follow-up date and next action when relevant. +Use Source list → Goal to update Outreach and Meetings metrics for the correct +goal. Never combine unrelated goal funnels. Read back every saved entry and metric +change. Close with what was saved, what happened, the next responsible person, +and a link to the entry. -Read back the saved entry and any context or preference updates. After an uncertain write, check whether it landed before retrying. Close with a short recap, the next action and a link to the outreach entry. +Humans send messages. Do not contact prospects or intermediaries, book meetings, +or change sharing settings unless the user separately authorizes it. From 150c5079fa8bc30bbaefb18b61e31b6d7ae516a5 Mon Sep 17 00:00:00 2001 From: macedo Date: Sat, 19 Sep 2026 14:09:53 +0100 Subject: [PATCH 2/5] refactor(skills): remove duplicated workflow detail --- skills/_shared/research-partner-workflow.md | 99 ++++++--------- skills/collect-company-context/SKILL.md | 67 +++------- skills/daily-opportunities/SKILL.md | 133 +++++++------------- skills/daily-outreach/SKILL.md | 119 +++++------------- 4 files changed, 132 insertions(+), 286 deletions(-) diff --git a/skills/_shared/research-partner-workflow.md b/skills/_shared/research-partner-workflow.md index e4c5e92..697d3da 100644 --- a/skills/_shared/research-partner-workflow.md +++ b/skills/_shared/research-partner-workflow.md @@ -1,39 +1,31 @@ # Research-partner workflow -Use one canonical dashboard per company. A company may have one or more goals; -each goal has its own segments, lists, outreach, and metrics. Use the same exact -goal name everywhere and never aggregate unrelated funnels. - -The operating loop is: +Use one dashboard per company and one consistent name for each goal. Keep metrics +separate by goal. **Goal → Segment rules → List → Review → Learning → Refinement → Outreach** ## Dashboard -Keep these sections in this order: +Keep this order: 1. **Progress** — goal statements and per-goal Meetings, Outreach, and - Opportunities metrics; Network remains a company-level snapshot. -2. **Next** — a short to-do list naming the goal, owner, and concrete action. -3. **Segments** — the current observable targeting rules. -4. **Lists** — every generated batch and its result. -5. **Outreach** — approved people, drafts, and confirmed outcomes. + Opportunities; Network is a company snapshot. +2. **Next** — short to-dos with goal, owner, and action. +3. **Segments** +4. **Lists** +5. **Outreach** 6. **Company** — link to the Company Brain. -Unknown values display as “—”. Update the last-measurement date only for verified -data. Keep goal metrics separate. +Use “—” for unknown values. Change Last measurement only when the data is verified. ## Segments -Database properties: **Segment** (title), **Goal**, **Priority**. - -The segment name is the short plain-language definition of who to find. There is -no separate “Who” field and no separate hypothesis page. A connector is simply a -segment with connector-specific rules. +Properties: **Segment**, **Goal**, **Priority**. -Every segment page uses a criteria table with **Category**, **Required**, -**Prefer**, and **Exclude**. Keep all categories present for consistency, leaving -unused cells empty: +The segment name is a short definition of who to find. A connector is a segment, +not a separate type. Each segment page has **Category**, **Required**, **Prefer**, +and **Exclude**, with these rows even when empty: - Geography - Sector / industry @@ -49,40 +41,39 @@ unused cells empty: - Revenue - Company age -Write comma-separated tags or short observable rules, not profile prose. Required -means every populated category must match; comma-separated alternatives within a -cell mean “or.” Prefer improves ranking but is optional. Exclude disqualifies. -Rules must be testable through public search or data enrichment. Facts discoverable -only through contact belong in a collapsed contact-only note, not in targeting. +Use comma-separated tags or short rules that public search or enrichment can +validate. Required categories must match; comma-separated values within a cell +mean “or.” Prefer affects ranking. Exclude disqualifies. Contact-only facts do not +belong in targeting rules. -Below the criteria, use **Learnings & refinements** with columns **Date**, -**Batch**, **Focus**, **Results · Yes / No / ?**, **Learned**, and -**Next refinement**. +The learning log columns are **Date**, **Batch**, **Focus**, +**Results · Yes / No / ?**, **Learned**, and **Next refinement**. ## Lists -Database properties: **List** (title), **Goal**, **List link**, **Segment** -(relation), **Generated**, **Review**, **Results**. +Properties: **List**, **Goal**, **List link**, **Segment**, **Generated**, +**Review**, **Results**. -One row represents one generated batch. Preserve every batch so progress and rule -changes remain traceable. The list review tests the statement: **“This looks like -a good fit. We should reach out to them.”** Use explicit Yes, No, and Not sure. +One row is one generated batch. Preserve every batch. Review the statement: +**“This looks like a good fit. We should reach out to them.”** using Yes, No, or +Not sure. ## Outreach -Database properties: **Person** (title), **Source list** (relation to Lists), -**Status**, **Sender**, **Channel**, **Sent on**. +Properties: **Person**, **Source list**, **Status**, **Sender**, **Channel**, +**Sent on**. Source list relates to Lists and supplies Segment and Goal. -Source list is required attribution. Its List row already supplies Segment and -Goal, so do not duplicate those properties in Outreach. +The page body contains **Message draft**, **Why reach out**, +**Introduction through**, and collapsed **History & source**. Outreach is an +evidence log, not a task-management system. -The entry body contains **Message draft**, **Why reach out**, -**Introduction through**, and a collapsed **History & source** toggle. Outreach -is an evidence log, not a sales-management system. +Statuses require evidence: **To contact** means approved and unsent; +**Contacted** means sent; **Replied** means the target replied; **Booked** means +the target accepted a meeting; **Dropped** means an explicit decision to stop. ## Company Brain -Use exactly these headings: +Use these headings: 1. Company & product 2. Pricing & business model @@ -92,23 +83,13 @@ Use exactly these headings: 6. Sales process & channels 7. Team & responsibilities -Keep original source links in a collapsed **Library** toggle. Mark facts that -need human confirmation in red. The Brain describes the company; targeting rules -and batch learning live in Segments and Lists. - -## Counting and write safety +Keep original sources in collapsed **Library**. Mark unconfirmed facts red. +Company facts live here; targeting and learning do not. -Yes rate measures suggestion quality. Reply rate measures target responses. -Meetings are accepted meetings. Deduplicate people within each goal. A connector's -reply does not count as the target's reply unless the connector is itself the -target of that goal. +## Invariants -Verify the selected company and destination data sources before writing. Reuse -existing rows before creating new ones and read back every write. After a timeout, -check whether the write landed before retrying. Preserve human edits, sources, -historic lists, verdicts, and results. +Yes rate measures human-approved suggestions. Reply rate measures target replies. +Meetings require acceptance. Deduplicate within each goal. -Customer context, IDs, credentials, and run history stay outside this public -plugin. Existing authorization covers the agreed dashboard/list/draft updates; -it does not authorize sending messages, enabling schedules, or writing to another -company's workspace. +Verify the company and destination before writing, reuse existing records, preserve +human edits and history, and read writes back before claiming completion. diff --git a/skills/collect-company-context/SKILL.md b/skills/collect-company-context/SKILL.md index 2e30386..3bd3872 100644 --- a/skills/collect-company-context/SKILL.md +++ b/skills/collect-company-context/SKILL.md @@ -1,60 +1,29 @@ --- name: collect-company-context -description: Gather, verify, and maintain the Company Brain for a research partner. Use during onboarding or when new company facts, goals, sources, or team information should update the shared Notion dashboard. +description: Gather and maintain the Company Brain for a research partner. Use during onboarding or when company facts, goals, sources, or responsibilities change. --- # Collect company context -Maintain the concise company source of truth that research and outreach agents use. -Company facts live in the Company Brain; targeting rules and batch learning do not. +Maintain the company facts that research and outreach need. Read the +[research-partner workflow](../_shared/research-partner-workflow.md) and locate the +company's canonical dashboard. Reuse it; create one from the directory's default +template only when none exists. -Read the [research-partner workflow](../_shared/research-partner-workflow.md). -Start from the partner directory, dashboard, or company page supplied in the -conversation. Resolve the canonical partner record and connected noticed team. -When a directory is available, reuse a matching partner; if absent, create it from -that directory's default template. Do not use a separate staging page. +Read the existing Company Brain, Library, call notes, and approved public sources +before asking questions. Capture the current offer and pricing, market position, +customer needs and examples, goals and baseline, sales process, and responsible +team members. Ask only about gaps or conflicts that materially affect research, +outreach, or measurement. -## Gather the useful facts - -Read the existing dashboard, Company Brain, Library, call notes, and approved -public sources before asking the user to repeat information. Establish the -company's current offer, business goals, measurable baseline, deadline, existing -sales process, and the team members responsible. - -Ask one focused question at a time only when the answer materially changes -research, outreach, or success measurement. Prefer concise grounded options when -they help, and allow free-form corrections. Resolve material source conflicts -with the user. - -## Maintain the Company Brain - -Keep exactly these seven headings: - -1. Company & product -2. Pricing & business model -3. Market & positioning -4. Customer needs & examples -5. Business goals & baseline -6. Sales process & channels -7. Team & responsibilities - -Capture current facts that help an agent understand what the company does and how -it works. Include only currently available offers and pricing. Distinguish -capabilities from plans and actual results from targets. Date baselines and -time-sensitive claims. - -Keep original source links in the collapsed **Library** toggle and cite a source -beside a fact when its provenance materially affects confidence. Mark text that -still needs human confirmation in red. Do not pad empty sections or invent -missing facts. +Use the seven Company Brain headings defined in the workflow contract. Record +current facts, distinguish plans from capabilities and targets from results, and +date time-sensitive claims. Keep original sources in Library. Mark facts that +still need human confirmation in red; do not invent or pad missing information. The Company Brain describes the company. Observable targeting rules belong in -Segments, and reviews belong in Lists and the segment's learning log. Do not add -ICP prose, target-profile definitions, or hypothesis pages here. A company may -have several goals; keep them on the same dashboard with consistent names rather -than creating duplicate company pages. +Segments; review results and refinements belong with Segments and Lists. Keep +multiple goals on the same company dashboard with consistent names. -Preserve useful existing facts, sources, and human edits. Read back the saved -content, then report the meaningful additions, conflicts, and facts still needing -confirmation. Keep partner-specific information in its workspace, never in this -skill. +Preserve useful facts, sources, and human edits. Read back the saved page and +report material additions, conflicts, and remaining confirmations. diff --git a/skills/daily-opportunities/SKILL.md b/skills/daily-opportunities/SKILL.md index 52cea3e..cee986e 100644 --- a/skills/daily-opportunities/SKILL.md +++ b/skills/daily-opportunities/SKILL.md @@ -1,116 +1,71 @@ --- name: daily-opportunities -description: Find and refine research-partner opportunities in 25-person batches using observable segment rules, publish lists for human review, and record what the reviews teach. Use for preparing a first or later list, processing Yes/No/Not sure feedback, or updating opportunity metrics. For a one-off network lookup, use search-network instead. +description: Prepare and refine 25-person opportunity lists from observable segment rules, publish them for Yes/No/Not sure review, and record what the feedback changes. For a one-off network lookup, use search-network instead. --- # Daily opportunities -Help a partner discover who is worth contacting. The working loop is: +Run the loop defined in the +[research-partner workflow](../_shared/research-partner-workflow.md): **Goal → Segment rules → List → Review → Learning → Refinement** -There is no separate hypothesis object. The current Required, Prefer, and Exclude -rules are the targeting hypothesis. The review tests one statement: -**“This looks like a good fit. We should reach out to them.”** +The segment's Required, Prefer, and Exclude rules are the hypothesis. Do not +create a separate hypothesis page. -Read the [research-partner workflow](../_shared/research-partner-workflow.md), the -selected partner's dashboard, and the current segment before acting. +## Choose the next action -Confirm the company and connected noticed team. Inspect the live Notion fields and -templates before writing, preserve human edits, and follow renamed fields by -meaning. Confirm that noticed exposes the current list, enrichment, review setup, -and review-reading tools needed for the requested step. +Confirm the company, noticed team, goal, and segment. Read its current rules, +historic Lists rows, noticed list, review evidence, and learning log. -## Decide what advances +- No list: prepare the first batch. +- Preparing: resume the same batch. +- Awaiting reviews: preserve the batch and report the remaining count. +- Reviewed: record learning before changing rules or starting another batch. +- Conflicting evidence: reconcile it rather than inferring a result. -Work on one selected goal and segment at a time. Read the latest rules, Lists rows, -noticed list, review evidence, and learning log. +A new day does not require a new list. An explicit Yes can move to Outreach while +the rest of the batch remains under review. -- **No list:** prepare the first batch from the current segment rules. -- **Preparing:** resume the same batch and repair missing work. -- **To review with unanswered people:** preserve the batch, report the remaining - count, and continue independent work for other segments if requested. -- **Reviewed:** record the result and learning before changing rules or starting - another batch. -- **Conflicting evidence:** reconcile it; do not infer completion from counters or - a filtered view. +## Prepare the batch -A new day does not automatically create a new list. An explicit Yes can move to -Outreach before the other people are reviewed. +Find 25 defensible people unless the user requests another batch size. Evaluate +only public or enrichment-supported facts. Required rules must match; Prefer +improves rank; Exclude disqualifies. Missing data is unknown. -The review API may expose only the authenticated reviewer's answers. Verify that -identity against the intended human reviewer. If it differs, report the mismatch -rather than treating zero visible answers as non-response. +Rank by rule fit, relevance now, and credible relationship path. Deduplicate +within the batch and against people already approved, contacted, or dropped for +the same goal. Connectors are ordinary segments defined by observable access or +community signals; never assume willingness to introduce. -## Find and rank 25 people - -Evaluate people only with publicly observable or enrichment-supported facts. -Required rules must match, preferred signals improve rank, and exclusions -disqualify. Missing data is unknown, not a negative fact. - -Search broadly, enrich eligible candidates, and read completed enrichment back -before using it. An asynchronous started/in-progress result is not evidence. -Select 25 defensible people unless the user requests another batch size. Rank by -rule fit, relevance now, and a credible relationship path. Deduplicate within the -batch and against previously approved, contacted, or dropped people for the same -goal. - -For each opportunity preserve enough evidence for a quick decision: person and -role, company, sector, geography, size or stage when available, matched signals, -relationship path, concise fit reason, and source links. Connectors are ordinary -segments whose rules describe observable access or community signals; do not -assume willingness to introduce. - -If fewer than 25 defensible people remain, keep the batch Preparing and report -the shortfall. Do not pad it or silently broaden the rules. +Each opportunity needs enough public evidence for a quick decision: person and +role, company, relevant company facts, matched signals, relationship path, concise +fit reason, and sources. If 25 credible matches are unavailable, keep the list +Preparing and report the shortfall rather than weakening the rules. ## Publish and verify -Create or reuse one noticed list for the batch. When the available contract uses -create_list, put the segment and fit criteria in its description, use the -selected team's organization ID, and keep uncontrolled AI suggestions disabled. -Add only the selected people and reuse the list when resuming. - -Create or reuse one dashboard Lists row with its exact **Goal**, **Segment**, -**Generated** date, **Review** state, and **List link**. Keep membership stable -once review starts. +Create or resume one noticed list and one Lists row for the batch. Follow the live +tool contracts and template instead of assuming stale field or argument names. +Keep membership stable once review starts. -Configure noticed review with the exact statement **“This looks like a good fit. -We should reach out to them.”** Put short segment context in the description. -Read existing configuration before retrying and preserve a matching locked -question. +Configure the review statement from the workflow contract. Read back the full +list and review setup, including all pages. Verify the intended people, reviewer, +question, access, and review URL. List membership is not human approval. Only then +mark the row To review and save the review URL. -Read back the full list and review configuration, following pagination rather -than relying on the first page. Verify the intended member IDs, total, question, -reviewer, access, and returned review URL. A list description or membership alone -does not configure review, and membership is not human approval. Do not use -acceptance tools to simulate reviews. - -Only after those checks set Review to **To review** and save the returned review -URL as the List link. Read the Notion row back before reporting success. - -When an approved person moves to Outreach, connect the Outreach entry to this -exact Lists row through **Source list**. +When a Yes moves to Outreach, connect it to this exact Lists row through Source +list. ## Learn and refine -Use explicit Yes, No, and Not sure verdicts. Not sure counts as reviewed but not -Yes. Silence, disappearance, or suggestion rejection is unreviewed. Encourage a -brief reason for No or Not sure when supported, but preserve a valid verdict -without one. - -After sufficient feedback: - -- update the Lists row's Results summary; -- append one segment learning row with Date, Batch, Focus, results, Learned, and - Next refinement; -- add, remove, or change the smallest useful segment rule; -- keep the historic list, membership, verdicts, and prior rules intact. +Use explicit Yes, No, and Not sure verdicts. Not sure is reviewed but not Yes; +silence or disappearance is unreviewed. Preserve valid verdicts without requiring +a reason. -Do not overfit to one example or treat a small rate change as proof. The aim is -progressive inference from revealed preferences, not a perfect ICP upfront. +After enough feedback, update Results, add one learning row, and make the smallest +supported rule change. Preserve the historic list, membership, verdicts, and +previous rules. Avoid conclusions from a single example or a small rate change. -Update opportunity metrics separately for the correct goal. Yes rate is explicit -Yes divided by all suggestions in the measured lists; use “—” when the denominator -is unknown. Finish with the review count, learning, rule change, and next -responsible person. +Update opportunity metrics for the correct goal and report the review count, +learning, refinement, and next responsible person. diff --git a/skills/daily-outreach/SKILL.md b/skills/daily-outreach/SKILL.md index 1706740..4381fc5 100644 --- a/skills/daily-outreach/SKILL.md +++ b/skills/daily-outreach/SKILL.md @@ -1,105 +1,46 @@ --- name: daily-outreach -description: Develop attributed outreach for human-approved research-partner opportunities, save the chosen draft, and reconcile confirmed sends, replies, and meetings. Use for one relationship at a time, not automatic sending or task management. +description: Develop attributed outreach for one human-approved opportunity, save the chosen draft, and record confirmed sends, replies, and meetings. This skill never sends automatically. --- # Daily outreach -Turn an explicit human Yes into a message a human can send while keeping every -reply and meeting attributable to the goal that produced it. A draft never counts -as contact. +Turn an explicit human Yes into outreach while preserving attribution. Read the +[research-partner workflow](../_shared/research-partner-workflow.md), the Company +Brain, segment, source list, noticed relationship, and existing Outreach entry. -Read the [research-partner workflow](../_shared/research-partner-workflow.md), the -partner dashboard, the approved opportunity, and its Lists row. +## Choose the opportunity -Confirm the company and connected noticed team. Inspect the live Notion fields and -templates before writing, preserve human edits, and follow renamed fields by -meaning. +Use the person selected by the user. If none is selected, suggest one approved +person who needs outreach and ask whether to proceed. List membership alone is not +approval. -## Choose and reconcile the relationship +Reuse the existing Outreach entry for the same person and goal. Set Source list to +the batch that motivated the outreach; keep evidence from other lists in History +& source. Reconcile confirmed sends, replies, bookings, and drops before drafting. -Use a relationship selected by the user. If none is selected, suggest one explicit -Yes that needs outreach, explain why, and ask whether to work on it. An individual -Yes is eligible even while the rest of its list remains unreviewed. +## Develop the message -Read the noticed relationship, Company Brain, segment rules, source list, existing -Outreach entry, drafts, and confirmed activity. List membership alone is not -approval. If approval is missing, withdrawn, or unclear, resolve it before -drafting. +Recommend a direct message or introduction, sender, channel, and angle from the +goal, company offer, segment evidence, relationship history, and real connections. +Ask one focused question only when a material choice cannot be inferred. -Reuse an existing Outreach entry for the same person and goal. Set **Source list** -to the list that actually motivated this outreach; this relation is the reliable -path to Segment and Goal. If several lists contain the person, preserve other -useful evidence in History & source rather than making attribution ambiguous. +Draft one short recommended message. Offer alternatives when there is a real +decision to compare. For email, include a subject. For an introduction, separate +the request to the intermediary from any forwardable text. Never invent +familiarity, relationships, or company facts. -Reconcile confirmed sends, target replies, bookings, and explicit drops before -deciding what to draft. Preserve actual dates and sources. Do not infer an event -from a plan or a stale status. +## Save and record -## Agree on the approach +Use the live Outreach template defined in the workflow contract. Save the chosen +copy, concise fit rationale, evidenced introduction route, and Source list. +Preserve human edits. -Work one relationship at a time. Ask one focused question at a time, preferably -with numbered choices, only when the available context does not support a -decision. +A human sends the message. Set Sent on and Contacted only after a confirmed send. +Record replies, accepted meetings, and drops from evidence. An intermediary's +reply is not the target's reply; a proposed time is not a booking. -Use the goal, company offer, segment evidence, relationship history, shared -connections, and sender preferences to recommend: - -- direct outreach or an introduction; -- the sender; -- the channel; -- what to lead with and why it matters to the recipient. - -Explain the recommendation briefly. Do not invent familiarity, a connection, -sender, channel, or company fact. - -Give three short message options with meaningfully different approaches, labeled -1, 2, and 3. For email, include a subject. For an introduction, identify the -intermediary and keep the request to them distinct from forwardable text. Let the -user choose, combine, or edit an option; preserve existing human edits. - -## Save the lean entry - -Use only these properties: - -- Person -- Source list -- Status -- Sender -- Channel -- Sent on - -The page body contains **Message draft**, **Why reach out**, -**Introduction through**, and the collapsed **History & source** toggle. Save the -chosen copy in Message draft, the concise fit rationale in Why reach out, and the -evidenced route in Introduction through. - -Do not add follow-up dates, next steps, activity-management fields, duplicate -Goal properties, or hidden reporting properties. The dashboard's simple Next -checklist holds current actions. - -The human sends the message. If it was sent, record the confirmed sender, channel, -and actual **Sent on** date. If it was not sent, keep the entry at **To contact**. -Never mark a draft as contact. - -## Record outcomes - -| Status | Evidence | -|---|---| -| To contact | Explicit Yes; no confirmed send. | -| Contacted | A direct message or introduction request was sent. | -| Replied | The target replied, regardless of sentiment. | -| Booked | The target accepted a meeting. | -| Dropped | There is an explicit decision to stop. | - -Record reply and meeting evidence in History & source or the configured activity -source. An intermediary's reply is not the target's reply. A proposed time is not -a booked meeting. - -Use Source list → Goal to update Outreach and Meetings metrics for the correct -goal. Never combine unrelated goal funnels. Read back every saved entry and metric -change. Close with what was saved, what happened, the next responsible person, -and a link to the entry. - -Humans send messages. Do not contact prospects or intermediaries, book meetings, -or change sharing settings unless the user separately authorizes it. +Use Source list → Goal for the correct Outreach and Meetings metrics. Read back +the entry, then report what was saved, the confirmed outcome, and the next +responsible person. Do not contact anyone or book a meeting without separate +authorization. From 9cf0cb7939c78a5de267700862eab71f997ad902 Mon Sep 17 00:00:00 2001 From: macedo Date: Sat, 19 Sep 2026 14:34:22 +0100 Subject: [PATCH 3/5] test(skills): add research partner behavior evals --- .github/workflows/test.yml | 19 +++ TESTING.md | 22 +++ evals/research_partner/cases.json | 124 ++++++++++++++ evals/research_partner/output.schema.json | 81 +++++++++ scripts/run_research_partner_evals.py | 194 ++++++++++++++++++++++ tests/test_research_partner_skills.py | 72 ++++++++ 6 files changed, 512 insertions(+) create mode 100644 .github/workflows/test.yml create mode 100644 evals/research_partner/cases.json create mode 100644 evals/research_partner/output.schema.json create mode 100755 scripts/run_research_partner_evals.py create mode 100644 tests/test_research_partner_skills.py diff --git a/.github/workflows/test.yml b/.github/workflows/test.yml new file mode 100644 index 0000000..dce7fbe --- /dev/null +++ b/.github/workflows/test.yml @@ -0,0 +1,19 @@ +name: Tests + +on: + pull_request: + push: + branches: [main] + +permissions: + contents: read + +jobs: + unittest: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v5 + with: + python-version: "3.11" + - run: python -m unittest discover -s tests -v diff --git a/TESTING.md b/TESTING.md index 10b2b5e..a9476b1 100644 --- a/TESTING.md +++ b/TESTING.md @@ -72,6 +72,28 @@ Use synthetic company context and fixture-only tools; inspect proposed operation | A draft exists but no send is confirmed; later the user confirms a LinkedIn message was sent. | Keep To contact and Sent on empty before confirmation; then record Contacted, the channel and actual send date. Do not add follow-up-management fields. | | A connector replies and offers an introduction, but the target has not replied and no meeting is accepted. | Preserve the evidence without counting a target reply or booked meeting. Attribute the activity through Source list → Goal. | +### Run the research-partner evals + +Contract checks run in CI and need no model: + +```bash +python -m unittest discover -s tests -v +``` + +Behavioral decision evals use synthetic fixtures and the checked-out skill text. +They require an authenticated Codex CLI but do not connect to noticed, Notion or +the web: + +```bash +python scripts/run_research_partner_evals.py --repeats 3 +``` + +The runner scores state transitions and forbidden behavior deterministically. +Run it several times because model behavior is nondeterministic. Before releasing +a material workflow change, also perform one smoke test with a duplicated Notion +dashboard and disposable noticed team; fixture evals do not prove live tool +integration. + ## what to watch for The behaviors the NYTW revisions were built to get right — confirm they hold across the tests above: diff --git a/evals/research_partner/cases.json b/evals/research_partner/cases.json new file mode 100644 index 0000000..9345402 --- /dev/null +++ b/evals/research_partner/cases.json @@ -0,0 +1,124 @@ +[ + { + "id": "company_context_updates_canonical_brain", + "skill": "skills/collect-company-context/SKILL.md", + "request": "Update the company context from these new notes.", + "fixture": { + "dashboard": {"id": "company-1", "exists": true}, + "facts": [ + {"id": "current-price", "text": "Implementation currently costs €3,000.", "confirmed": true}, + {"id": "planned-subscription", "text": "A subscription may be introduced later.", "confirmed": false} + ], + "sources": ["call-2026-09-18", "pricing-page"], + "targeting_idea": "Prefer companies with active hiring." + }, + "assertions": [ + {"path": "action", "op": "eq", "value": "update_company_brain"}, + {"path": "creates_new_dashboard", "op": "eq", "value": false}, + {"path": "company_brain_sections", "op": "contains_all", "value": ["Company & product", "Pricing & business model", "Market & positioning", "Customer needs & examples", "Business goals & baseline", "Sales process & channels", "Team & responsibilities"]}, + {"path": "unconfirmed_facts", "op": "contains_all", "value": ["planned-subscription"]}, + {"path": "writes_targeting_rules_to_brain", "op": "eq", "value": false}, + {"path": "preserves_history", "op": "eq", "value": true} + ] + }, + { + "id": "partial_review_resumes_existing_batch", + "skill": "skills/daily-opportunities/SKILL.md", + "request": "Run daily opportunities for the Partners goal and the founder-led sales segment.", + "fixture": { + "goal": "Partners", + "segment": "Founder-led B2B software sales", + "list": {"id": "list-partners-1", "review_state": "To review", "member_count": 25}, + "reviews": { + "yes": ["p1", "p2", "p3"], + "no": ["p4", "p5", "p6", "p7", "p8"], + "not_sure": ["p9", "p10"] + } + }, + "assertions": [ + {"path": "action", "op": "eq", "value": "wait_for_reviews"}, + {"path": "creates_new_list", "op": "eq", "value": false}, + {"path": "list_id", "op": "eq", "value": "list-partners-1"}, + {"path": "reviewed_count", "op": "eq", "value": 10}, + {"path": "remaining_reviews", "op": "eq", "value": 15}, + {"path": "approved_for_outreach", "op": "contains_all", "value": ["p1", "p2", "p3"]}, + {"path": "rule_changes", "op": "len_eq", "value": 0}, + {"path": "preserves_history", "op": "eq", "value": true} + ] + }, + { + "id": "completed_review_records_one_refinement", + "skill": "skills/daily-opportunities/SKILL.md", + "request": "Process the completed review for this Partners list.", + "fixture": { + "goal": "Partners", + "segment": "Founder-led B2B software sales", + "rules": {"required": ["B2B software", "Founder or CEO"], "prefer": ["15-50 employees"], "exclude": []}, + "list": {"id": "list-partners-2", "review_state": "To review", "member_count": 25}, + "reviews": { + "yes_count": 10, + "no_count": 12, + "not_sure_count": 3, + "repeated_reason": "Companies under 10 employees were consistently rejected as too early." + } + }, + "assertions": [ + {"path": "action", "op": "eq", "value": "record_learning"}, + {"path": "creates_new_list", "op": "eq", "value": false}, + {"path": "list_id", "op": "eq", "value": "list-partners-2"}, + {"path": "reviewed_count", "op": "eq", "value": 25}, + {"path": "remaining_reviews", "op": "eq", "value": 0}, + {"path": "rule_changes", "op": "len_lte", "value": 1}, + {"path": "metric_goal", "op": "eq", "value": "Partners"}, + {"path": "preserves_history", "op": "eq", "value": true} + ] + }, + { + "id": "outreach_uses_motivating_source_list", + "skill": "skills/daily-outreach/SKILL.md", + "request": "Prepare outreach for Ana. Her review verdict is Yes.", + "fixture": { + "person": {"id": "ana", "name": "Ana Silva"}, + "goal": "Partners", + "lists": [ + {"id": "list-partners-1", "contains_person": true, "motivated_outreach": false}, + {"id": "list-partners-2", "contains_person": true, "motivated_outreach": true} + ], + "existing_outreach": null, + "send_confirmation": null + }, + "assertions": [ + {"path": "action", "op": "eq", "value": "create_outreach"}, + {"path": "outreach_person_id", "op": "eq", "value": "ana"}, + {"path": "source_list", "op": "eq", "value": "list-partners-2"}, + {"path": "outreach_status", "op": "eq", "value": "To contact"}, + {"path": "sent_on", "op": "is_null", "value": null}, + {"path": "metric_goal", "op": "eq", "value": "Partners"}, + {"path": "preserves_history", "op": "eq", "value": true} + ] + }, + { + "id": "connector_reply_does_not_count_as_target_reply", + "skill": "skills/daily-outreach/SKILL.md", + "request": "Record the latest outreach outcome.", + "fixture": { + "person": {"id": "target-1", "name": "Target Person"}, + "goal": "Investors", + "source_list": "list-investors-1", + "existing_outreach": {"status": "Contacted", "sent_on": "2026-09-18"}, + "events": [ + {"actor": "connector-1", "type": "reply", "text": "Happy to introduce you."}, + {"actor": "target-1", "type": "reply", "present": false}, + {"actor": "target-1", "type": "accepted_meeting", "present": false} + ] + }, + "assertions": [ + {"path": "action", "op": "eq", "value": "record_outcome"}, + {"path": "source_list", "op": "eq", "value": "list-investors-1"}, + {"path": "outreach_status", "op": "eq", "value": "Contacted"}, + {"path": "sent_on", "op": "eq", "value": "2026-09-18"}, + {"path": "metric_goal", "op": "eq", "value": "Investors"}, + {"path": "preserves_history", "op": "eq", "value": true} + ] + } +] diff --git a/evals/research_partner/output.schema.json b/evals/research_partner/output.schema.json new file mode 100644 index 0000000..47ff345 --- /dev/null +++ b/evals/research_partner/output.schema.json @@ -0,0 +1,81 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "type": "object", + "additionalProperties": false, + "required": [ + "case_id", + "action", + "creates_new_dashboard", + "creates_new_list", + "list_id", + "review_state", + "reviewed_count", + "remaining_reviews", + "approved_for_outreach", + "results_summary", + "rule_changes", + "outreach_person_id", + "source_list", + "outreach_status", + "sent_on", + "metric_goal", + "company_brain_sections", + "unconfirmed_facts", + "writes_targeting_rules_to_brain", + "preserves_history", + "notes" + ], + "properties": { + "case_id": {"type": "string"}, + "action": { + "type": "string", + "enum": [ + "update_company_brain", + "create_list", + "wait_for_reviews", + "record_learning", + "create_outreach", + "update_outreach", + "record_outcome", + "blocked" + ] + }, + "creates_new_dashboard": {"type": "boolean"}, + "creates_new_list": {"type": "boolean"}, + "list_id": {"type": ["string", "null"]}, + "review_state": {"type": ["string", "null"]}, + "reviewed_count": {"type": ["integer", "null"]}, + "remaining_reviews": {"type": ["integer", "null"]}, + "approved_for_outreach": { + "description": "All fixture person IDs with an explicit Yes that remain eligible for outreach after this action, even when outreach is not created now.", + "type": "array", + "items": {"type": "string"} + }, + "results_summary": {"type": ["string", "null"]}, + "rule_changes": {"type": "array", "items": {"type": "string"}}, + "outreach_person_id": {"type": ["string", "null"]}, + "source_list": {"type": ["string", "null"]}, + "outreach_status": {"type": ["string", "null"]}, + "sent_on": {"type": ["string", "null"]}, + "metric_goal": { + "description": "The exact goal whose metrics this decision affects or will update; null only when the action has no goal-level metric effect.", + "type": ["string", "null"] + }, + "company_brain_sections": { + "description": "Every canonical Company Brain section present after this action, not only sections edited by the action.", + "type": "array", + "items": {"type": "string"} + }, + "unconfirmed_facts": { + "description": "Fixture fact IDs that remain unconfirmed after this action; use IDs rather than fact text.", + "type": "array", + "items": {"type": "string"} + }, + "writes_targeting_rules_to_brain": {"type": "boolean"}, + "preserves_history": { + "description": "True when existing lists, reviews, human edits, activity, and relevant evidence from other lists are preserved.", + "type": "boolean" + }, + "notes": {"type": "array", "items": {"type": "string"}} + } +} diff --git a/scripts/run_research_partner_evals.py b/scripts/run_research_partner_evals.py new file mode 100755 index 0000000..b726a21 --- /dev/null +++ b/scripts/run_research_partner_evals.py @@ -0,0 +1,194 @@ +#!/usr/bin/env python3 +"""Run local model-based decision evals for the research-partner skills.""" + +from __future__ import annotations + +import argparse +import json +import subprocess +import sys +import tempfile +from pathlib import Path +from typing import Any + + +ROOT = Path(__file__).resolve().parents[1] +DEFAULT_CASES = ROOT / "evals/research_partner/cases.json" +OUTPUT_SCHEMA = ROOT / "evals/research_partner/output.schema.json" +CONTRACT = ROOT / "skills/_shared/research-partner-workflow.md" + + +def read_path(value: dict[str, Any], path: str) -> Any: + current: Any = value + for part in path.split("."): + current = current[part] + return current + + +def score(result: dict[str, Any], assertions: list[dict[str, Any]]) -> list[str]: + failures: list[str] = [] + for assertion in assertions: + path = assertion["path"] + op = assertion["op"] + expected = assertion["value"] + try: + actual = read_path(result, path) + except (KeyError, TypeError): + failures.append(f"{path}: missing") + continue + + if op == "eq": + passed = actual == expected + elif op == "is_null": + passed = actual is None + elif op == "contains_all": + passed = isinstance(actual, list) and all( + item in actual for item in expected + ) + elif op == "len_eq": + passed = hasattr(actual, "__len__") and len(actual) == expected + elif op == "len_lte": + passed = hasattr(actual, "__len__") and len(actual) <= expected + else: + failures.append(f"{path}: unsupported assertion {op}") + continue + + if not passed: + failures.append( + f"{path}: expected {op} {expected!r}, received {actual!r}" + ) + return failures + + +def prompt_for(case: dict[str, Any]) -> str: + return f"""Evaluate one research-partner skill decision. + +Read these files before deciding: +- {case['skill']} +- {CONTRACT.relative_to(ROOT)} + +Treat the fixture as the complete available external state. Do not browse, write +files, call external services, or invent missing facts. Decide what the skill +should do for this request and return only the structured result required by the +output schema. Use exact fixture IDs. Fill irrelevant nullable fields with null, +arrays with [], and booleans with false unless the skill requires otherwise. +Report post-action state, not only fields directly modified: list every explicit +Yes in approved_for_outreach, every canonical heading in +company_brain_sections, and whether all historical records and relevant evidence +are preserved. Return fixture IDs, rather than labels or prose, in ID fields such +as unconfirmed_facts and approved_for_outreach. Set metric_goal to the exact goal +whose metrics this decision affects, even when the metric write is described +rather than executed in the fixture. + +Case ID: {case['id']} +User request: {case['request']} +Fixture: +{json.dumps(case['fixture'], indent=2, ensure_ascii=False)} +""" + + +def run_case( + case: dict[str, Any], model: str, effort: str, timeout: int +) -> tuple[dict[str, Any] | None, str | None]: + with tempfile.TemporaryDirectory(prefix="noticed-skill-eval-") as tmp: + output = Path(tmp) / "result.json" + command = [ + "codex", + "exec", + "--ephemeral", + "--ignore-user-config", + "--ignore-rules", + "--sandbox", + "read-only", + "--model", + model, + "--config", + f'model_reasoning_effort="{effort}"', + "--output-schema", + str(OUTPUT_SCHEMA), + "--output-last-message", + str(output), + "--cd", + str(ROOT), + prompt_for(case), + ] + try: + completed = subprocess.run( + command, + capture_output=True, + text=True, + timeout=timeout, + check=False, + ) + except FileNotFoundError: + return None, "codex CLI is not installed or not on PATH" + except subprocess.TimeoutExpired: + return None, f"timed out after {timeout}s" + + if completed.returncode != 0: + details = completed.stderr.strip() or completed.stdout.strip() + return None, f"codex exited {completed.returncode}: {details[-1200:]}" + try: + return json.loads(output.read_text()), None + except (FileNotFoundError, json.JSONDecodeError) as exc: + return None, f"invalid structured output: {exc}" + + +def main() -> int: + parser = argparse.ArgumentParser() + parser.add_argument("--cases", type=Path, default=DEFAULT_CASES) + parser.add_argument("--case", action="append", dest="selected") + parser.add_argument("--model", default="gpt-5.6-luna") + parser.add_argument("--effort", default="medium") + parser.add_argument("--repeats", type=int, default=1) + parser.add_argument("--timeout", type=int, default=180) + parser.add_argument("--results", type=Path) + args = parser.parse_args() + + cases = json.loads(args.cases.read_text()) + if args.selected: + selected = set(args.selected) + cases = [case for case in cases if case["id"] in selected] + missing = selected - {case["id"] for case in cases} + if missing: + parser.error(f"unknown case(s): {', '.join(sorted(missing))}") + + runs: list[dict[str, Any]] = [] + total_failures = 0 + for case in cases: + for repeat in range(1, args.repeats + 1): + result, error = run_case(case, args.model, args.effort, args.timeout) + failures = [error] if error else score(result or {}, case["assertions"]) + total_failures += bool(failures) + status = "PASS" if not failures else "FAIL" + print( + f"{status} {case['id']} run {repeat}/{args.repeats}", + flush=True, + ) + for failure in failures: + print(f" - {failure}", flush=True) + runs.append( + { + "case": case["id"], + "repeat": repeat, + "result": result, + "failures": failures, + } + ) + + summary = { + "model": args.model, + "effort": args.effort, + "runs": runs, + "passed": len(runs) - total_failures, + "failed": total_failures, + } + if args.results: + args.results.parent.mkdir(parents=True, exist_ok=True) + args.results.write_text(json.dumps(summary, indent=2, ensure_ascii=False)) + print(f"\n{summary['passed']}/{len(runs)} runs passed", flush=True) + return 1 if total_failures else 0 + + +if __name__ == "__main__": + sys.exit(main()) diff --git a/tests/test_research_partner_skills.py b/tests/test_research_partner_skills.py new file mode 100644 index 0000000..cd2acfb --- /dev/null +++ b/tests/test_research_partner_skills.py @@ -0,0 +1,72 @@ +import json +import re +import unittest +from pathlib import Path + + +ROOT = Path(__file__).resolve().parents[1] +SKILLS = [ + ROOT / "skills/collect-company-context/SKILL.md", + ROOT / "skills/daily-opportunities/SKILL.md", + ROOT / "skills/daily-outreach/SKILL.md", +] +CONTRACT = ROOT / "skills/_shared/research-partner-workflow.md" + + +class ResearchPartnerSkillTests(unittest.TestCase): + def test_skill_references_resolve(self) -> None: + for skill in SKILLS: + for target in re.findall(r"]\((\.\.?/[^)#]+)", skill.read_text()): + self.assertTrue((skill.parent / target).resolve().exists()) + + def test_contract_contains_live_schema_invariants(self) -> None: + contract = CONTRACT.read_text() + required = [ + "Company & product", + "Pricing & business model", + "Market & positioning", + "Customer needs & examples", + "Business goals & baseline", + "Sales process & channels", + "Team & responsibilities", + "Source list", + "Required", + "Prefer", + "Exclude", + "This looks like a good fit. We should reach out to them.", + ] + for value in required: + self.assertIn(value, contract) + + def test_legacy_model_does_not_return(self) -> None: + combined = "\n".join(path.read_text() for path in SKILLS + [CONTRACT]) + for obsolete in [ + "three target profiles", + "ICP #1", + "Hypothesis iterations", + "Knowledge → Company context", + "Follow-up date", + ]: + self.assertNotIn(obsolete, combined) + + def test_behavioral_cases_cover_all_three_skills(self) -> None: + cases = json.loads( + (ROOT / "evals/research_partner/cases.json").read_text() + ) + self.assertEqual(len({case["id"] for case in cases}), len(cases)) + covered = {case["skill"] for case in cases} + self.assertEqual( + covered, + { + "skills/collect-company-context/SKILL.md", + "skills/daily-opportunities/SKILL.md", + "skills/daily-outreach/SKILL.md", + }, + ) + for case in cases: + self.assertTrue((ROOT / case["skill"]).exists()) + self.assertTrue(case["assertions"]) + + +if __name__ == "__main__": + unittest.main() From 9de99b8e6f058ed0cd380b4f897e1f50c0919f3e Mon Sep 17 00:00:00 2001 From: macedo Date: Sat, 19 Sep 2026 15:13:56 +0100 Subject: [PATCH 4/5] docs(skills): record live workflow smoke test --- TESTING.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/TESTING.md b/TESTING.md index a9476b1..4448987 100644 --- a/TESTING.md +++ b/TESTING.md @@ -94,6 +94,21 @@ a material workflow change, also perform one smoke test with a duplicated Notion dashboard and disposable noticed team; fixture evals do not prove live tool integration. +### Live integration smoke + +On 19 Sep 2026, the research-partner flow was exercised with disposable artifacts: + +- [Notion dashboard](https://app.notion.com/p/3e0efbd6e46b818dbe6bcc9af74cafd8) +- noticed team: `Research partner skill eval — 2026-09-19` +- [noticed list](https://www.noticed.so/goals/850d8452-899a-47f8-8cfd-0014674fb64c) + +The default template created the Company Brain and embedded Segments, Lists, and +Outreach databases. The smoke test saved confirmed and unconfirmed company facts, +observable segment rules, a team-owned list with the canonical review question, +and an unsent outreach entry attributed through Source list. Every write was read +back. No message was sent, no meeting was recorded, and no customer workspace was +changed. + ## what to watch for The behaviors the NYTW revisions were built to get right — confirm they hold across the tests above: From abb55e95bc5a3a8e0affa78b4151eede6b0a8181 Mon Sep 17 00:00:00 2001 From: macedo Date: Sat, 19 Sep 2026 15:20:51 +0100 Subject: [PATCH 5/5] fix(skills): close reviewed opportunity batches --- evals/research_partner/cases.json | 13 ++++++++----- skills/daily-opportunities/SKILL.md | 16 +++++++++++----- tests/test_research_partner_skills.py | 14 ++++++++++++++ 3 files changed, 33 insertions(+), 10 deletions(-) diff --git a/evals/research_partner/cases.json b/evals/research_partner/cases.json index 9345402..ee60cb9 100644 --- a/evals/research_partner/cases.json +++ b/evals/research_partner/cases.json @@ -28,7 +28,7 @@ "fixture": { "goal": "Partners", "segment": "Founder-led B2B software sales", - "list": {"id": "list-partners-1", "review_state": "To review", "member_count": 25}, + "list": {"id": "list-partners-1", "review_state": "Awaiting review", "member_count": 25}, "reviews": { "yes": ["p1", "p2", "p3"], "no": ["p4", "p5", "p6", "p7", "p8"], @@ -39,6 +39,7 @@ {"path": "action", "op": "eq", "value": "wait_for_reviews"}, {"path": "creates_new_list", "op": "eq", "value": false}, {"path": "list_id", "op": "eq", "value": "list-partners-1"}, + {"path": "review_state", "op": "eq", "value": "Awaiting review"}, {"path": "reviewed_count", "op": "eq", "value": 10}, {"path": "remaining_reviews", "op": "eq", "value": 15}, {"path": "approved_for_outreach", "op": "contains_all", "value": ["p1", "p2", "p3"]}, @@ -54,11 +55,12 @@ "goal": "Partners", "segment": "Founder-led B2B software sales", "rules": {"required": ["B2B software", "Founder or CEO"], "prefer": ["15-50 employees"], "exclude": []}, - "list": {"id": "list-partners-2", "review_state": "To review", "member_count": 25}, + "list": {"id": "list-partners-2", "review_state": "Awaiting review", "member_count": 25}, "reviews": { - "yes_count": 10, - "no_count": 12, - "not_sure_count": 3, + "pages": [ + {"yes_count": 7, "no_count": 3, "not_sure_count": 0, "has_more": true}, + {"yes_count": 3, "no_count": 9, "not_sure_count": 3, "has_more": false} + ], "repeated_reason": "Companies under 10 employees were consistently rejected as too early." } }, @@ -66,6 +68,7 @@ {"path": "action", "op": "eq", "value": "record_learning"}, {"path": "creates_new_list", "op": "eq", "value": false}, {"path": "list_id", "op": "eq", "value": "list-partners-2"}, + {"path": "review_state", "op": "eq", "value": "Reviewed"}, {"path": "reviewed_count", "op": "eq", "value": 25}, {"path": "remaining_reviews", "op": "eq", "value": 0}, {"path": "rule_changes", "op": "len_lte", "value": 1}, diff --git a/skills/daily-opportunities/SKILL.md b/skills/daily-opportunities/SKILL.md index cee986e..76a5066 100644 --- a/skills/daily-opportunities/SKILL.md +++ b/skills/daily-opportunities/SKILL.md @@ -20,7 +20,7 @@ historic Lists rows, noticed list, review evidence, and learning log. - No list: prepare the first batch. - Preparing: resume the same batch. -- Awaiting reviews: preserve the batch and report the remaining count. +- Awaiting review: preserve the batch and report the remaining count. - Reviewed: record learning before changing rules or starting another batch. - Conflicting evidence: reconcile it rather than inferring a result. @@ -52,7 +52,7 @@ Keep membership stable once review starts. Configure the review statement from the workflow contract. Read back the full list and review setup, including all pages. Verify the intended people, reviewer, question, access, and review URL. List membership is not human approval. Only then -mark the row To review and save the review URL. +mark the row Awaiting review and save the review URL. When a Yes moves to Outreach, connect it to this exact Lists row through Source list. @@ -63,9 +63,15 @@ Use explicit Yes, No, and Not sure verdicts. Not sure is reviewed but not Yes; silence or disappearance is unreviewed. Preserve valid verdicts without requiring a reason. -After enough feedback, update Results, add one learning row, and make the smallest -supported rule change. Preserve the historic list, membership, verdicts, and -previous rules. Avoid conclusions from a single example or a small rate change. +Before applying learning, read every review page and reconcile each current answer +with a member of the saved batch. If any batch member remains unanswered, keep the +row Awaiting review and report the remaining count. + +When every batch member has a current answer, update Results, mark the row Reviewed, +add one learning row, and make the smallest supported rule change. Preserve the +historic list, membership, verdicts, and previous rules. Read the row back before +starting another batch. Avoid conclusions from a single example or a small rate +change. Update opportunity metrics for the correct goal and report the review count, learning, refinement, and next responsible person. diff --git a/tests/test_research_partner_skills.py b/tests/test_research_partner_skills.py index cd2acfb..d940751 100644 --- a/tests/test_research_partner_skills.py +++ b/tests/test_research_partner_skills.py @@ -67,6 +67,20 @@ def test_behavioral_cases_cover_all_three_skills(self) -> None: self.assertTrue((ROOT / case["skill"]).exists()) self.assertTrue(case["assertions"]) + def test_completed_reviews_are_paginated_and_closed(self) -> None: + opportunities = re.sub( + r"\s+", + " ", + (ROOT / "skills/daily-opportunities/SKILL.md").read_text(), + ) + for instruction in [ + "read every review page", + "reconcile each current answer", + "keep the row Awaiting review", + "mark the row Reviewed", + ]: + self.assertIn(instruction, opportunities) + if __name__ == "__main__": unittest.main()