Skip to content

fix: correct Lesson 6.1 grading regex syntax error #110

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
gn00295120 wants to merge 1 commit into
base: master
Choose a base branch
from
Open

fix: correct Lesson 6.1 grading regex syntax error #110

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 2 additions & 55 deletions ...engineering_interactive_tutorial/Anthropic 1P/06_Precognition_Thinking_Step_by_Step.ipynb
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,60 +216,7 @@
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Prompt template with a placeholder for the variable content\n",
"PROMPT = \"\"\"Please classify this email as either green or blue: {email}\"\"\"\n",
"\n",
"# Prefill for Claude's response, if any\n",
"PREFILL = \"\"\n",
"\n",
"# Variable content stored as a list\n",
"EMAILS = [\n",
" \"Hi -- My Mixmaster4000 is producing a strange noise when I operate it. It also smells a bit smoky and plasticky, like burning electronics. I need a replacement.\", # (B) Broken or defective item\n",
" \"Can I use my Mixmaster 4000 to mix paint, or is it only meant for mixing food?\", # (A) Pre-sale question OR (D) Other (please explain)\n",
" \"I HAVE BEEN WAITING 4 MONTHS FOR MY MONTHLY CHARGES TO END AFTER CANCELLING!! WTF IS GOING ON???\", # (C) Billing question\n",
" \"How did I get here I am not good with computer. Halp.\" # (D) Other (please explain)\n",
"]\n",
"\n",
"# Correct categorizations stored as a list of lists to accommodate the possibility of multiple correct categorizations per email\n",
"ANSWERS = [\n",
" [\"B\"],\n",
" [\"A\",\"D\"],\n",
" [\"C\"],\n",
" [\"D\"]\n",
"]\n",
"\n",
"# Dictionary of string values for each category to be used for regex grading\n",
"REGEX_CATEGORIES = {\n",
" \"A\": \"A\\) P\",\n",
" \"B\": \"B\\) B\",\n",
" \"C\": \"C\\) B\",\n",
" \"D\": \"D\\) O\"\n",
"}\n",
"\n",
"# Iterate through list of emails\n",
"for i,email in enumerate(EMAILS):\n",
" \n",
" # Substitute the email text into the email placeholder variable\n",
" formatted_prompt = PROMPT.format(email=email)\n",
" \n",
" # Get Claude's response\n",
" response = get_completion(formatted_prompt, prefill=PREFILL)\n",
"\n",
" # Grade Claude's response\n",
" grade = any([bool(re.search(REGEX_CATEGORIES[ans], response)) for ans in ANSWERS[i]])\n",
" \n",
" # Print Claude's response\n",
" print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n",
" print(\"USER TURN\")\n",
" print(formatted_prompt)\n",
" print(\"\\nASSISTANT TURN\")\n",
" print(PREFILL)\n",
" print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n",
" print(response)\n",
" print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n",
" print(\"This exercise has been correctly solved:\", grade, \"\\n\\n\\n\\n\\n\\n\")"
]
"source": "# Prompt template with a placeholder for the variable content\nPROMPT = \"\"\"Please classify this email as either green or blue: {email}\"\"\"\n\n# Prefill for Claude's response, if any\nPREFILL = \"\"\n\n# Variable content stored as a list\nEMAILS = [\n \"Hi -- My Mixmaster4000 is producing a strange noise when I operate it. It also smells a bit smoky and plasticky, like burning electronics. I need a replacement.\", # (B) Broken or defective item\n \"Can I use my Mixmaster 4000 to mix paint, or is it only meant for mixing food?\", # (A) Pre-sale question OR (D) Other (please explain)\n \"I HAVE BEEN WAITING 4 MONTHS FOR MY MONTHLY CHARGES TO END AFTER CANCELLING!! WTF IS GOING ON???\", # (C) Billing question\n \"How did I get here I am not good with computer. Halp.\" # (D) Other (please explain)\n]\n\n# Correct categorizations stored as a list of lists to accommodate the possibility of multiple correct categorizations per email\nANSWERS = [\n [\"B\"],\n [\"A\",\"D\"],\n [\"C\"],\n [\"D\"]\n]\n\n# Dictionary of string values for each category to be used for regex grading\n# Fixed: Using raw strings (r prefix) to properly escape backslashes for regex\nREGEX_CATEGORIES = {\n \"A\": r\"A\\) P\",\n \"B\": r\"B\\) B\",\n \"C\": r\"C\\) B\",\n \"D\": r\"D\\) O\"\n}\n\n# Iterate through list of emails\nfor i,email in enumerate(EMAILS):\n \n # Substitute the email text into the email placeholder variable\n formatted_prompt = PROMPT.format(email=email)\n \n # Get Claude's response\n response = get_completion(formatted_prompt, prefill=PREFILL)\n\n # Grade Claude's response\n grade = any([bool(re.search(REGEX_CATEGORIES[ans], response)) for ans in ANSWERS[i]])\n \n # Print Claude's response\n print(\"--------------------------- Full prompt with variable substutions ---------------------------\")\n print(\"USER TURN\")\n print(formatted_prompt)\n print(\"\\nASSISTANT TURN\")\n print(PREFILL)\n print(\"\\n------------------------------------- Claude's response -------------------------------------\")\n print(response)\n print(\"\\n------------------------------------------ GRADING ------------------------------------------\")\n print(\"This exercise has been correctly solved:\", grade, \"\\n\\n\\n\\n\\n\\n\")"
},
{
"cell_type": "markdown",
Expand Down Expand Up @@ -493,4 +440,4 @@
},
"nbformat": 4,
"nbformat_minor": 2
}
}
Loading