Parser Improvements and newly labeled speeches - #90
Open
jutton1 wants to merge 8 commits into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Sorry about the delay with the PR... was busier than intended.
I also spent around 30 min a day hand labeling starts of speeches within sections of the record.
Both the old and new parses were run over the whole corpus and compared over 858,732 documents
29.7% of speech items (799,542) get their speaker from the fallback rather than from
mods.xml, but for proof of concept, I reparsed without asmods.xmlwas hiding some of the failings of the old parser.mods.xmlThe hand auditing was 250 windows of 40 lines each. 150 were centred on an item the
change moves, 100 from documents that parse identically. I did do this blind, so i had no clue which was which.
Metadata shows I spent around 182 minutes on hand labeling. I have placed the hand labeled in tests, I'll wire in something to utilize it later.
Exact match means the rate at which the old and new parser matched what I hand labeled.
Precision is the rate at which i matched what the parser stated.
Speaker IoU is exact match over union of parser + human
closes #36 , closes #77, closes #79 , closes #80 , closes #81 , closes #82 , closes #83 , closes #84 , closes #85 , closes #86 , closes #87 , closes #88 , closes #89