releaseparser: adopt fixed positional model, auto-detect catalogues, add GB doc-blocks
Parse releases with an explicit fixed positional model and document the whole logic in British English doc-blocks. Model: - [0]=Artist, [1]=Title are fixed and never matched against dictionaries, so titles like "Split"/"Demo"/"De" are never eaten. - [len-1]=Group is always read from the line, not from the query key. - [len-2]=Year when it is a 4-digit 19xx|20xx token; otherwise the slot is resolved as a middle-zone token (e.g. FLAC still picked up as format). - The middle zone is resolved via case-insensitive dictionaries in the order format -> source -> bit-depth -> resolution -> edition -> language -> catalogue number. Position within the zone is irrelevant. Catalogue numbers: - Any "(...)" token in the middle zone is a catalogue number regardless of content (digits, roman numerals, dots, underscores, mixed case). Removed the hasDigit requirement and character whitelist that rejected real catalogues like (DYN.3140-2) and (GOWP_DCCCLXXXIX). - Splitting never breaks tokens inside parentheses. Case handling: lookups are case-insensitive; stored values keep their original case from the line. Documentation: replaced Ukrainian comments in parser.go and analyzer.go with British-English doc-blocks describing the model, tail resolution order, catalogue rule and confidence scoring. Tests: - Add TestParseCatalogNumberWithDotAndRoman for (DYN.3140-2)/(GOWP_DCCCLXXXIX). - Update case-insensitive, split-title, no-year and language expectations. - go build, go vet and go test all pass.
This commit is contained in:
@@ -55,5 +55,25 @@ func main() {
|
||||
log.Fatal(err)
|
||||
}
|
||||
|
||||
// Ручна ревізія: всі невідомі токени — у файл для правки бази.
|
||||
review, err := os.OpenFile("review_unknown_tokens.txt",
|
||||
os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0o644)
|
||||
if err != nil {
|
||||
log.Fatal(err)
|
||||
}
|
||||
defer review.Close()
|
||||
for releaser, stats := range out {
|
||||
for _, unk := range stats.UnknownTokens {
|
||||
for _, rel := range unk.Releases {
|
||||
fmt.Fprintf(review, "%s\t%s\t%s\n", releaser, unk.Token, rel)
|
||||
}
|
||||
}
|
||||
for _, warn := range stats.WarningStats {
|
||||
for _, rel := range warn.Releases {
|
||||
fmt.Fprintf(review, "%s\tWARN\t%s\t%s\n", releaser, warn.Token, rel)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fmt.Println(string(data))
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user