releaseparser: adopt fixed positional model, auto-detect catalogues, add GB doc-blocks
Parse releases with an explicit fixed positional model and document the whole logic in British English doc-blocks. Model: - [0]=Artist, [1]=Title are fixed and never matched against dictionaries, so titles like "Split"/"Demo"/"De" are never eaten. - [len-1]=Group is always read from the line, not from the query key. - [len-2]=Year when it is a 4-digit 19xx|20xx token; otherwise the slot is resolved as a middle-zone token (e.g. FLAC still picked up as format). - The middle zone is resolved via case-insensitive dictionaries in the order format -> source -> bit-depth -> resolution -> edition -> language -> catalogue number. Position within the zone is irrelevant. Catalogue numbers: - Any "(...)" token in the middle zone is a catalogue number regardless of content (digits, roman numerals, dots, underscores, mixed case). Removed the hasDigit requirement and character whitelist that rejected real catalogues like (DYN.3140-2) and (GOWP_DCCCLXXXIX). - Splitting never breaks tokens inside parentheses. Case handling: lookups are case-insensitive; stored values keep their original case from the line. Documentation: replaced Ukrainian comments in parser.go and analyzer.go with British-English doc-blocks describing the model, tail resolution order, catalogue rule and confidence scoring. Tests: - Add TestParseCatalogNumberWithDotAndRoman for (DYN.3140-2)/(GOWP_DCCCLXXXIX). - Update case-insensitive, split-title, no-year and language expectations. - go build, go vet and go test all pass.
This commit is contained in:
@@ -9,6 +9,7 @@ type GroupProfile struct {
|
||||
ResolutionTokens map[string]bool
|
||||
LanguageMarkers map[string]bool
|
||||
EditionTokens map[string]bool
|
||||
CommentTokens map[string]bool
|
||||
}
|
||||
|
||||
type ParsedRelease struct {
|
||||
@@ -18,6 +19,7 @@ type ParsedRelease struct {
|
||||
BitDepth string
|
||||
Resolution string
|
||||
Comment string
|
||||
CatalogNo string
|
||||
Source string
|
||||
Format string
|
||||
Language string
|
||||
@@ -44,6 +46,8 @@ type GroupStats struct {
|
||||
ResolutionDistribution map[string]int
|
||||
LanguageDistribution map[string]int
|
||||
EditionDistribution map[string]int
|
||||
CommentDistribution map[string]int
|
||||
UnknownTokens []UnknownTokenStats
|
||||
WarningStats []UnknownTokenStats
|
||||
AvgConfidence float64
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user