Yevhen Odynets
f2629f002c
Generalise the parser: the MOONBLOOD group profile is gone, the parser and its dictionaries are now group-agnostic under internal/releaseparser/groups. Positional model: - [0]=Artist, [1]=Title are fixed and never matched against dictionaries, so titles like "Split"/"Demo"/"De" are never eaten. - [len-1]=Group is always read from the line, not from the query key. - [len-2]=Year when it is a 4-digit 19xx|20xx token; otherwise the slot is resolved as a middle-zone token (e.g. FLAC is still picked up as format). - The middle zone is resolved via case-insensitive dictionaries in the order format -> source -> bit-depth -> resolution -> edition -> language -> catalogue number; stored values keep their original case. Catalogue numbers: any "(...)" token in the middle zone is a catalogue number regardless of content (digits, roman numerals, dots, underscores, mixed case); splitting never breaks tokens inside parentheses. Dictionaries moved to internal/releaseparser/groups/resources/dicts and embedded via go:embed; dead profile fields (Name, ExpectedTailOrder) removed; profile comments rewritten in British English. File-based AnalyzeGroup/TestParseMoonblood removed — titles now come from the database (cmd/srrdb: srrdb <releaser>..., config/secret.yaml DSN). Validation output (cmd/srrdb): - tmp/releaseparser/<group>.json — per-group parsed ParsedRelease records - tmp/releaseparser/<releaser>.report.json — per-request aggregate (same as console) - review_unknown_tokens.txt — unknown/warning rows for manual review Tests updated for the positional model; added TestParseCatalogNumberWithDotAndRoman for (DYN.3140-2) and (GOWP_DCCCLXXXIX). go build, go vet and go test all pass.
52 lines
1.2 KiB
Go
52 lines
1.2 KiB
Go
package releaseparser
|
|
|
|
type GroupProfile struct {
|
|
FormatTokens map[string]bool
|
|
SourceTokens map[string]bool
|
|
BitDepthTokens map[string]bool
|
|
ResolutionTokens map[string]bool
|
|
LanguageMarkers map[string]bool
|
|
EditionTokens map[string]bool
|
|
CommentTokens map[string]bool
|
|
}
|
|
|
|
type ParsedRelease struct {
|
|
Raw string
|
|
Group string
|
|
Artist string
|
|
Year int
|
|
Title string
|
|
Source string
|
|
Edition string
|
|
CatalogNo string
|
|
Language string
|
|
BitDepth string
|
|
Resolution string
|
|
Comment string
|
|
Format string
|
|
Core string
|
|
UnknownTokens []string
|
|
Warnings []string
|
|
Confidence float64
|
|
}
|
|
|
|
type UnknownTokenStats struct {
|
|
Token string
|
|
Count int
|
|
Releases []string
|
|
}
|
|
|
|
type GroupStats struct {
|
|
Total int
|
|
YearDistribution map[int]int
|
|
SourceDistribution map[string]int
|
|
BitDepthDistribution map[string]int
|
|
ResolutionDistribution map[string]int
|
|
LanguageDistribution map[string]int
|
|
EditionDistribution map[string]int
|
|
CommentDistribution map[string]int
|
|
UnknownTokens []UnknownTokenStats
|
|
WarningStats []UnknownTokenStats
|
|
AvgConfidence float64
|
|
}
|