Yevhen Odynets
347f525801
Three-level core resolution with priority overrides → dictionaries/model → suspects: 1. dicts/overrides.txt (highest priority) Point fixes via exact lookup on Raw BEFORE the positional logic. Format: <original line> => artist | title. An override fixes Artist/Title and shifts the dictionary tail zone by the actual core length (overrideCoreLength, fallback 2 tokens). 2. dicts/artists.txt Whitelist of hyphenated artists (Iku-Turso, Post-Mortem, ...): a [0]-[1] hit → Artist = 2 tokens, Title = [2], tail zone shifted. One entry covers all releases of the artist automatically. 3. Suspected title continuation Unknown tokens between the title zone and the tail (followed later by dictionary markers) no longer pollute UnknownTokens; they go to a dedicated TitleExtra field + warning "suspected title continuation" — a reviewer hint that the core is probably longer than 2 tokens. Catalogue numbers (parenthesized tokens) are resolved with FIRST priority and never land in suspects. Type field (normalized base release type): - new normalization map dicts/types.txt (<Base> = variants): EP = EP, CDS, CDM, 7INCH_VINYL, CDEP, ..., REMASTERED_EP; Demo = ...; Split; Single; Compilation; Collaboration (curator-extendable) - tokens matched against edition/source are additionally normalized to the base type (first match wins); edition.txt/source.txt untouched, Source stays raw - TypeDistribution map[string]int added to GroupStats and the report (console/JSON) - OST/INSTRUMENTAL are release traits, not types; excluded from the map Tests: old cases updated to the new suspect semantics; acceptance tests added for overrides (Milicia_Oscura-Anti-Cristo), artist dictionary (Iku-Turso), the suspect section, Type normalization (CDEP → EP with raw Source=CDEP, first-match-wins) and TypeDistribution. go build / go vet / go test ./internal/releaseparser/... — all green.
64 lines
1.5 KiB
Go
64 lines
1.5 KiB
Go
package releaseparser
|
|
|
|
type Override struct {
|
|
Artist string
|
|
Title string
|
|
}
|
|
|
|
type GroupProfile struct {
|
|
ArtistTokens map[string]bool // hyphenated artist whitelist
|
|
TypeVariants map[string]string // variant (upper) -> normalized Type
|
|
Overrides map[string]Override
|
|
FormatTokens map[string]bool
|
|
SourceTokens map[string]bool
|
|
BitDepthTokens map[string]bool
|
|
ResolutionTokens map[string]bool
|
|
LanguageMarkers map[string]bool
|
|
EditionTokens map[string]bool
|
|
CommentTokens map[string]bool
|
|
}
|
|
|
|
type ParsedRelease struct {
|
|
Raw string
|
|
Group string
|
|
Artist string
|
|
Year int
|
|
Title string
|
|
TitleExtra []string
|
|
Type string
|
|
Source string
|
|
Edition string
|
|
CatalogNo string
|
|
Language string
|
|
BitDepth string
|
|
Resolution string
|
|
Comment string
|
|
Format string
|
|
Core string
|
|
Override Override
|
|
UnknownTokens []string
|
|
Warnings []string
|
|
Confidence float64
|
|
}
|
|
|
|
type UnknownTokenStats struct {
|
|
Token string
|
|
Count int
|
|
Releases []string
|
|
}
|
|
|
|
type GroupStats struct {
|
|
Total int
|
|
TypeDistribution map[string]int
|
|
YearDistribution map[int]int
|
|
SourceDistribution map[string]int
|
|
BitDepthDistribution map[string]int
|
|
ResolutionDistribution map[string]int
|
|
LanguageDistribution map[string]int
|
|
EditionDistribution map[string]int
|
|
CommentDistribution map[string]int
|
|
UnknownTokens []UnknownTokenStats
|
|
WarningStats []UnknownTokenStats
|
|
AvgConfidence float64
|
|
}
|