releaseparser: move group-agnostic parser to groups/, GB doc-blocks, per-group JSON reports

Generalise the parser: the MOONBLOOD group profile is gone, the parser and
its dictionaries are now group-agnostic under internal/releaseparser/groups.

Positional model:
- [0]=Artist, [1]=Title are fixed and never matched against dictionaries, so
  titles like "Split"/"Demo"/"De" are never eaten.
- [len-1]=Group is always read from the line, not from the query key.
- [len-2]=Year when it is a 4-digit 19xx|20xx token; otherwise the slot is
  resolved as a middle-zone token (e.g. FLAC is still picked up as format).
- The middle zone is resolved via case-insensitive dictionaries in the order
  format -> source -> bit-depth -> resolution -> edition -> language ->
  catalogue number; stored values keep their original case.

Catalogue numbers: any "(...)" token in the middle zone is a catalogue number
regardless of content (digits, roman numerals, dots, underscores, mixed case);
splitting never breaks tokens inside parentheses.

Dictionaries moved to internal/releaseparser/groups/resources/dicts and
embedded via go:embed; dead profile fields (Name, ExpectedTailOrder) removed;
profile comments rewritten in British English.

File-based AnalyzeGroup/TestParseMoonblood removed — titles now come from the
database (cmd/srrdb: srrdb <releaser>..., config/secret.yaml DSN).

Validation output (cmd/srrdb):
- tmp/releaseparser/<group>.json         — per-group parsed ParsedRelease records
- tmp/releaseparser/<releaser>.report.json — per-request aggregate (same as console)
- review_unknown_tokens.txt              — unknown/warning rows for manual review

Tests updated for the positional model; added
TestParseCatalogNumberWithDotAndRoman for (DYN.3140-2) and (GOWP_DCCCLXXXIX).
go build, go vet and go test all pass.
This commit is contained in:
Yevhen Odynets
2026-09-01 11:28:19 +03:00
parent feef080d80
commit f2629f002c
15 changed files with 150 additions and 293 deletions
+45
View File
@@ -0,0 +1,45 @@
package groups
import (
"embed"
"strings"
"git.kplus.net.ua/yevhen/resource-scraper/internal/releaseparser"
)
// The token dictionaries live under resources/dicts/*.txt — one token (in any
// case) per line; blank lines and lines starting with "#" are ignored. They
// can be edited without recompiling the parsing logic: go:embed pulls them
// into the binary at build time.
//go:embed resources/dicts/*.txt
var dictFS embed.FS
func loadDict(name string) map[string]bool {
data, err := dictFS.ReadFile("resources/dicts/" + name + ".txt")
if err != nil {
panic("release parser: cannot load dictionary " + name + ": " + err.Error())
}
out := make(map[string]bool)
for line := range strings.SplitSeq(string(data), "\n") {
line = strings.ToUpper(strings.TrimSpace(line))
if line == "" || strings.HasPrefix(line, "#") {
continue
}
out[line] = true
}
return out
}
// Profile holds the shared dictionaries for the release parser. The parser is
// group-agnostic: the group name is read from each release line, never from
// this profile, so there is a single set of dictionaries covering all groups.
var Profile = releaseparser.GroupProfile{
FormatTokens: loadDict("format"),
SourceTokens: loadDict("source"),
BitDepthTokens: loadDict("bitdepth"),
ResolutionTokens: loadDict("resolution"),
LanguageMarkers: loadDict("language"),
EditionTokens: loadDict("edition"),
CommentTokens: loadDict("comment"),
}