New scope reading the API instead of scraping HTML: one search path per run,
passed raw through --single-uri, and the answer stored as-is into its own
SQLite file created from the embedded schema.
Command line: --single-uri carries the path after /v1/search/
(category:flac/date:today/page:1) and must name a category; --sleep sets the
pause between pages, with the config value as fallback. A day named on the
command line is always fetched again and its dump rewritten, so cron can
refresh today hourly and yesterday daily.
Date placeholders: date:auto walks backwards to the newest day in
[srrdb.floor..yesterday] that has no dump yet, closing holes instead of
declaring the archive covered. date:today, date:yesterday and date:tomorrow
stand for the neighbouring days. Today is never taken by the walk, because a
day still being filled would be frozen by its own dump after the first run.
Dumps are the cursor as well as the record, so a missing dump directory is an
error instead of "nothing harvested yet": a renamed, unmounted or mistyped
directory must not send the run silently back to the beginning. An empty day is
a normal result and is remembered as harvested; API warnings never are, and
they reach the exit code.
Writes go through one transaction, so a day lands completely or not at all.
group and category come from the release line and the search path; status, nfo,
content and synced_at stay NULL. srrdb.log.jsonl records one line per finished
run, srrdb.error_log.jsonl keeps failures and API warnings apart by a kind
field.
Also: the console command contract returns (string, error), so a failing scope
reaches the process exit code and cron can tell a failed run from a successful
one. config/scope/srrdb.yaml is untracked because its paths differ per host.
modernc.org/sqlite replaces the archived libsql client, which only ever talked
to a server. pkg/repository/srrdb.go is split by responsibility and covered by
table-driven tests.
Generalise the parser: the MOONBLOOD group profile is gone, the parser and
its dictionaries are now group-agnostic under internal/releaseparser/groups.
Positional model:
- [0]=Artist, [1]=Title are fixed and never matched against dictionaries, so
titles like "Split"/"Demo"/"De" are never eaten.
- [len-1]=Group is always read from the line, not from the query key.
- [len-2]=Year when it is a 4-digit 19xx|20xx token; otherwise the slot is
resolved as a middle-zone token (e.g. FLAC is still picked up as format).
- The middle zone is resolved via case-insensitive dictionaries in the order
format -> source -> bit-depth -> resolution -> edition -> language ->
catalogue number; stored values keep their original case.
Catalogue numbers: any "(...)" token in the middle zone is a catalogue number
regardless of content (digits, roman numerals, dots, underscores, mixed case);
splitting never breaks tokens inside parentheses.
Dictionaries moved to internal/releaseparser/groups/resources/dicts and
embedded via go:embed; dead profile fields (Name, ExpectedTailOrder) removed;
profile comments rewritten in British English.
File-based AnalyzeGroup/TestParseMoonblood removed — titles now come from the
database (cmd/srrdb: srrdb <releaser>..., config/secret.yaml DSN).
Validation output (cmd/srrdb):
- tmp/releaseparser/<group>.json — per-group parsed ParsedRelease records
- tmp/releaseparser/<releaser>.report.json — per-request aggregate (same as console)
- review_unknown_tokens.txt — unknown/warning rows for manual review
Tests updated for the positional model; added
TestParseCatalogNumberWithDotAndRoman for (DYN.3140-2) and (GOWP_DCCCLXXXIX).
go build, go vet and go test all pass.
Yevhen Odynets
yevhen