Add srrdb_log table with composite primary key (day, category) and counters
found, stored, ignored plus run_at.
Replace the release-only persistence path with UpsertAllSrrdbWithDay so releases
and their corresponding srrdb_log row are written in one SQLite transaction.
Preserve existing error handling: rollback reports the release/day that failed,
and commit errors return without writing a dump.
Handle fully ignored days as closed. When every fetched release is skipped by
ignore rules, still write the ledger row with zero stored releases. This prevents
the automatic day walk from endlessly retrying a day whose releases are filtered
out.
Derive and validate the ledger day from the resolved request URI. Missing or
invalid date: segments now fail before database writes. Keep run_at independent
from created_at semantics and continue populating created_at from the run time.
Preserve column and upsert semantics. category continues to use NOT NULL DEFAULT
empty string; status keeps COALESCE behavior on conflict; nfo, content, and
synced_at remain unchanged.
New scope reading the API instead of scraping HTML: one search path per run,
passed raw through --single-uri, and the answer stored as-is into its own
SQLite file created from the embedded schema.
Command line: --single-uri carries the path after /v1/search/
(category:flac/date:today/page:1) and must name a category; --sleep sets the
pause between pages, with the config value as fallback. A day named on the
command line is always fetched again and its dump rewritten, so cron can
refresh today hourly and yesterday daily.
Date placeholders: date:auto walks backwards to the newest day in
[srrdb.floor..yesterday] that has no dump yet, closing holes instead of
declaring the archive covered. date:today, date:yesterday and date:tomorrow
stand for the neighbouring days. Today is never taken by the walk, because a
day still being filled would be frozen by its own dump after the first run.
Dumps are the cursor as well as the record, so a missing dump directory is an
error instead of "nothing harvested yet": a renamed, unmounted or mistyped
directory must not send the run silently back to the beginning. An empty day is
a normal result and is remembered as harvested; API warnings never are, and
they reach the exit code.
Writes go through one transaction, so a day lands completely or not at all.
group and category come from the release line and the search path; status, nfo,
content and synced_at stay NULL. srrdb.log.jsonl records one line per finished
run, srrdb.error_log.jsonl keeps failures and API warnings apart by a kind
field.
Also: the console command contract returns (string, error), so a failing scope
reaches the process exit code and cron can tell a failed run from a successful
one. config/scope/srrdb.yaml is untracked because its paths differ per host.
modernc.org/sqlite replaces the archived libsql client, which only ever talked
to a server. pkg/repository/srrdb.go is split by responsibility and covered by
table-driven tests.
Yevhen Odynets