Data on disk

The data folder and the database connection#

Source core/rymembra-app/src/db/mod.rs core/rymembra-app/src/trasloco.rs apps/desktop/src-tauri/src/stato.rs apps/desktop/src-tauri/src/main.rs apps/desktop/src-tauri/msix/LEGGIMI.md

Everything Rymembra keeps lives in the application data folder, %APPDATA%\io.github.federicobaratti.rymembra on Windows (asked to Tauri, never composed by hand). The environment variable RYMEMBRA_DATI replaces it at startup — the way to try an import or a migration on a copy without touching the real library; it also disables the single-instance guard.

A copy from the Microsoft Store asks for the same folder, and Windows (1903 and later) redirects by itself the files it creates there to the package's private folder under %LOCALAPPDATA%\Packages\, which uninstalling the package removes. Files that already exist are used where they are: on a PC where the installer copy has already made a library, the Store copy would open that rymembra.db while its new -wal and -shm went to the private folder. That has not been tried, which is why the site says to keep one copy per PC; keychain entries are not virtualized, so connected accounts are the same for both.

Coming from the previous version. On the first start the data of the previous version are transferred automatically, before the log or the database is opened (trasloco.rs, only for the default folder): the old data folder is renamed to this one — library, settings, covers, skins, drafts, profile copies and log files, which take the new names — and an interrupted transfer resumes at the next start. Nothing is overwritten and nothing is deleted: if the new folder already holds a library, it wins and the old one is left untouched; if the previous version is still running (its database is open), nothing is moved and Rymembra says so in a dialog and closes. Skins and profile archives written by the previous version still import, and its keychain entries and Drive files are picked up on first use (OAuth, Drive backup).

Opening

db::open runs, in order: connection pragmas, an FTS5 probe (create and drop a temporary FTS5 table), PRAGMA user_version, the pre-migration snapshot when 0 < version < LATEST_VERSION, the migrations, and PRAGMA optimize=0x10002. Paths longer than 240 characters are opened in their \\?\ verbatim form (files::percorso_lungo). If opening fails, Stato keeps the AppError instead of the connection, and every command returns it — the window still opens and can show the path and the reason.

Pragmas and the journal

1. busy_timeout = 5000 ms (BUSY_TIMEOUT_MS) — first, so the WAL probe below does not fail on a concurrent writer.
2. foreign_keys = ON — the cascades that remove playlist rows and history when a track goes.
3. journal_mode = WAL, then a probe: an empty BEGIN IMMEDIATE; COMMIT; (≈ 5.5 µs), because on an SMB share the pragma often answers wal and the -shm file only fails at the first transaction. SQLITE_BUSY/LOCKED count as a pass. On success: synchronous = NORMAL.
4. If WAL is not usable: fall back to journal_mode = TRUNCATE with synchronous = FULL. The open never fails because of the journal; the startup log line says giornale=wal|truncate|memory|<mode>-non-usabile.

With the fallback, readers and writers serialize, so the UI stutters during scans; the data folder is never moved automatically to get out of the fallback.

Connections

One main connection, behind the library mutex (Runtime). Two secondary connections to the same file — the enrichment response cache and the Folders tree index — get the same 5 s busy timeout and foreign keys via prepara_connessione_secondaria; they inherit the journal from the file and never migrate. Writes that read first use TransactionBehavior::Immediate (library::transazione_di_scrittura) to avoid SQLITE_BUSY_SNAPSHOT.

SQLite errors

codice_da_sqlite maps SQLite result codes to the catalog and appends the extended code to the cause:

See also: The schema (version 35), Migrations and the pre-migration copy, Health check, optimize and cover cleanup, Music on a NAS or a mapped drive

The schema (version 35)#

Source core/rymembra-app/src/db/schema core/rymembra-app/src/db/schema/001_baseline.sql core/rymembra-app/src/db/schema/026_brani_di_catalogo.sql core/rymembra-app/src/db/schema/030_libreria_leggera.sql core/rymembra-app/src/db/schema/035_insieme.sql

Times are milliseconds since the epoch, durations milliseconds. There are no views. The central table is tracks; almost everything else hangs off tracks.id (with ON DELETE CASCADE) or off a stable key.

tracks

One row per audio file or per catalog stream. Since migration 026 a CHECK enforces exactly one of the two shapes: a file row has path, file_size, date_modified and no catalog columns; a catalog row has path = NULL and non-null source_service, fonte_url, licenza, disponibilita. idx_tracks_riferimento makes (source_service, fonte_url) unique.

Triggers on tracks: the three tracks_fts_* keep the FTS index current; statistiche_restano_orfane / correzioni_restano_orfane (BEFORE DELETE) and statistiche_tornano / correzioni_tornano (AFTER INSERT) park and restore statistics and corrections by content_key for 90 days — see Removal; lyrics_segue_track_key moves lyrics when the key changes; the three generazione_dopo_* write a new random() into libreria_stato.generazione on every write.

Everything else

See also: Migrations and the pre-migration copy, Track identity: keys, folding and paths

Migrations and the pre-migration copy#

Source core/rymembra-app/src/db/migrations.rs core/rymembra-app/src/db/mod.rs core/rymembra-app/src/db/manutenzione.rs CHANGELOG.md

MIGRATIONS is an ordered list of 35 Migration { version, name, sql, ricostruisce }, each SQL file embedded with include_str!. LATEST_VERSION is computed from the list in a const block; a test checks that versions are consecutive from 1 and that every .sql file is listed.

Forward only

There are no down migrations. A database whose user_version is greater than LATEST_VERSION is refused with db.versionAhead (fatal) rather than opened with queries that ignore half its columns. Going back to an older Rymembra means restoring the copy below.

One transaction per migration

Each runs as BEGIN; <sql>; PRAGMA user_version = N; COMMIT. If the third fails, the first two stay applied and user_version says so — one transaction for all would turn a partial upgrade into one that never starts. A failure is db.migrationFailed { from, to, step }.

Migration 026 is the only one with ricostruisce: true: it rebuilds tracks to drop a NOT NULL. DROP TABLE with foreign keys on would fire every ON DELETE CASCADE into tracks, so foreign_keys is switched off before BEGIN (inside a transaction the pragma is ignored), PRAGMA foreign_key_check must find nothing before COMMIT, and afterwards the code verifies that foreign keys are back on.

The copy taken before migrating

When there is something to migrate and something to protect (0 < version < LATEST_VERSION, a file database), prima_di_migrare runs:

1. if rymembra.db.schema-NNN.bak (NNN = the version the copy contains) already exists, keep it — it predates every failed attempt;
2. estimate the copy as used pages × page size;
3. reserve that size plus a margin (half the database, at least MARGINE_MINIMO = 32 MiB) by extending a temporary file with set_len, so a full disk fails now and not halfway;
4. VACUUM INTO <name>.parziale, then rename: a file with the final name is always complete;
5. keep the ISTANTANEE_TENUTE = 2 newest by modification time.

The copy never blocks the migration: if it cannot be made, the result is Istantanea::Saltata { motivo } and the reason goes to the startup log. Every release that migrates names its migrations in CHANGELOG.md.

See also: The schema (version 35), Health check, optimize and cover cleanup

Health check, optimize and cover cleanup#

Source core/rymembra-app/src/db/manutenzione.rs apps/desktop/src-tauri/src/manutenzione.rs core/rymembra-app/src/covers.rs apps/desktop/src/parti/Avvisi.tsx

The thread rymembra-manutenzione uses the periodic template with ATTESA_AVVIO = 60 s, RAFFICA = 10 s and INTERVALLO = 6 h. It is also woken by every scan that ends (the scan's TurnoDiScansione sends the signal when it drops).

The quick check

db::controlla_il_file opens its own connection without SQLITE_OPEN_CREATE (a missing file must not be created and declared healthy) and with query_only, so a long read never holds the library lock. A watchdog thread (rymembra-scadenza-sqlite) calls sqlite3_interrupt after TEMPO_DEL_CONTROLLO = 30 s. Outcomes: Sano, Interrotto (unknown, not damage), or Err(db.corrupt) with the first 8 lines of the answer. The check runs only in WAL mode: with the TRUNCATE fallback a long reader would block every writer, so it is skipped and reported as such.

The result (daFare, sano, interrotto, saltato, danneggiato, nonRiuscito) is kept in StatoManutenzione, emitted as manutenzione:salute, and readable at any time with manutenzione_salute so a reloaded window can still show the damaged-database banner. After danneggiato the thread stops writing: no optimize, no cleanup.

Cover cleanup

When covers::quante_orfane is non-zero, pulisci takes the scan turn (from the door that does not wake anyone) and the enrichment turn — both save cover files outside the lock and register them later, so a cleanup in between could delete a file about to be referenced. Under the lock it deletes orphan cover_art rows; without the lock it deletes their files and leftovers older than ETA_DEGLI_AVANZI = 24 h. manutenzione_stato reports cache size, orphan count and health; manutenzione_pulisci_copertine runs the cleanup on demand and returns library.scanBusy / metadata.enrichBusy when it has to wait.

See also: Covers, The data folder and the database connection

Settings and preferences#

Source core/rymembra-app/src/settings.rs core/rymembra-app/src/preferenze.rs core/rymembra-app/src/playback.rs core/rymembra-app/src/profilo/mod.rs

All preferences are rows of the settings table (key TEXT PRIMARY KEY, value TEXT). settings::read returns Ok(None) for a missing key — the normal state of a first start; the caller decides what absence means. read_json / write_json store typed values. Preferences that once lived in the WebView's localStorage (theme, shortcuts, language) moved here so that backup and profile carry them.

Secrets never go in this table: rymembra.db is readable by any process of the user and is copied by backups. Tokens and keys are in the OS keychain.

State keys also live here: enrich.ultimo_ms, aggiornamenti.ultimo_ms / .saltata / .trovata, nuvola.*, sincronia.*, settimana.ultimo_lunedi, profilo.identita, sync.roots, ia.profili / ia.profilo, scrobble.*.utente, and the tray labels vassoio.*.

See also: The profile archive (.ryprofile), Design principles

The profile archive (.ryprofile)#

Source core/rymembra-app/src/profilo/mod.rs core/rymembra-app/src/profilo/archivio.rs core/rymembra-app/src/profilo/biblioteca.rs core/rymembra-app/src/profilo/radici.rs apps/desktop/src-tauri/src/profilo.rs apps/desktop/src/parti/Profilo.tsx

A profile is a single file you carry by hand: preferences and the library's irreplaceable data. It overlaps with the Drive backup on purpose — the backup is automatic and needs an account; the profile is a gesture and needs nothing.

Format

A zip with signature rymembra.profilo, format version 2, extension .ryprofile. Format 1 (a JSON of preferences) is still read — recognised by its first byte { against PK\x03\x04 — and no longer written. Entry names form a closed list; any other name rejects the whole archive (no path ever needs normalizing, so ../ cannot exist).

JSON entries are deflated; already-compressed entries are stored. Reading and writing stream one entry at a time — no archive is ever held in memory. Limits (archivio::limiti), checked from the zip index: archive 2 GiB, 200,000 entries, 64 MiB per entry, 4 GiB total uncompressed, compression ratio ≤ 200; above 500 MB the export warns.

What travels

Preferences use an inclusion list, CATALOGO: a new key travels only once someone adds it, so a machine-specific key such as nuvola.dispositivo can never leak by default. Left out on purpose: the queue (row ids), player.output, player.spectrum.quality, audio.latenza_ms, audio.latenza_per_uscita, the insieme.* keys and the insieme_pari table, ui.zoom, the Folders tree state, the Last.fm API key, and all derived data (affinity, fingerprints, loudness, weekly, enrich_undo, track_meta_arricchita).

Import

profilo_piano and profilo_importa run the same code (esegui): the plan is the import inside a transaction that is rolled back. Steps:

1. sincronia::allinea brings the sync tables up to date (its own transaction);
2. identity check (profilo.identita): a library with no identity adopts the profile's; a different identity gets only preferences, covers and skins, unless the user says it is the same library (unisci_comunque);
3. outside the lock, path answers are computed: each remapped root is checked with a 3 s timeout, and for each remapping the plan samples CAMPIONE_DEL_PIANO = 64 files (the real import checks them all) with three answers — exists, missing, unknown;
4. in one Immediate transaction: root remapping (tracks.path is rewritten only where the new file exists), preferences, corrections first, the sync merge (rymembra_sync::fondi of local and imported documents), then the library complement — history INSERT OR IGNORE, newest correction and lyrics win (never over hand-made lyrics), covers attached only to tracks without one.

Nothing is ever deleted by an import: no tombstone from the profile is applied. Before writing, a safety copy goes to profilo/prima-<ms>.ryprofile (COPIE_TENUTE = 3); profilo_annulla restores preferences and the active skin from it — the additive library part cannot be undone. All four commands run in in_disparte, since a profile usually sits on a USB stick or a share; progress is the profilo:avanzamento event.

See also: Device sync, Backup on Google Drive, Settings and preferences