Use Arkilian
in your language.

# Task: integrate Arkilian into this project

Arkilian is an embedded SQLite database with built-in replication. Your code keeps using ordinary SQL. Arkilian captures every write and streams it to S3-compatible object storage (SHA-256-verified chunks plus periodic snapshots), so a new machine can restore the database from the bucket.

Your job: replace this project's SQLite access with Arkilian, configure storage, and prove replication works. Reference docs: https://arkilian.com/documentation

## Ground rules (read first)
1. Make the smallest change that works. Do not rewrite unrelated code. Do not change the database schema unless a step below says so.
2. If a step fails, or this document disagrees with what you see (package not found, different method names, unsupported OS), STOP and tell the user exactly what happened. Do not improvise: do not swap in another database library, invent credentials, fake a verification result, or skip a step.
3. Secrets (S3 keys, HMAC key) live only in .env or the environment. Never hardcode, print, log or commit them. Ask the user for the real values.
4. Before the first run against an existing database file, copy it (for example: cp app.sqlite app.sqlite.bak). Opening it with Arkilian adds capture triggers (named trg_...) and internal tables (_pending_backup, _dead_backup, _arkilian_meta).

## Step 1. Inspect before changing anything
Find and write down:
a) the language and package manager;
b) every place SQLite is opened (better-sqlite3, sqlite3, database/sql with a SQLite driver, rusqlite, PDO, ORMs such as Prisma, Drizzle, TypeORM, SQLAlchemy);
c) every place the schema is created or migrated;
d) every table that has no PRIMARY KEY;
e) whether more than one process opens the same database file (cluster mode, multiple workers, per-request processes).
Then give the user a plan in 3 to 5 lines and continue.

## Step 2. Install (use ONLY the entry for your language)
- Node.js / Bun: npm install arkilian
  THEN ALWAYS check that the native binary loads: node --input-type=module -e "import 'arkilian'"
  Prebuilt binaries exist for Linux x64 and arm64, Alpine x64, macOS x64/arm64 and Windows x64. The Linux (glibc) ones need BOTH glibc 2.34+ AND a libstdc++ that provides GLIBCXX_3.4.31 (GCC 13 or newer): Ubuntu 24.04, Debian 13 and Fedora 38+ qualify. Ubuntu 22.04 does NOT (confirmed: it fails to load), and Debian 12 (the default node:20 and node:22 Docker images) and Amazon Linux 2023 ship an older libstdc++ and will likely fail the same way. On those systems "npm install" still SUCCEEDS and the error only appears at import.
  If the import check fails with an error that mentions GLIBCXX or GLIBC, compile from source instead (needs g++, make, python3 and libcurl4-openssl-dev installed):
    rm -rf node_modules/arkilian && ARKILIAN_SKIP_DOWNLOAD=1 npm install arkilian
  then run the import check again. In a Dockerfile, install those build packages before "npm install" (or use a base image with GCC 13+, such as ubuntu:24.04) and set ARKILIAN_SKIP_DOWNLOAD=1 so the image always builds from source.
  Linux also needs libcurl at runtime (apt install libcurl4). Other platforms (for example Alpine arm64) compile from source and need a C compiler plus libcurl headers.
- Python: do NOT run "pip install arkilian" (the PyPI package is an outdated stub with no native library). Install the release wheel instead:
  pip install https://github.com/arkiliandb/Arkilian/releases/download/v1.6.3/arkilian-1.6.3-py3-none-any.whl
  It bundles a Linux x86-64 library that needs glibc 2.34+ (Ubuntu 22.04 and newer) and libcurl. On macOS, Windows, ARM or an older Linux, STOP and ask the user.
- Go: "go get" does not work for this module (it relies on symlinks into the repo). Clone https://github.com/arkiliandb/Arkilian next to this project, then add to go.mod:
  require github.com/arkiliandb/Arkilian/bindings/go v1.6.3
  replace github.com/arkiliandb/Arkilian/bindings/go => ../Arkilian/bindings/go
  Needs cgo (a C compiler) and libcurl development headers.
- Rust: do NOT use the crates.io package (outdated 1.0.0). Clone the repo and depend on it by path:
  arkilian = { path = "../Arkilian/bindings/rust/arkilian" }
  Needs libcurl headers and libclang.
- PHP (7.4+, ext-ffi enabled): copy bindings/php/Arkilian.php from the repo, take libarkilian.so (or .dylib) from the C release archive, and set ARKILIAN_LIB_PATH to its full path.
- C / C++: download arkilian-c-v1.6.3-<os>-<arch> (.tar.gz, or .zip on Windows) from https://github.com/arkiliandb/Arkilian/releases. It contains include/arkilian/*.h and lib/libarkilian.so and libarkilian.a (the Linux build needs glibc 2.34+ and libcurl). Compile with -I<dir>/include/arkilian and link -larkilian -lcurl -lpthread -lm.

## Step 3. Change the code
Open the database ONCE at startup and reuse that single handle everywhere. Keep your SQL exactly as it is.

Node.js (ES modules; from CommonJS use: const { default: Arkilian } = await import('arkilian')). All calls are synchronous, so do not add await:
```js
import Arkilian from 'arkilian';
const db = new Arkilian('app.sqlite');
db.exec('CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, name TEXT)');
db.run('INSERT INTO users (name) VALUES (?)', ['Ada']);   // write; params are an array
const rows = db.all('SELECT * FROM users WHERE id = ?', [1]);  // array of plain objects
const first = rows[0];                                     // "get one row"
db.transaction((tx) => {                                   // rolls back if the callback throws
  tx.run('INSERT INTO users (name) VALUES (?)', ['Bob']);
});
db.close();                                                // on shutdown
```
If the project used better-sqlite3: db.prepare(sql).run(a, b) becomes db.run(sql, [a, b]); .all(a) becomes db.all(sql, [a]); .get(a) becomes db.all(sql, [a])[0].

Python (rows come back as dicts, NOT tuples: change row[0] to row["column_name"]; each run() commits immediately unless you call begin() first):
```python
from arkilian import Arkilian
db = Arkilian("app.sqlite")
db.exec("CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, name TEXT)")
db.run("INSERT INTO users (name) VALUES (?)", ["Ada"])
rows = db.all("SELECT * FROM users WHERE id = ?", [1])   # list of dicts
db.begin()
try:
    db.run("INSERT INTO users (name) VALUES (?)", ["Bob"])
    db.commit()
except Exception:
    db.rollback()
    raise
db.close()
```

Go (Exec takes plain SQL with no ? parameters; use Prepare for parameterized writes and Query for reads):
```go
import "github.com/arkiliandb/Arkilian/bindings/go/arkilian"

db, err := arkilian.OpenDB("app.sqlite")
if err != nil { log.Fatal(err) }
defer db.Close()
err = db.Exec("CREATE TABLE IF NOT EXISTS users (id INTEGER PRIMARY KEY, name TEXT)")
stmt, err := db.Prepare("INSERT INTO users (name) VALUES (?)")
err = stmt.BindText(1, "Ada")        // parameter numbers start at 1
_, err = stmt.Step()
err = stmt.Finalize()
rows, err := db.Query("SELECT * FROM users WHERE id = ?", 1)   // []map[string]interface{}
// transactions: db.Begin(), db.Commit(), db.Rollback()
```
Check every err value; the lines above omit that for brevity.

Rust: use arkilian::Database; let db = Database::new("app.sqlite")?; db.exec(sql)?; db.prepare(sql)?; db.use_stmt(0)?; db.bind_text(1, "x")?; db.bind_int(2, 5)?; db.step(); db.finalize()?; db.begin()? / db.commit()? / db.rollback()?
PHP: require_once 'Arkilian.php'; $db = new Arkilian('app.sqlite'); $db->exec($sql); $db->run($sql, [$a]); $rows = $db->all($sql, [$a]); $db->begin(); $db->commit(); $db->rollback(); $db->close();
C: arkilian *db = NULL; db_init(&db, "app.sqlite"); db_exec(db, sql); db_prepare(db, sql); db_bind_text(db, 1, "x"); db_bind_int(db, 2, 5); db_step(db); db_finalize(db); db_close(db);

## Step 4. Rules (do not skip any)
1. Run ALL DDL (CREATE, ALTER, DROP and every migration) through the Arkilian handle, never through a raw SQLite handle or a separate ORM connection. If a migration tool opens its own connection, route its SQL through Arkilian. If that is impossible, call resync after every migration: Node db.resyncTriggers(), Python db.resync_triggers(), Go db.ResyncTriggers(), Rust db.resync_triggers(), PHP $db->resyncTriggers(), C db_resync_triggers(db).
2. Every table you want replicated needs a PRIMARY KEY. A table without one is NOT replicated and the only sign is a warning in the log. Do not change the schema yourself; list those tables for the user.
3. One process per database file, one handle per process. Arkilian runs background threads, so open it once at startup and keep it open. Never open and close it per request. For multi-worker servers (PM2/cluster, gunicorn/uvicorn workers, php-fpm) or short-lived processes (serverless, CLI scripts), STOP and ask the user how the database should be owned.
4. One writer per S3 prefix. Give each database or tenant its own ARKILIAN_S3_PREFIX. A second live writer on the same prefix (for example a staging copy using the production prefix) freezes shipping.
5. Never read, edit or drop _pending_backup, _dead_backup, _arkilian_meta or any trigger whose name starts with trg_.
6. Call close() on graceful shutdown. It flushes and publishes the last chunks (up to 10 seconds by default).
7. In C, never share one handle across threads.

## Step 5. Configure storage
Settings come from the real environment or a .env file in the process WORKING DIRECTORY (not the database folder). The real environment wins. Required for replication:
```
ARKILIAN_S3_ENDPOINT=https://s3.amazonaws.com
ARKILIAN_S3_BUCKET=<bucket>
ARKILIAN_S3_REGION=us-east-1
ARKILIAN_S3_ACCESS_KEY=<access key>
ARKILIAN_S3_SECRET_KEY=<secret key>
ARKILIAN_S3_PREFIX=<one unique prefix per database>
ARKILIAN_MANIFEST_HMAC_KEY=<long random secret>
```
- Generate the HMAC key with: openssl rand -hex 32. It is REQUIRED: without it nothing is published and nothing can be restored. Keep a copy: every machine that restores this database needs the same key.
- The endpoint must be https://. Plain http:// is accepted only for localhost or private-network addresses (for example a local MinIO). Otherwise backup is disabled with an error in the log; ARKILIAN_ALLOW_INSECURE=1 overrides this for local development only.
- If the log says "storage endpoint refused ... SSRF guard", add the endpoint host to ARKILIAN_STORAGE_HOSTS (comma-separated).
- Replication is ON by default. If you have no storage yet (local development, tests, CI), set ARKILIAN_ENABLE_BACKUP=0. With backup on but settings incomplete, Arkilian only logs a warning and changes pile up in a local queue.
- Add .env to .gitignore and commit a .env.example containing the variable names with empty values.

## Step 6. Restore on a new machine (ONLY if the user asked for restore or failover)
Skip this step otherwise. On the very first deployment the prefix is empty, so there is nothing to restore and the call fails; ask the user how first start should behave.
- Call restore BEFORE any handle is opened, and only when the local database file does not exist.
- It needs ARKILIAN_MANIFEST_HMAC_KEY (the same key the writer used). It refuses tampered or unsigned data and refuses if the local file is newer than the snapshot. Never catch and ignore its errors.
- Node: Arkilian.hydrateS3('app.sqlite', { endpoint, bucket, region, accessKey, secretKey, prefix })  // all but region required
- Python: Arkilian.hydrate_s3(db_path, endpoint, bucket, region, access_key, secret_key, prefix)
- Go: err := arkilian.HydrateS3("app.sqlite", arkilian.S3Config{Endpoint: ..., Bucket: ..., Region: ..., AccessKey: ..., SecretKey: ..., Prefix: ...})
- Rust: arkilian::hydrate_s3(path, prefix, &s3config)
- PHP: Arkilian::hydrateS3($path, $prefix, ['endpoint' => ..., 'bucket' => ..., 'region' => ..., 'accessKey' => ..., 'secretKey' => ...])
- C: arkilian_hydrate_s3(path, endpoint, bucket, region, access_key, secret_key, prefix, NULL, NULL) returns 0 on success, a negative number on failure.
Then open the database normally.

## Step 7. Verify (do both parts, in order)
Part A, no storage: set ARKILIAN_ENABLE_BACKUP=0 and run the project's tests and start the app. They must behave exactly as before. In this mode the health value is false and the queue count grows; that is expected, so do not assert on them.
Part B, with real storage (ask the user for a test bucket or prefix):
1. Start the app and perform one write through Arkilian.
2. Poll for up to 60 seconds (chunks ship about every second, the manifest at most every 30 seconds) until ALL are true: queue depth is 0, chunk count is 1 or more, health is true.
   Node: db.backupQueueDepth, db.backupChunkCount, db.backupHealthy
   Python: db.backup_queue_depth, db.backup_chunk_count, db.backup_healthy
   Go: db.BackupQueueDepth(), db.BackupChunkCount(), db.BackupIsHealthy()
   Rust: db.backup_queue_depth(), db.backup_is_healthy()    PHP: $db->backupQueueDepth(), $db->backupChunkCount(), $db->backupIsHealthy()
   C: db_backup_queue_depth(db), db_backup_chunk_count(db), db_backup_is_healthy(db)
3. Also confirm skipped-table count and dead-letter count are both 0 (Node backupSkippedTableCount and backupDeadLetterCount; Python backup_skipped_table_count and backup_dead_letter_count; Go BackupSkippedTableCount() and BackupDeadLetterCount()).
4. If the aws CLI is available, list s3://<bucket>/<prefix>/ and confirm manifest.json and manifest.sig exist.
If any check fails or you could not run Part B, say so plainly. Never claim replication works unless you observed these results.

## Step 8. Report back
Tell the user: the files you changed; the environment variables they still must set; any tables skipped for lacking a PRIMARY KEY; any ground rule or step you stopped on; and which verification parts you actually ran.