Syncing

How rawquery syncs data from your sources, and how to control when and how it happens.

Starting a Sync

You can sync a connection from the Connections page in the dashboard. Click Sync on any connection to open the sync modal.

Select tables - Choose which tables to sync. All are selected by default.
Choose sync mode - If a table supports multiple modes, pick the one you want from the dropdown.
Start sync - Click "Sync N tables" to submit the job.

From the CLI:

bash

rq connections sync stripe
rq connections sync stripe --mode incremental_dedupe

rq connections sync stripe
rq connections sync stripe --mode incremental_dedupe

Sync Modes

Each table uses a sync mode that determines how data is fetched and written. See Sync Strategies for full details.

Mode	Label in UI	Behavior
`full_refresh`	Full refresh	Drop and replace all data
`incremental`	Append	Fetch only new records (cursor-based)
`incremental_dedupe`	Merge	Fetch new records + upsert on primary key
`window`	Window	Re-sync a sliding window of the last N days

Changing sync modes

You can change the sync mode per table in the connection detail panel. Expand a connection, find the Streams table, and use the dropdown in the Sync Mode column. Changes save immediately and apply to both manual and scheduled syncs.

From the CLI:

bash

rq connections update stripe --sync-mode customers:incremental_dedupe
rq connections update stripe --sync-mode charges:window:30

rq connections update stripe --sync-mode customers:incremental_dedupe
rq connections update stripe --sync-mode charges:window:30

Tables without a cursor field only support Full refresh (no dropdown).

Scheduling

Set a sync schedule in the connection detail panel under Sync Schedule. Options:

Manual - No automatic syncing
Hourly - Every hour
Daily - Once per day, pick the hour (UTC)
Weekly - Once per week, pick the day and hour (UTC)

The schedule composes a cron expression. Click Save to apply.

From the CLI:

bash

rq connections update my-db --schedule daily
rq connections update my-db --schedule "0 14 * * *"   # daily at 14:00 UTC
rq connections update my-db --schedule ""              # back to manual

rq connections update my-db --schedule daily
rq connections update my-db --schedule "0 14 * * *"   # daily at 14:00 UTC
rq connections update my-db --schedule ""              # back to manual

The Next sync time is displayed below the schedule picker. The Jobs tab shows upcoming scheduled syncs across all connections.

Sync Progress

After starting a sync, the modal shows a progress view with elapsed time. Syncs run in the background - you can close the modal and the sync continues.

The connection status in the table changes to syncing while a job is running. When complete, it returns to active.

Results

When the sync finishes, the modal shows a per-table breakdown:

Success - number of records synced for each table
Failure - error message for tables that failed

Partial failures are possible - some tables may succeed while others fail. The connection status will show error if any table failed.

What Happens During a Sync

Your sync job is queued and picked up by a background worker
The worker connects to your data source using stored credentials
Data is fetched and written to Iceberg tables on S3-compatible storage
Sync state (cursor, record count) is updated for the next incremental run
The connection status is updated and results are available in the modal

Synced data is immediately available for querying in the Query Editor, via the Execute API, or through any Postgres-compatible client connected to the wire protocol.