Sync
How the engine keeps your local copy and your provider in step: IMAP and IDLE, Gmail and Microsoft specifics, the task queue, offline changes, drafts, calendars and contacts.
Your provider holds the real copy of your mail. The engine keeps a local copy in airpost.db, watches the server for changes, and replays your changes on the server. This page explains how, for each kind of data.
The sync code lives in the airpost-sync crate (the worker threads), airpost-imap and airpost-smtp (the protocols), airpost-tasks (your changes), airpost-dav and airpost-pim (calendars and contacts), and airpost-providers (per-server quirks).
The shape of it
Mail over IMAP
Each account keeps two IMAP connections, one per worker.
The background worker lists your folders and labels, then scans them. It visits folders by role first (inbox, sent, drafts, all mail, archive, trash, spam), then the rest. A shallow scan of recent changes runs about every 2 minutes and a deeper one about every 10. Servers with CONDSTORE and QRESYNC report every flag change and deletion directly, so on those the full attributes-only scan only runs once a day to repair gaps. After a scan, the orphan sweep removes local messages that no longer exist on the server.
The foreground worker holds an IMAP IDLE on your inbox (All Mail on Gmail), so new mail shows up as soon as the server announces it. IDLE is restarted every 28 minutes, inside the 29 minutes RFC 3501 allows. The foreground also fetches message bodies when you open them and runs the server side of your changes. When you queue a task, IDLE is interrupted after a 300 ms debounce, so several quick actions go to the server together.
New messages are processed by MailProcessor: headers are parsed, the message is matched to a thread, and its placements (which folders or labels it’s in, under which UID) are recorded. A message in several places is one row with several placements.
Provider differences
Code outside airpost-providers never checks a host name. Each provider is one file under crates/airpost-providers/src/providers/ with a detect function and a profile of quirk flags; the generic profile applies when nothing else matches. Profiles exist for Gmail, Office 365, iCloud, Yahoo, Yandex, Zoho, Dovecot, Cyrus, Courier, ProtonMail Bridge, DavMail, hMailServer and a few others.
Detected by the X-GM-EXT-1 capability.
- Labels, not folders. Gmail uses the labels mailbox model: All Mail is the folder every message lives in, and labels come from
X-GM-LABELS. Archiving removes the Inbox label; it doesn’t move the message. - Threads. Gmail’s own thread id (
X-GM-THRID) is used to thread messages. - Sent copies. Gmail files its own copy of what you send, so the engine looks for it instead of uploading one, and puts the labels of the thread you replied to on it.
- OAuth. Sign-in is Google OAuth; the engine refreshes the access token itself.
- Calendar and contacts. Calendars over Google’s CalDAV endpoint (
https://apidata.googleusercontent.com/caldav/v2/<email>), contacts and groups through the Google People API.
Tasks and the task queue
Every change, from marking a message read to sending it, is a task. Clients queue it with task.queue or with an intent such as threads.archive, which the engine turns into tasks.
- Local phase. Runs on the account’s
mainthread as soon as you queue the task. It changes the database (the thread moves, the star appears) and the engine emits deltas, so every client updates immediately. - Remote phase. Runs on the
foregroundworker against the server: a flag change, a MOVE, an SMTP send. - Done. The task ends as
complete(witherrorset if it failed) orcancelled, and clients gettask.done.
Tasks are stored in the database, so a task that hasn’t reached the server survives a restart and runs on the next launch.
Offline changes and conflicts
Because the local phase runs first, you can archive, flag, move and label while offline. The changes show at once and wait in remote.
Retries. Read, star, move, label, folder create and rename, folder delete and empty-folder tasks retry a retryable failure (a network error, a timeout) with a backoff from 5 seconds doubling to 5 minutes, up to 8 attempts. Other tasks don’t retry automatically.
Sends. A send the engine started itself, such as a scheduled send or an undo-send that came due, and that failed before SMTP accepted it, goes back to wait until the account is online again, for up to a day. A send runs at most once on the server: the engine records that it started before anything else, and a retry is a new send you start.
When the server says no for good. For read, star, move and label changes, the engine puts the local change back, so your local copy matches the server again. Other failures end the task with its error, which the app shows.
The server wins on what it reports. Scans and IDLE bring in changes made elsewhere (webmail, your phone). The engine doesn’t merge by field: what the server reports after your task ran is what you see. A message deleted on the server is removed locally by the orphan sweep.
Drafts
Drafts are saved locally first, then, with Save drafts to server on (the default, per account in Preferences > Accounts), pushed to the server’s Drafts folder so other clients see them.
- One draft, one id. A draft’s id comes from its account and Message-ID, so every copy of it (old and new uploads, other devices, webmail) maps onto the same local row.
- Push. The foreground worker uploads a draft once it’s been quiet for about 20 seconds (about 120 seconds with more than 1 MB of attachments), as a full MIME message with
X-Airpost-Draft: 1, then deletes the older copy. It never leaves zero copies; a crash leaves two, which are cleaned up on the next push. Failed pushes back off from 30 seconds up to an hour. A draft larger than the server’s APPENDLIMIT stays local. - Pristine drafts. A reply you opened but didn’t type in isn’t pushed.
- Edits from elsewhere. If another device saved a newer copy and you have no unpushed edits, the engine adopts it and an open composer takes it only in fields you haven’t changed. If you do have unpushed edits, yours win and the other copy goes to Trash.
- Sent and deleted drafts stay gone. After a send or a delete, the engine records a tombstone for the Message-ID for 30 days. A server copy matching a tombstone is deleted instead of brought back.
With the setting off, drafts stay on this computer only.
Calendars and contacts
The calContacts thread syncs calendars and contacts on a fixed schedule: starting 15 seconds after the account starts, calendars about every 15 minutes and contacts about every 75 minutes. The schedule ignores manual “sync now” requests; when every manual sync also synced contacts, Google’s quotas ran out within hours.
| Account | Calendars | Contacts |
|---|---|---|
| Gmail | Google CalDAV | Google People API |
| Other providers | CalDAV | CardDAV |
For non-Gmail accounts the engine finds the servers from DNS SRV and TXT records on your email domain (RFC 6764), or from a built-in table for providers that publish none, then follows .well-known, the principal and the home set to the collections. If neither exists, the account has no calendars or contacts.
Edits are tasks, like mail. Event and contact changes run their remote phase on the foreground worker: a new event or card is a PUT with If-None-Match: *, so it can’t overwrite something the engine hasn’t seen. A delete answered 404 or 410 removes the item locally. Recurrence expansion, RSVPs, counter-proposals and vCard import and export are in airpost-pim.
Related
- The engine: the threads and the socket.
- Local features: what the engine does on its own schedule.
- Accounts: adding Gmail, Microsoft and IMAP accounts.