Offline-First Markdown Sync with GitHub and 3-Way Merge

How do you synchronize a collection of Markdown notes across a desktop computer and a mobile phone without relying on commercial subscription cloud storage or running local Git binaries inside mobile sandboxes? Engineers managing personal knowledge bases often hit a wall when attempting to sync local file vaults across varied operating systems. Relying on traditional file synchronization tools introduces silent data loss, filesystem timestamp discrepancies, and corrupted YAML metadata blocks when edits occur offline on multiple devices. For a related implementation, see Managing Concurrent Git Commits During Automated.

This article examines an architectural approach to building a subscription-free synchronization engine. By combining cryptographic content hashing, direct integration with GitHub Git Data REST APIs, and a specialized 3-way merge algorithm tailored for Markdown documents, you can achieve reliable version control on mobile and desktop platforms alike.

The Limitations of Timestamp Sync and Mobile Git

Traditional file synchronization strategies typically rely on filesystem modification timestamps, known as mtime. If a file’s timestamp is newer than the last recorded sync time, the synchronization client uploads it. This mechanism fails frequently in practice. Operating systems handle precision differently, timezone shifts alter file attributes during transit, and cloud storage intermediaries often reset modification times upon download.

Furthermore, developers attempting to use standard Git clients face a strict environment boundary on mobile platforms. iOS and Android restrict native binary execution, preventing standard git command-line tools from running inside local application sandboxes. Attempting to bundle heavy WebAssembly Git wrappers often results in high memory consumption, slow performance on large vaults, and unstable network interactions.

When conflicts do arise, naive synchronization scripts execute destructive blind overwrites. If a note is edited offline on a smartphone while simultaneously updated on a laptop, a simple overwrite strategy discards one version completely. Standard Git conflict markers like <<<<<<< HEAD break YAML frontmatter parsers instantly, corrupting tags, aliases, and internal document links. For a related implementation, see Offline First Event Pipeline.

Content-Hash Change Detection

To determine whether a note has genuinely changed, the synchronization engine must inspect the file contents rather than its metadata. Computing a cryptographic hash of the note body provides a deterministic fingerprint of its exact state.

The synchronization engine calculates a SHA-256 hash for every Markdown file in the local vault during startup and after every write operation. The resulting hash is stored in a local metadata index alongside the file path and the last known remote commit SHA.

{
  "path": "notes/architecture.md",
  "hash": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855",
  "last_sync_commit": "7b4f9d2"
}

When scanning the vault for changes, the engine compares the current SHA-256 hash of each file against the stored local index. If the hashes match, the file remains untouched, avoiding unnecessary network transfers. If the hash differs, the engine flags the file as locally modified and queues it for synchronization. This approach completely bypasses filesystem timestamp discrepancies and ensures that only substantive text modifications trigger sync operations.

REST-Native Git Architecture

Because mobile sandboxes prohibit local Git binaries, the synchronization engine interacts directly with GitHub’s Git Database REST API. By communicating with low-level endpoints (/git/blobs, /git/trees, and /git/commits), the client constructs valid Git commit objects entirely over HTTP.

When pushing local changes, the synchronization workflow follows an explicit sequence:

1. Create Blobs: The client sends the raw text content of each modified Markdown file to the /git/blobs endpoint, receiving a content SHA for each uploaded blob.
2. Construct Tree: Using the base tree of the previous remote commit, the client posts a new tree structure to /git/trees, referencing the new blob SHAs against their respective file paths.
3. Create Commit: The client submits a new commit object to /git/commits, setting the parent commit to the current remote head and linking the newly created tree.
4. Update Reference: Finally, the client updates the remote branch reference via a fast-forward patch to /git/refs/heads/main.

This REST-native pipeline allows mobile and desktop clients to participate equally in a shared Git repository without executing a single local shell command.

The 3-Way Merge Engine for Markdown

When a local change and a remote change both target the same file, a simple fast-forward push is rejected by the remote repository. The synchronization engine must resolve the divergence using a 3-way merge algorithm. The algorithm evaluates three distinct states: the common ancestor revision, the local revision, and the remote revision.

Markdown documents present a unique structural challenge because they combine structured metadata at the top of the file with freeform prose below. To prevent metadata corruption, the merge engine splits the document parsing process into two distinct phases.

YAML Frontmatter Isolation

Before executing line-based text merging, the engine isolates the YAML frontmatter block from the body content. Frontmatter dictionaries contain tags, aliases, and custom properties that require attribute-level merging rather than raw text diffing.

function isolateFrontmatter(content: string): { frontmatter: string; body: string } {
  const match = content.match(/^---\r?\n([\s\S]*?)\r?\n---\r?\n([\s\S]*)$/);
  if (!match) {
    return { frontmatter: "", body: content };
  }
  return { frontmatter: match[1], body: match[2] };
}

By parsing the frontmatter strings into key-value pairs, the engine can combine tag arrays and update individual metadata fields safely without introducing invalid syntax.

Line-Based Text Merging

For the document body, the engine applies a standard line-based diff algorithm. If the local version modifies the conclusion of a note while the remote version adds a new paragraph to the introduction, the 3-way merge engine combines both modifications automatically into a unified document.

Offline Queueing and Resilient Reconnection

Knowledge workers frequently edit notes while offline on flights, in transit, or during network disruptions. A resilient synchronization client must handle disconnection gracefully without losing user input.

Changes made while offline are appended to a persistent local change journal stored in the application’s local database. Each journal entry records the operation type, file path, timestamp, and local hash. The user interface reflects the offline status by displaying a subtle indicator without blocking note editing.

When network connectivity is restored, the synchronization engine executes an atomic push-pull sequence:

1. It pings the remote repository to check for incoming changes.
2. If remote changes exist, it pulls the latest tree and performs automated 3-way merges against local files.
3. Once the local workspace is up to date, it flushes the offline change journal, pushing local modifications upstream in a single coordinated commit.

Graceful Conflict Handling

Despite the safety mechanisms of 3-way merging, genuine line collisions occur when conflicting edits target the exact same sentence or block. When an automatic merge cannot be resolved safely, injecting raw conflict markers into active Markdown prose is unacceptable because it disrupts reading flow and breaks document parsers.

Instead, the engine follows a safer fallback strategy. When an unresolvable conflict occurs, the synchronization client preserves the local version under its original file path, downloads the conflicting remote version into a clean duplicate file with a .conflict extension, and notifies the user via a non-intrusive notification banner.

notes/
├── project-plan.md
└── project-plan.conflict.md

This approach leaves the primary note clean and fully readable while placing the divergent version safely alongside it for manual review and reconciliation.

Conclusion

Synchronizing personal knowledge vaults across multiple devices requires moving beyond simple file timestamps and heavy command-line tools. By adopting SHA-256 content hashes, communicating directly with GitHub Git Data APIs, and isolating YAML frontmatter during 3-way merges, developers can build a reliable synchronization engine that operates smoothly within mobile sandboxes. Prioritizing data integrity and graceful conflict resolution ensures that your notes remain accessible, well-structured, and fully under your control.