verstak
/
verstak-desktop
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Harden core snapshot sync and workspace lifecycle

This commit is contained in:
mirivlad 2026-07-17 04:10:59 +08:00
parent ba0ba5f8c4
commit e3d8078ad5
22 changed files with 2891 additions and 132 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
README.md
View File

@ -224,6 +224,24 @@ For synchronization between devices, deploy the optional self-hosted
Each vault is connected separately. The local vault remains the primary copy of your data.
Desktop core maintains an atomic local sync snapshot under `.verstak/sync/`.
It scans when a vault opens, before a manual sync, and after debounced external
file-watcher events, so files changed in an editor or while Desktop was closed
are reconciled too. The watcher only speeds up discovery. A first connection
pulls before it publishes a local bootstrap: an empty local vault does not send
deletes, and incompatible files become visible conflicts rather than silent
overwrites. Pairing can optionally use an existing remote vault ID to restore
that scope on a new device.
The current operation transport supports ordinary files and folders plus
workspace (Deal) create/rename/trash/restore with a durable workspace UUID.
`.verstak`, trash, temporary files, and symlinks are excluded from ordinary
file sync. Text remains bounded to 2 MB and binary payloads to 8 MB; a larger
or unsupported file is left as a visible unresolved sync warning instead of
being treated as synchronized. Blob transfer, quotas, pagination, retention,
and Secrets, plugin settings, Todo, Journal, Activity, and Browser Inbox sync
are future milestones.
## Build from source
### Requirements

39
docs/PLUGIN_RUNTIME.md
View File

@ -477,8 +477,8 @@ contributions summary.
- All paths are canonical vault-relative slash paths. Backslashes, POSIX
absolute paths, Windows drive paths, UNC/network paths, `..`, null bytes,
symlink traversal, and public access to `.verstak/` are rejected.
- `.verstak` is reserved case-insensitively: `.verstak`, `.Verstak`, and any
first path segment with that spelling are internal-only.
- `.verstak` is reserved case-insensitively in every path segment: its marker,
sync snapshot, trash, and workspace identity are internal-only.
- `files.metadata` may report a final symlink as `type: "symlink"`, but
`files.list` through a symlink directory and all read/write/move/trash
operations through symlinks are forbidden in Milestone 6a.
@ -486,20 +486,37 @@ contributions summary.
`writeBytes` are bounded byte contracts up to 8 MB; chunked streaming is
deferred.
- Live watcher refresh is active while Verstak is running and a vault is open.
It performs an initial no-event snapshot, then publishes `file.changed` for
external creates, updates, and deletes outside `.verstak/`. It does not keep a
persistent snapshot or report what changed while Verstak was closed.
It publishes `file.changed` as a UI hint and debounces a full core scanner;
the persistent `.verstak/sync/snapshot.json` scanner, not the watcher, is the
source of truth. It runs on open, before manual sync, after watcher events,
and therefore detects changes made while Verstak was closed. Internal paths,
trash, temporary files, and symlinks are excluded.
`sync`
- `sync.now()` pushes local operations, pulls remote operations, and returns
`{ pushed, pulled, serverSequence, conflicts?, applyErrors? }`.
- `sync.now()` scans local files, pulls/reconciles in `server_sequence` order,
then pushes local operations and returns `{ pushed, pulled, serverSequence,
conflicts? }`. A remote apply failure stops the batch immediately, keeps that
operation and later sequences unacknowledged, and retries it on the next run.
- `conflicts` is an array of server-reported sync conflicts. Conflict objects
may include `op_id`, `entity_type`, `entity_id`, `reason`, and additional
server fields. The Sync plugin must show conflict details instead of only a
count, and it must not silently resolve or overwrite local data.
- `applyErrors` lists local apply failures for pulled operations. These are
user-visible warnings and do not imply that sync was fully successful.
- `sync.status()` includes `vaultId` (the paired remote scope) and
`lastWarning` for unresolved scanner input. A first connection pulls before
bootstrap: an empty local snapshot never publishes deletes, and incompatible
local/remote content is an explicit conflict rather than an overwrite.
- File and folder operations come from the scanner. An external rename is
intentionally represented as delete + create in this milestone. Workspace
lifecycle is a separate core `workspace` entity (`create`, `rename`,
`trash`, `restore`) carrying the durable `workspaceId`; plugins never obtain
access to the nested workspace marker.
- File transport remains bounded: UTF-8 text uses the 2 MB Files API limit and
binary/other regular files use the existing base64 path up to 8 MB. Larger or
unsupported files remain visible unresolved warnings and are not added to a
successful snapshot. Blob transport, quotas, pagination, retention, and
synchronization of Secrets, plugin settings, Todo, Journal, Activity, and
Browser Inbox are future work.
- Transport push/pull uses bounded retry/backoff for transient HTTP/network
failures. Client/auth errors are not retried.
@ -585,8 +602,8 @@ bundled runtime. Это реальный runtime contract для cooperative bun
| `api.files.showInFolder(relativePath)` | ✅ Работает | Показывает vault file/folder в системном файловом менеджере, требует `files.openExternal` |
| `api.workbench.openResource(request)` | ✅ Работает | Routes vault resources to `openProviders` |
| `api.workbench.editResource(request)` | ✅ Работает | Same routing, forcing `mode: "edit"` |
| `api.sync.now()` | ✅ Работает | Push/pull с bounded retry/backoff для transient HTTP/network failures |
| `api.sync.status()` | ✅ Работает | Возвращает configured/connected/error/revoked state, lastError, unpushed count |
| `api.sync.now()` | ✅ Работает | Snapshot scan, строгий ordered pull/retry и bounded retry/backoff для transient HTTP/network failures |
| `api.sync.status()` | ✅ Работает | Возвращает configured/connected/error/revoked, remote `vaultId`, lastError/lastWarning и unpushed count |
| `api.dispose()` | ✅ Работает | Очищает command handlers и event subscriptions текущего API instance |
Ограничения:

2
frontend/e2e/command-palette.spec.js
View File

@ -99,7 +99,7 @@ test.describe('Command Palette', () => {
test('runs sync workflow commands', async ({ page }) => {
await page.evaluate(async () => {
const err = await window.go.api.App.PluginSyncConfigure('verstak.sync', 'https://sync.example.test', 'alice', 'secret');
const err = await window.go.api.App.PluginSyncConfigure('verstak.sync', 'https://sync.example.test', 'alice', 'secret', '');
if (err) throw new Error(err);
});

4
frontend/e2e/plugin-api-bridge.spec.js
View File

@ -102,7 +102,7 @@ test.describe('D: Plugin API bridge', () => {
const api = window.createPluginAPI('verstak.sync');
const initial = await api.sync.status();
await api.sync.testConnection('https://sync.example.test', 'alice', 'secret');
await api.sync.configure('https://sync.example.test', 'alice', 'secret');
await api.sync.configure('https://sync.example.test', 'alice', 'secret', 'existing-remote-vault');
await api.sync.setInterval(15);
const configured = await api.sync.status();
const syncNow = await api.sync.now();
@ -117,6 +117,8 @@ test.describe('D: Plugin API bridge', () => {
expect(result.initial.statusLabel).toBe('disabled');
expect(result.configured.configured).toBe(true);
expect(result.configured.serverUrl).toBe('https://sync.example.test');
expect(result.configured.vaultId).toBe('existing-remote-vault');
expect(result.configured.lastWarning).toBe('');
expect(result.configured.syncInterval).toBe(15);
expect(result.syncNow).toEqual({ pushed: 0, pulled: 0, serverSequence: 0 });
expect(result.reset.configured).toBe(false);

4
frontend/src/lib/plugin-host/VerstakPluginAPI.js
View File

@ -468,10 +468,10 @@ export function createPluginAPI(pluginId) {
return App.PluginSyncStatus(pluginId);
});
},
configure: function(serverURL, username, password) {
configure: function(serverURL, username, password, vaultId) {
assertActive('sync.configure');
return callBackendErrorString(pluginId, 'sync.configure', function() {
return App.PluginSyncConfigure(pluginId, serverURL || '', username || '', password || '');
return App.PluginSyncConfigure(pluginId, serverURL || '', username || '', password || '', vaultId || '');
});
},
disconnect: function() {

10
frontend/src/lib/test/wails-mock.js
View File

@ -626,6 +626,7 @@ import journalSource from '../../../../../verstak-official-plugins/plugins/journ
return {
configured: false,
serverUrl: '',
vaultId: '',
deviceId: 'mock-device',
deviceName: '',
connected: false,
@ -635,6 +636,7 @@ import journalSource from '../../../../../verstak-official-plugins/plugins/journ
lastSyncAt: '',
syncInterval: 0,
lastError: '',
lastWarning: '',
statusLabel: 'disabled',
serverSequence: 0
};
@ -649,7 +651,7 @@ import journalSource from '../../../../../verstak-official-plugins/plugins/journ
if (p.charAt(0) === '/' || /^[A-Za-z]:/.test(p)) return { error: 'invalid-path: absolute path rejected' };
var parts = p.split('/').filter(Boolean);
if (parts.indexOf('..') !== -1) return { error: 'invalid-path: path-traversal' };
if (parts[0] && parts[0].toLowerCase() === '.verstak') return { error: 'reserved-path: .verstak is internal' };
if (parts.some(function(part) { return part.toLowerCase() === '.verstak'; })) return { error: 'reserved-path: .verstak is internal' };
return { path: parts.join('/') };
}
@ -751,6 +753,7 @@ import journalSource from '../../../../../verstak-official-plugins/plugins/journ
return {
configured: syncState.configured,
serverUrl: syncState.serverUrl,
vaultId: syncState.vaultId,
deviceId: syncState.deviceId,
deviceName: syncState.deviceName,
connected: syncState.connected,
@ -760,6 +763,7 @@ import journalSource from '../../../../../verstak-official-plugins/plugins/journ
lastSyncAt: syncState.lastSyncAt,
syncInterval: syncState.syncInterval,
lastError: syncState.lastError,
lastWarning: syncState.lastWarning,
statusLabel: syncState.statusLabel
};
}
@ -3477,11 +3481,12 @@ import journalSource from '../../../../../verstak-official-plugins/plugins/journ
if (err) return Promise.resolve([{}, err]);
return Promise.resolve([syncStatusDTO(), '']);
},
PluginSyncConfigure: function (pluginId, serverUrl) {
PluginSyncConfigure: function (pluginId, serverUrl, username, password, vaultId) {
var err = requirePluginSyncPermission(pluginId, true);
if (err) return Promise.resolve(err);
syncState.configured = true;
syncState.serverUrl = serverUrl || '';
syncState.vaultId = vaultId || 'test-vault-001';
syncState.deviceId = 'mock-device';
syncState.deviceName = 'mock-device';
syncState.connected = true;
@ -3491,6 +3496,7 @@ import journalSource from '../../../../../verstak-official-plugins/plugins/journ
syncState.statusLabel = 'connected';
pluginSettings[pluginId] = Object.assign({}, pluginSettings[pluginId] || {}, {
serverUrl: syncState.serverUrl,
vaultId: syncState.vaultId,
syncStatus: syncState.statusLabel
});
return Promise.resolve('');

2
frontend/wailsjs/go/api/App.d.ts vendored
View File

@ -114,7 +114,7 @@ export function PluginSecretsUnlock(arg1:string,arg2:string):Promise<string>;
export function PluginSecretsWrite(arg1:string,arg2:Record<string, any>):Promise<Record<string, any>|string>;
export function PluginSyncConfigure(arg1:string,arg2:string,arg3:string,arg4:string):Promise<string>;
export function PluginSyncConfigure(arg1:string,arg2:string,arg3:string,arg4:string,arg5:string):Promise<string>;
export function PluginSyncDisconnect(arg1:string):Promise<string>;

4
frontend/wailsjs/go/api/App.js
View File

@ -210,8 +210,8 @@ export function PluginSecretsWrite(arg1, arg2) {
return window['go']['api']['App']['PluginSecretsWrite'](arg1, arg2);
}
export function PluginSyncConfigure(arg1, arg2, arg3, arg4) {
return window['go']['api']['App']['PluginSyncConfigure'](arg1, arg2, arg3, arg4);
export function PluginSyncConfigure(arg1, arg2, arg3, arg4, arg5) {
return window['go']['api']['App']['PluginSyncConfigure'](arg1, arg2, arg3, arg4, arg5);
}
export function PluginSyncDisconnect(arg1) {

4
frontend/wailsjs/go/models.ts
View File

@ -310,6 +310,7 @@ export namespace api {
export class SyncStatusDTO {
configured: boolean;
serverUrl: string;
vaultId: string;
deviceId: string;
deviceName: string;
connected: boolean;
@ -319,6 +320,7 @@ export namespace api {
lastSyncAt: string;
syncInterval: number;
lastError: string;
lastWarning: string;
statusLabel: string;
static createFrom(source: any = {}) {
@ -329,6 +331,7 @@ export namespace api {
if ('string' === typeof source) source = JSON.parse(source);
this.configured = source["configured"];
this.serverUrl = source["serverUrl"];
this.vaultId = source["vaultId"];
this.deviceId = source["deviceId"];
this.deviceName = source["deviceName"];
this.connected = source["connected"];
@ -338,6 +341,7 @@ export namespace api {
this.lastSyncAt = source["lastSyncAt"];
this.syncInterval = source["syncInterval"];
this.lastError = source["lastError"];
this.lastWarning = source["lastWarning"];
this.statusLabel = source["statusLabel"];
}
}

529
internal/api/app.go
View File

@ -3,8 +3,11 @@ package api
import (
"context"
"crypto/sha256"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"log"
"net/url"
"os"
@ -95,6 +98,9 @@ type App struct {
browserReceiver *browserreceiver.Receiver
secretsSession *coresecrets.VaultSession
fileWatcher *filewatcher.Service
syncRunMu sync.Mutex
syncTimerMu sync.Mutex
syncScanTimer *time.Timer
notifications notificationService
debug bool
activityEvents map[string]bool
@ -1228,6 +1234,7 @@ func (a *App) CloseVault() error {
if a.fileWatcher != nil {
a.fileWatcher.Stop()
}
a.stopScheduledSnapshotScan()
a.vault.CloseVault()
a.syncSvc = nil
a.secretsSession = nil
@ -1738,10 +1745,30 @@ func (a *App) externalOpenService() externalOpenService {
}
func (a *App) recordFileSyncOp(entityType, entityID, opType string, payload interface{}) error {
if a.syncSvc == nil {
_, err := a.scanLocalChanges()
return err
}
func (a *App) recordWorkspaceSyncOp(opType, workspaceID, path, previousPath, name string) error {
if a.syncSvc == nil || a.workspace == nil {
return nil
}
return a.syncSvc.RecordOp(entityType, entityID, opType, payload)
meta, err := a.workspace.GetWorkspaceMetadata(path)
if err != nil && opType != syncsvc.OpTrash {
return err
}
payload := syncWorkspacePayload{
WorkspaceID: workspaceID,
Path: path,
PreviousPath: previousPath,
Name: name,
Metadata: meta,
}
if err := a.syncSvc.RecordOp(syncsvc.EntityWorkspace, workspaceID, opType, payload); err != nil {
return err
}
_, err = a.syncSvc.RebaseSnapshot()
return err
}
func (a *App) publishFileActivity(eventName, pluginID, relativePath string, extra map[string]interface{}) {
@ -2314,9 +2341,69 @@ func (a *App) startFileWatcherForOpenVault() {
if a.fileWatcher == nil {
a.fileWatcher = filewatcher.NewService(a.eventBus, 0)
}
a.fileWatcher.SetOnChange(a.scheduleSnapshotScan)
if err := a.fileWatcher.Start(a.vault.GetVaultPath()); err != nil {
log.Printf("[api] file watcher start failed: %v", err)
}
if _, err := a.scanLocalChanges(); err != nil {
log.Printf("[api] initial sync snapshot scan failed: %v", err)
}
}
const snapshotScanDebounce = 300 * time.Millisecond
func (a *App) scheduleSnapshotScan() {
if a == nil {
return
}
a.syncTimerMu.Lock()
if a.syncScanTimer != nil {
a.syncScanTimer.Stop()
}
a.syncScanTimer = time.AfterFunc(snapshotScanDebounce, func() {
if _, err := a.scanLocalChanges(); err != nil {
log.Printf("[api] watcher sync snapshot scan failed: %v", err)
}
})
a.syncTimerMu.Unlock()
}
func (a *App) stopScheduledSnapshotScan() {
if a == nil {
return
}
a.syncTimerMu.Lock()
if a.syncScanTimer != nil {
a.syncScanTimer.Stop()
a.syncScanTimer = nil
}
a.syncTimerMu.Unlock()
}
func (a *App) scanLocalChanges() ([]string, error) {
if a == nil {
return nil, nil
}
a.syncRunMu.Lock()
defer a.syncRunMu.Unlock()
return a.scanLocalChangesLocked()
}
func (a *App) scanLocalChangesLocked() ([]string, error) {
if a.syncSvc == nil {
return nil, nil
}
warnings, err := a.syncSvc.ScanAndRecord()
if err != nil {
if a.appSettings != nil {
_ = a.updateSyncError("snapshot scan: " + err.Error())
}
return nil, err
}
if err := a.syncSvc.SetLastWarning(strings.Join(warnings, "\n")); err != nil {
return nil, err
}
return warnings, nil
}
// ─── Workspace API ─────────────────────────────────────────
@ -2373,6 +2460,9 @@ func (a *App) CreateWorkspace(name, templateID string) (workspace.Workspace, str
if err != nil {
return workspace.Workspace{}, err.Error()
}
if err := a.recordWorkspaceSyncOp(syncsvc.OpCreate, ws.ID, ws.RootPath, "", ws.Name); err != nil {
return workspace.Workspace{}, err.Error()
}
a.publishWorkspaceLifecycleEvent(workspaceCreatedEventName, map[string]interface{}{
"operation": "create",
"workspaceId": ws.ID,
@ -2395,6 +2485,9 @@ func (a *App) RenameWorkspace(oldName, newName string) string {
if err != nil {
return err.Error()
}
if err := a.recordWorkspaceSyncOp(syncsvc.OpRename, identity.WorkspaceID, newName, oldName, newName); err != nil {
return err.Error()
}
a.publishWorkspaceLifecycleEvent(workspaceRenamedEventName, map[string]interface{}{
"operation": "rename",
"workspaceId": identity.WorkspaceID,
@ -2415,6 +2508,9 @@ func (a *App) TrashWorkspace(name string) (workspace.TrashResult, string) {
if err != nil {
return workspace.TrashResult{}, err.Error()
}
if err := a.recordWorkspaceSyncOp(syncsvc.OpTrash, result.WorkspaceID, name, "", name); err != nil {
return workspace.TrashResult{}, err.Error()
}
a.publishWorkspaceLifecycleEvent(workspaceTrashedEventName, map[string]interface{}{
"operation": "trash",
"workspaceId": result.WorkspaceID,
@ -2436,6 +2532,9 @@ func (a *App) RestoreWorkspaceTrash(trashID, targetName string) (workspace.Works
if err != nil {
return workspace.Workspace{}, err.Error()
}
if err := a.recordWorkspaceSyncOp(syncsvc.OpRestore, restored.ID, restored.RootPath, "", restored.Name); err != nil {
return workspace.Workspace{}, err.Error()
}
a.publishWorkspaceLifecycleEvent(workspaceRestoredEventName, map[string]interface{}{
"operation": "restore",
"workspaceId": restored.ID,
@ -2943,6 +3042,7 @@ func (a *App) vaultPath() string {
type SyncStatusDTO struct {
Configured bool `json:"configured"`
ServerURL string `json:"serverUrl"`
VaultID string `json:"vaultId"`
DeviceID string `json:"deviceId"`
DeviceName string `json:"deviceName"`
Connected bool `json:"connected"`
@ -2952,6 +3052,7 @@ type SyncStatusDTO struct {
LastSyncAt string `json:"lastSyncAt"`
SyncInterval int `json:"syncInterval"`
LastError string `json:"lastError"`
LastWarning string `json:"lastWarning"`
StatusLabel string `json:"statusLabel"`
}
@ -2972,15 +3073,19 @@ func (a *App) syncStatus() (*SyncStatusDTO, error) {
cfg := a.appSettings.Get()
deviceToken := syncsvc.LoadDeviceToken(vaultPath)
remoteVaultID, _ := a.syncSvc.RemoteVaultID()
lastWarning, _ := a.syncSvc.LastWarning()
dto := &SyncStatusDTO{
Configured: serverURL != "" && (apiKey != "" || deviceToken != ""),
ServerURL: serverURL,
VaultID: remoteVaultID,
LastSyncAt: lastSyncAt,
UnpushedOps: 0,
TokenStored: deviceToken != "",
SyncInterval: cfg.Sync.SyncInterval,
LastError: cfg.Sync.LastError,
LastWarning: lastWarning,
}
if deviceID := a.syncSvc.GetDeviceID(); deviceID != "" {
@ -3048,7 +3153,7 @@ func (a *App) PluginSyncStatus(pluginID string) (*SyncStatusDTO, string) {
return dto, ""
}
func (a *App) syncConfigure(serverURL, username, password string) error {
func (a *App) syncConfigure(serverURL, username, password, remoteVaultID string) error {
if err := a.requireVault(); err != nil {
return err
}
@ -3061,8 +3166,28 @@ func (a *App) syncConfigure(serverURL, username, password string) error {
if hostname == "" {
hostname = "unknown"
}
targetVaultID := strings.TrimSpace(remoteVaultID)
if targetVaultID == "" {
targetVaultID = meta.VaultID
}
if a.syncSvc != nil {
previousServerURL, _, _, _, stateErr := a.syncSvc.GetState()
previousVaultID, _ := a.syncSvc.RemoteVaultID()
if previousVaultID == "" {
previousVaultID = meta.VaultID
}
if stateErr == nil && previousServerURL != "" && previousVaultID != targetVaultID {
pending, err := a.syncSvc.GetUnpushedOps()
if err != nil {
return fmt.Errorf("read pending operations before changing remote vault: %w", err)
}
if len(pending) > 0 {
return fmt.Errorf("cannot change remote vault with %d unpushed local operation(s); synchronize or resolve them first", len(pending))
}
}
}
client := newSyncClient(serverURL, "", "", vaultPath)
deviceID, deviceToken, err := client.PairDevice(serverURL, username, password, hostname, "verstak-desktop/v2", meta.VaultID)
deviceID, deviceToken, err := client.PairDevice(serverURL, username, password, hostname, "verstak-desktop/v2", targetVaultID)
if err != nil {
return fmt.Errorf("pair: %w", err)
}
@ -3073,6 +3198,18 @@ func (a *App) syncConfigure(serverURL, username, password string) error {
if err := a.syncSvc.SetState(serverURL, ""); err != nil {
return err
}
if err := a.syncSvc.SetRemoteVaultID(targetVaultID); err != nil {
return err
}
if err := a.syncSvc.SetLastPullSeq(0); err != nil {
return err
}
if err := a.syncSvc.SetBootstrapComplete(false); err != nil {
return err
}
if err := a.syncSvc.SetLastWarning(""); err != nil {
return err
}
cfg := a.appSettings.Get()
cfg.Sync.Enabled = true
@ -3087,11 +3224,11 @@ func (a *App) syncConfigure(serverURL, username, password string) error {
}
// PluginSyncConfigure pairs the current vault with a sync server for a plugin.
func (a *App) PluginSyncConfigure(pluginID, serverURL, username, password string) string {
func (a *App) PluginSyncConfigure(pluginID, serverURL, username, password, remoteVaultID string) string {
if err := a.requirePluginSyncAccess(pluginID, true); err != nil {
return err.Error()
}
if err := a.syncConfigure(serverURL, username, password); err != nil {
if err := a.syncConfigure(serverURL, username, password, remoteVaultID); err != nil {
return err.Error()
}
return ""
@ -3130,7 +3267,13 @@ func (a *App) syncDisconnect() error {
if a.syncSvc == nil {
return nil
}
return a.syncSvc.SetState("", "")
if err := a.syncSvc.SetState("", ""); err != nil {
return err
}
if err := a.syncSvc.SetRemoteVaultID(""); err != nil {
return err
}
return a.syncSvc.SetLastWarning("")
}
// PluginSyncDisconnect disconnects sync for a plugin with sync permission.
@ -3230,6 +3373,8 @@ func (a *App) PluginSyncResetKey(pluginID string) string {
}
func (a *App) syncNow() (map[string]interface{}, error) {
a.syncRunMu.Lock()
defer a.syncRunMu.Unlock()
if err := a.requireVault(); err != nil {
return nil, err
}
@ -3240,6 +3385,9 @@ func (a *App) syncNow() (map[string]interface{}, error) {
if a.syncSvc == nil {
return nil, fmt.Errorf("sync service not initialized")
}
if _, err := a.scanLocalChangesLocked(); err != nil {
return nil, fmt.Errorf("snapshot scan: %w", err)
}
serverURL, apiKey, lastPullSeq, _, err := a.syncSvc.GetState()
deviceToken := syncsvc.LoadDeviceToken(vaultPath)
@ -3270,12 +3418,44 @@ func (a *App) syncNow() (map[string]interface{}, error) {
client.DeviceID = deviceID
}
bootstrapComplete, err := a.syncSvc.BootstrapComplete()
if err != nil {
return nil, fmt.Errorf("read bootstrap state: %w", err)
}
initialSnapshot, err := a.syncSvc.LoadSnapshot()
if err != nil {
return nil, fmt.Errorf("load initial snapshot: %w", err)
}
// Pull before publishing. This makes the first reconciliation safe: remote
// state is applied or reported as a conflict before a pre-existing local
// vault can enqueue its bootstrap creates.
pulled, cursor, serverSequence, err := a.pullRemoteOps(client, lastPullSeq, !bootstrapComplete)
if err != nil {
return nil, err
}
if pulled > 0 {
if warnings, err := a.syncSvc.RebaseSnapshot(); err != nil {
return nil, fmt.Errorf("rebase remote snapshot: %w", err)
} else if err := a.syncSvc.SetLastWarning(strings.Join(warnings, "\n")); err != nil {
return nil, fmt.Errorf("save sync warning: %w", err)
}
}
if !bootstrapComplete {
if err := a.syncSvc.RecordBootstrapOps(initialSnapshot); err != nil {
return nil, fmt.Errorf("record initial local snapshot: %w", err)
}
if err := a.syncSvc.SetBootstrapComplete(true); err != nil {
return nil, fmt.Errorf("save bootstrap state: %w", err)
}
}
unpushed, err := a.syncSvc.GetUnpushedOps()
if err != nil {
return nil, fmt.Errorf("get ops: %w", err)
}
for i := range unpushed {
unpushed[i].LastSeenServerSeq = lastPullSeq
unpushed[i].LastSeenServerSeq = cursor
}
pushResult := &syncsvc.PushResponse{}
if len(unpushed) > 0 {
@ -3289,50 +3469,35 @@ func (a *App) syncNow() (map[string]interface{}, error) {
}
}
pullResult, err := client.Pull(lastPullSeq)
// Pull once more so this device durably acknowledges its own accepted
// operations and any concurrent remote operations without reapplying its own.
pulledAfterPush, _, finalServerSequence, err := a.pullRemoteOps(client, cursor, false)
if err != nil {
_ = a.updateSyncError(fmt.Sprintf("pull: %v", err))
return nil, fmt.Errorf("pull: %w", err)
return nil, err
}
var applyErrors []string
for _, op := range pullResult.Ops {
if err := a.applyRemoteOp(op); err != nil {
applyErrors = append(applyErrors, fmt.Sprintf("%s/%s: %v", op.EntityType, op.OpID, err))
if pulledAfterPush > 0 {
if warnings, err := a.syncSvc.RebaseSnapshot(); err != nil {
return nil, fmt.Errorf("rebase final remote snapshot: %w", err)
} else if err := a.syncSvc.SetLastWarning(strings.Join(warnings, "\n")); err != nil {
return nil, fmt.Errorf("save sync warning: %w", err)
}
_ = a.syncSvc.RecordRemoteOp(op)
}
if len(pullResult.Ops) > 0 {
opIDs := make([]string, len(pullResult.Ops))
for i, op := range pullResult.Ops {
opIDs[i] = op.OpID
if finalServerSequence > serverSequence {
serverSequence = finalServerSequence
}
_ = a.syncSvc.MarkApplied(opIDs)
}
if len(pushResult.Conflicts) > 0 {
log.Printf("[sync] %d conflict(s) detected on push", len(pushResult.Conflicts))
for _, c := range pushResult.Conflicts {
log.Printf("[sync] conflict: op=%v entity=%v/%v",
c["op_id"], c["entity_type"], c["entity_id"])
}
}
if pullResult.ServerSequence > lastPullSeq {
_ = a.syncSvc.SetLastPullSeq(pullResult.ServerSequence)
}
_ = a.syncSvc.SetLastSyncAt(time.Now().UTC().Format(time.RFC3339))
now := time.Now().UTC().Format(time.RFC3339)
a.updateSyncSuccess(now)
if err := a.syncSvc.SetLastSyncAt(now); err != nil {
return nil, fmt.Errorf("save sync time: %w", err)
}
_ = a.updateSyncSuccess(now)
result := map[string]interface{}{
"pushed": len(pushResult.Accepted),
"pulled": len(pullResult.Ops),
"serverSequence": pullResult.ServerSequence,
}
if len(applyErrors) > 0 {
result["applyErrors"] = applyErrors
"pulled": pulled + pulledAfterPush,
"serverSequence": serverSequence,
}
if len(pushResult.Conflicts) > 0 {
result["conflicts"] = pushResult.Conflicts
@ -3340,6 +3505,74 @@ func (a *App) syncNow() (map[string]interface{}, error) {
return result, nil
}
func (a *App) pullRemoteOps(client *syncsvc.Client, cursor int, initialReconciliation bool) (pulled, nextCursor, serverSequence int, err error) {
pullResult, err := client.Pull(cursor)
if err != nil {
_ = a.updateSyncError(fmt.Sprintf("pull: %v", err))
return 0, cursor, cursor, fmt.Errorf("pull: %w", err)
}
nextCursor = cursor
lastSequenceInBatch := cursor
for _, op := range pullResult.Ops {
if op.ServerSequence <= cursor {
continue
}
if op.ServerSequence <= lastSequenceInBatch {
errMsg := fmt.Sprintf("pull response is not strictly ordered at sequence %d (%s)", op.ServerSequence, op.OpID)
_ = a.updateSyncError(errMsg)
return pulled, nextCursor, pullResult.ServerSequence, fmt.Errorf("%s", errMsg)
}
lastSequenceInBatch = op.ServerSequence
if err := a.applyRemoteOpForReconciliation(op, initialReconciliation); err != nil {
path := syncOperationPath(op)
errMsg := fmt.Sprintf("pull apply failed at sequence %d for %s %s (%s): %v", op.ServerSequence, op.EntityType, path, op.OpID, err)
_ = a.updateSyncError(errMsg)
return pulled, nextCursor, pullResult.ServerSequence, fmt.Errorf("%s", errMsg)
}
if err := a.syncSvc.RecordRemoteOp(op); err != nil {
errMsg := fmt.Sprintf("record applied remote operation at sequence %d (%s): %v", op.ServerSequence, op.OpID, err)
_ = a.updateSyncError(errMsg)
return pulled, nextCursor, pullResult.ServerSequence, fmt.Errorf("%s", errMsg)
}
if err := a.syncSvc.SetLastPullSeq(op.ServerSequence); err != nil {
errMsg := fmt.Sprintf("save pull cursor at sequence %d (%s): %v", op.ServerSequence, op.OpID, err)
_ = a.updateSyncError(errMsg)
return pulled, nextCursor, pullResult.ServerSequence, fmt.Errorf("%s", errMsg)
}
if warnings, err := a.syncSvc.RebaseSnapshot(); err != nil {
errMsg := fmt.Sprintf("rebase snapshot after sequence %d (%s): %v", op.ServerSequence, op.OpID, err)
_ = a.updateSyncError(errMsg)
return pulled, nextCursor, pullResult.ServerSequence, fmt.Errorf("%s", errMsg)
} else if err := a.syncSvc.SetLastWarning(strings.Join(warnings, "\n")); err != nil {
return pulled, nextCursor, pullResult.ServerSequence, fmt.Errorf("save sync warning: %w", err)
}
nextCursor = op.ServerSequence
pulled++
}
if pullResult.ServerSequence > nextCursor {
if err := a.syncSvc.SetLastPullSeq(pullResult.ServerSequence); err != nil {
_ = a.updateSyncError(fmt.Sprintf("save pull cursor at sequence %d: %v", pullResult.ServerSequence, err))
return pulled, nextCursor, pullResult.ServerSequence, fmt.Errorf("save pull cursor: %w", err)
}
nextCursor = pullResult.ServerSequence
}
return pulled, nextCursor, pullResult.ServerSequence, nil
}
func syncOperationPath(op syncsvc.Op) string {
if op.EntityType == syncsvc.EntityWorkspace {
if payload, err := parseSyncWorkspacePayload(op.PayloadJSON); err == nil && payload.Path != "" {
return payload.Path
}
return op.EntityID
}
payload, _ := parseSyncFilePayload(op.PayloadJSON)
if path := syncPayloadPath(op, payload); path != "" {
return path
}
return op.EntityID
}
// PluginSyncNow triggers sync for a plugin with sync permission.
func (a *App) PluginSyncNow(pluginID string) (map[string]interface{}, string) {
if err := a.requirePluginSyncAccess(pluginID, true); err != nil {
@ -3368,12 +3601,23 @@ func (a *App) updateSyncSuccess(lastSyncAt string) error {
}
func (a *App) applyRemoteOp(op syncsvc.Op) error {
return a.applyRemoteOpForReconciliation(op, false)
}
func (a *App) applyRemoteOpForReconciliation(op syncsvc.Op, initialReconciliation bool) error {
if a.debug {
log.Printf("[sync] applyRemoteOp: type=%s entity=%s/%s", op.OpType, op.EntityType, op.EntityID)
}
if op.DeviceID != "" && op.DeviceID == a.localSyncDeviceID() {
return nil
}
if op.EntityType == syncsvc.EntityWorkspace {
payload, err := parseSyncWorkspacePayload(op.PayloadJSON)
if err != nil {
return err
}
return a.applyRemoteWorkspaceOp(op, payload)
}
if a.files == nil {
return fmt.Errorf("files service not initialized")
}
@ -3384,9 +3628,9 @@ func (a *App) applyRemoteOp(op syncsvc.Op) error {
}
switch op.EntityType {
case syncsvc.EntityFile:
return a.applyRemoteFileOp(op, payload)
return a.applyRemoteFileOp(op, payload, initialReconciliation)
case syncsvc.EntityFolder:
return a.applyRemoteFolderOp(op, payload)
return a.applyRemoteFolderOp(op, payload, initialReconciliation)
default:
return fmt.Errorf("unsupported sync entity type: %s", op.EntityType)
}
@ -3396,10 +3640,19 @@ type syncFilePayload struct {
Path string `json:"path"`
Content string `json:"content"`
DataBase64 *string `json:"dataBase64"`
ContentHash string `json:"contentHash"`
FromPath string `json:"fromPath"`
ToPath string `json:"toPath"`
}
type syncWorkspacePayload struct {
WorkspaceID string `json:"workspaceId"`
Path string `json:"path"`
PreviousPath string `json:"previousPath,omitempty"`
Name string `json:"name"`
Metadata workspace.Metadata `json:"metadata"`
}
func parseSyncFilePayload(payloadJSON string) (syncFilePayload, error) {
if payloadJSON == "" {
return syncFilePayload{}, nil
@ -3411,13 +3664,64 @@ func parseSyncFilePayload(payloadJSON string) (syncFilePayload, error) {
return payload, nil
}
func (a *App) applyRemoteFileOp(op syncsvc.Op, payload syncFilePayload) error {
func parseSyncWorkspacePayload(payloadJSON string) (syncWorkspacePayload, error) {
if payloadJSON == "" {
return syncWorkspacePayload{}, fmt.Errorf("workspace sync payload is empty")
}
var payload syncWorkspacePayload
if err := json.Unmarshal([]byte(payloadJSON), &payload); err != nil {
return syncWorkspacePayload{}, fmt.Errorf("invalid workspace sync payload: %w", err)
}
if payload.WorkspaceID == "" {
return syncWorkspacePayload{}, fmt.Errorf("workspace sync payload is missing workspaceId")
}
if payload.Path == "" {
payload.Path = payload.Name
}
return payload, nil
}
func (a *App) applyRemoteWorkspaceOp(op syncsvc.Op, payload syncWorkspacePayload) error {
if a.workspace == nil {
return fmt.Errorf("workspace service not initialized")
}
if payload.WorkspaceID != op.EntityID {
return fmt.Errorf("workspace identity mismatch: entity %s payload %s", op.EntityID, payload.WorkspaceID)
}
switch op.OpType {
case syncsvc.OpCreate:
_, err := a.workspace.CreateWorkspaceFromSync(payload.Path, payload.WorkspaceID, payload.Metadata)
return err
case syncsvc.OpRename:
return a.workspace.RenameWorkspaceFromSync(payload.WorkspaceID, payload.PreviousPath, payload.Path)
case syncsvc.OpTrash:
_, err := a.workspace.TrashWorkspaceFromSync(payload.WorkspaceID, payload.Path)
return err
case syncsvc.OpRestore:
_, err := a.workspace.RestoreWorkspaceFromSync(payload.WorkspaceID, payload.Path)
return err
default:
return fmt.Errorf("unsupported workspace sync op type: %s", op.OpType)
}
}
func (a *App) applyRemoteFileOp(op syncsvc.Op, payload syncFilePayload, initialReconciliation bool) error {
switch op.OpType {
case syncsvc.OpCreate:
path := syncPayloadPath(op, payload)
if path == "" {
return fmt.Errorf("missing file path")
}
matches, exists, err := a.remoteFileMatches(path, payload)
if err != nil {
return err
}
if matches {
return nil
}
if exists {
return fmt.Errorf("conflict: remote create would replace local file %s", path)
}
if payload.DataBase64 != nil {
return a.files.WriteVaultFileBytes(path, *payload.DataBase64, corefiles.WriteOptions{CreateIfMissing: true})
}
@ -3427,15 +3731,44 @@ func (a *App) applyRemoteFileOp(op syncsvc.Op, payload syncFilePayload) error {
if path == "" {
return fmt.Errorf("missing file path")
}
if payload.DataBase64 != nil {
return a.files.WriteVaultFileBytes(path, *payload.DataBase64, corefiles.WriteOptions{CreateIfMissing: true, Overwrite: true})
matches, exists, err := a.remoteFileMatches(path, payload)
if err != nil {
return err
}
return a.files.WriteVaultTextFile(path, payload.Content, corefiles.WriteOptions{CreateIfMissing: true, Overwrite: true})
if matches {
return nil
}
if initialReconciliation && exists {
return fmt.Errorf("conflict: initial reconciliation would replace local file %s", path)
}
if pending, err := a.syncSvc.HasUnpushedPath(path); err != nil {
return err
} else if pending {
return fmt.Errorf("conflict: remote update would replace unpushed local file %s", path)
}
if payload.DataBase64 != nil {
return a.files.WriteVaultFileBytes(path, *payload.DataBase64, corefiles.WriteOptions{CreateIfMissing: !exists, Overwrite: exists})
}
return a.files.WriteVaultTextFile(path, payload.Content, corefiles.WriteOptions{CreateIfMissing: !exists, Overwrite: exists})
case syncsvc.OpDelete:
path := syncPayloadPath(op, payload)
if path == "" {
return fmt.Errorf("missing file path")
}
if initialReconciliation {
exists, err := a.syncPathExists(path)
if err != nil {
return err
}
if exists {
return fmt.Errorf("conflict: initial reconciliation would delete local file %s", path)
}
}
if pending, err := a.syncSvc.HasUnpushedPath(path); err != nil {
return err
} else if pending {
return fmt.Errorf("conflict: remote delete would remove unpushed local file %s", path)
}
_, err := a.files.TrashVaultPath(path)
if isSyncNotFound(err) {
return nil
@ -3449,6 +3782,24 @@ func (a *App) applyRemoteFileOp(op syncsvc.Op, payload syncFilePayload) error {
if fromPath == "" || payload.ToPath == "" {
return fmt.Errorf("missing file move path")
}
if initialReconciliation {
fromExists, err := a.syncPathExists(fromPath)
if err != nil {
return err
}
toExists, err := a.syncPathExists(payload.ToPath)
if err != nil {
return err
}
if fromExists || toExists {
return fmt.Errorf("conflict: initial reconciliation would move local file %s", fromPath)
}
}
if pending, err := a.syncSvc.HasUnpushedPath(fromPath); err != nil {
return err
} else if pending {
return fmt.Errorf("conflict: remote move would replace unpushed local file %s", fromPath)
}
err := a.files.MoveVaultPath(fromPath, payload.ToPath, corefiles.MoveOptions{})
if isSyncNotFound(err) {
return nil
@ -3459,7 +3810,54 @@ func (a *App) applyRemoteFileOp(op syncsvc.Op, payload syncFilePayload) error {
}
}
func (a *App) applyRemoteFolderOp(op syncsvc.Op, payload syncFilePayload) error {
func (a *App) remoteFileMatches(path string, payload syncFilePayload) (matches, exists bool, err error) {
normalized, err := corefiles.NormalizeRelativeFile(path)
if err != nil {
return false, false, err
}
info, err := os.Lstat(filepath.Join(a.vaultPath(), filepath.FromSlash(normalized)))
if err != nil {
if os.IsNotExist(err) {
return false, false, nil
}
return false, false, err
}
if info.Mode()&os.ModeSymlink != 0 || !info.Mode().IsRegular() {
return false, true, fmt.Errorf("conflict: remote file target is not a regular file: %s", normalized)
}
want, err := remotePayloadHash(payload)
if err != nil {
return false, true, err
}
file, err := os.Open(filepath.Join(a.vaultPath(), filepath.FromSlash(normalized)))
if err != nil {
return false, true, err
}
defer file.Close()
hash := sha256.New()
if _, err := io.Copy(hash, file); err != nil {
return false, true, err
}
return fmt.Sprintf("%x", hash.Sum(nil)) == want, true, nil
}
func remotePayloadHash(payload syncFilePayload) (string, error) {
if payload.ContentHash != "" {
return payload.ContentHash, nil
}
data := []byte(payload.Content)
if payload.DataBase64 != nil {
decoded, err := base64.StdEncoding.DecodeString(*payload.DataBase64)
if err != nil {
return "", fmt.Errorf("invalid remote base64 payload: %w", err)
}
data = decoded
}
sum := sha256.Sum256(data)
return fmt.Sprintf("%x", sum[:]), nil
}
func (a *App) applyRemoteFolderOp(op syncsvc.Op, payload syncFilePayload, initialReconciliation bool) error {
switch op.OpType {
case syncsvc.OpCreate:
path := syncPayloadPath(op, payload)
@ -3476,6 +3874,15 @@ func (a *App) applyRemoteFolderOp(op syncsvc.Op, payload syncFilePayload) error
if path == "" {
return fmt.Errorf("missing folder path")
}
if initialReconciliation {
exists, err := a.syncPathExists(path)
if err != nil {
return err
}
if exists {
return fmt.Errorf("conflict: initial reconciliation would delete local folder %s", path)
}
}
_, err := a.files.TrashVaultPath(path)
if isSyncNotFound(err) {
return nil
@ -3489,6 +3896,19 @@ func (a *App) applyRemoteFolderOp(op syncsvc.Op, payload syncFilePayload) error
if fromPath == "" || payload.ToPath == "" {
return fmt.Errorf("missing folder move path")
}
if initialReconciliation {
fromExists, err := a.syncPathExists(fromPath)
if err != nil {
return err
}
toExists, err := a.syncPathExists(payload.ToPath)
if err != nil {
return err
}
if fromExists || toExists {
return fmt.Errorf("conflict: initial reconciliation would move local folder %s", fromPath)
}
}
err := a.files.MoveVaultPath(fromPath, payload.ToPath, corefiles.MoveOptions{})
if isSyncNotFound(err) {
return nil
@ -3499,6 +3919,21 @@ func (a *App) applyRemoteFolderOp(op syncsvc.Op, payload syncFilePayload) error
}
}
func (a *App) syncPathExists(path string) (bool, error) {
normalized, err := corefiles.NormalizeRelativeFile(path)
if err != nil {
return false, err
}
_, err = os.Lstat(filepath.Join(a.vaultPath(), filepath.FromSlash(normalized)))
if err == nil {
return true, nil
}
if os.IsNotExist(err) {
return false, nil
}
return false, err
}
func syncPayloadPath(op syncsvc.Op, payload syncFilePayload) string {
if payload.Path != "" {
return payload.Path

468
internal/api/app_test.go
View File

@ -175,6 +175,12 @@ func newSyncFilesTestApp(t *testing.T, perms []string, deviceID string) (*App, s
t.Helper()
app, root := newFilesTestApp(t, perms)
app.syncSvc = syncsvc.NewService(root, deviceID)
if _, err := app.syncSvc.ScanAndRecord(); err != nil {
t.Fatalf("initial sync snapshot: %v", err)
}
if err := app.syncSvc.SetBootstrapComplete(true); err != nil {
t.Fatalf("mark test sync bootstrap complete: %v", err)
}
app.appSettings = appsettings.NewManager(filepath.Join(t.TempDir(), "config.json"))
if err := app.appSettings.Load(); err != nil {
t.Fatalf("settings Load: %v", err)
@ -1692,6 +1698,63 @@ func TestApplyRemoteOpSkipsLocalDevice(t *testing.T) {
}
}
func TestApplyRemoteWorkspaceLifecyclePreservesDurableIdentity(t *testing.T) {
app, root := newSyncFilesTestApp(t, []string{"files.read", "files.write", "files.delete"}, "local-device")
app.workspace = workspace.NewManager(root)
if err := app.workspace.Load(); err != nil {
t.Fatal(err)
}
workspaceID := "5f0f96d9-61c8-4b6b-8c3a-a1b9a0f40002"
payload := syncWorkspacePayload{
WorkspaceID: workspaceID,
Path: "Remote",
Name: "Remote",
Metadata: workspace.Metadata{
WorkspaceID: workspaceID,
WorkspaceName: "Remote",
Folders: map[string]string{"notes": "Notes"},
Features: map[string]bool{"files": true},
},
}
encoded, err := json.Marshal(payload)
if err != nil {
t.Fatal(err)
}
create := syncsvc.Op{OpID: "workspace-create", DeviceID: "remote-device", EntityType: syncsvc.EntityWorkspace, EntityID: workspaceID, OpType: syncsvc.OpCreate, PayloadJSON: string(encoded)}
if err := app.applyRemoteOp(create); err != nil {
t.Fatalf("apply workspace create: %v", err)
}
if err := app.applyRemoteOp(create); err != nil {
t.Fatalf("replay workspace create: %v", err)
}
if identity, err := app.workspace.GetWorkspaceIdentity("Remote"); err != nil || identity.WorkspaceID != workspaceID {
t.Fatalf("remote workspace identity = %+v err=%v", identity, err)
}
payload.PreviousPath = "Remote"
payload.Path = "Remote-Renamed"
payload.Name = "Remote-Renamed"
encoded, _ = json.Marshal(payload)
rename := syncsvc.Op{OpID: "workspace-rename", DeviceID: "remote-device", EntityType: syncsvc.EntityWorkspace, EntityID: workspaceID, OpType: syncsvc.OpRename, PayloadJSON: string(encoded)}
if err := app.applyRemoteOp(rename); err != nil {
t.Fatalf("apply workspace rename: %v", err)
}
trash := syncsvc.Op{OpID: "workspace-trash", DeviceID: "remote-device", EntityType: syncsvc.EntityWorkspace, EntityID: workspaceID, OpType: syncsvc.OpTrash, PayloadJSON: string(encoded)}
if err := app.applyRemoteOp(trash); err != nil {
t.Fatalf("apply workspace trash: %v", err)
}
payload.Path = "Remote-Restored"
payload.Name = "Remote-Restored"
encoded, _ = json.Marshal(payload)
restore := syncsvc.Op{OpID: "workspace-restore", DeviceID: "remote-device", EntityType: syncsvc.EntityWorkspace, EntityID: workspaceID, OpType: syncsvc.OpRestore, PayloadJSON: string(encoded)}
if err := app.applyRemoteOp(restore); err != nil {
t.Fatalf("apply workspace restore: %v", err)
}
if identity, err := app.workspace.GetWorkspaceIdentity("Remote-Restored"); err != nil || identity.WorkspaceID != workspaceID {
t.Fatalf("restored workspace identity = %+v err=%v", identity, err)
}
}
func TestFileBridgeRecordsSyncOps(t *testing.T) {
app, _ := newSyncFilesTestApp(t, []string{"files.read", "files.write", "files.delete"}, "local-device")
@ -1718,8 +1781,8 @@ func TestFileBridgeRecordsSyncOps(t *testing.T) {
if err != nil {
t.Fatalf("GetUnpushedOps: %v", err)
}
if len(ops) != 6 {
t.Fatalf("ops len = %d, want 6: %#v", len(ops), ops)
if len(ops) != 7 {
t.Fatalf("ops len = %d, want 7: %#v", len(ops), ops)
}
want := []struct {
@ -1732,7 +1795,10 @@ func TestFileBridgeRecordsSyncOps(t *testing.T) {
{syncsvc.EntityFile, "Docs/one.txt", syncsvc.OpCreate, `"content":"hello"`},
{syncsvc.EntityFile, "Docs/one.txt", syncsvc.OpUpdate, `"content":"updated"`},
{syncsvc.EntityFile, "Docs/image.bin", syncsvc.OpCreate, `"dataBase64":"AQID"`},
{syncsvc.EntityFile, "Docs/one.txt", syncsvc.OpMove, `"toPath":"Docs/two.txt"`},
// Snapshot reconciliation represents external and API renames uniformly
// as a create at the destination followed by a delete at the source.
{syncsvc.EntityFile, "Docs/two.txt", syncsvc.OpCreate, `"content":"updated"`},
{syncsvc.EntityFile, "Docs/one.txt", syncsvc.OpDelete, `"path":"Docs/one.txt"`},
{syncsvc.EntityFile, "Docs/two.txt", syncsvc.OpDelete, `"path":"Docs/two.txt"`},
}
for i, w := range want {
@ -1854,8 +1920,8 @@ func TestSyncNowPushesLocalOpsAndAppliesPulledFileOps(t *testing.T) {
t.Fatalf("pushed ops len = %d, want 2", len(pushedOps))
}
for i, op := range pushedOps {
if op.LastSeenServerSeq != 0 {
t.Fatalf("pushed op[%d] last seen = %d, want 0", i, op.LastSeenServerSeq)
if op.LastSeenServerSeq != 2 {
t.Fatalf("pushed op[%d] last seen = %d, want 2 after initial pull", i, op.LastSeenServerSeq)
}
}
@ -1882,6 +1948,323 @@ func TestSyncNowPushesLocalOpsAndAppliesPulledFileOps(t *testing.T) {
}
}
func TestSyncNowStopsAtFailedRemoteOperationAndRetriesAfterRestart(t *testing.T) {
app, root := newSyncFilesTestApp(t, []string{"files.read", "files.write", "files.delete"}, "local-device")
pullCalls := 0
server := newLocalHTTPTestServer(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer device-token" {
http.Error(w, "missing auth", http.StatusUnauthorized)
return
}
w.Header().Set("Content-Type", "application/json")
switch r.URL.Path {
case "/api/v1/sync/pull":
pullCalls++
_ = json.NewEncoder(w).Encode(map[string]interface{}{
"server_sequence": 3,
"ops": []map[string]interface{}{
{
"op_id": "remote-folder",
"server_sequence": 1,
"device_id": "remote-device",
"entity_type": syncsvc.EntityFolder,
"entity_id": "Remote",
"op_type": syncsvc.OpCreate,
"payload_json": `{"path":"Remote"}`,
},
{
"op_id": "blocked-file",
"server_sequence": 2,
"device_id": "remote-device",
"entity_type": syncsvc.EntityFile,
"entity_id": "Missing/blocked.txt",
"op_type": syncsvc.OpCreate,
"payload_json": `{"path":"Missing/blocked.txt","content":"blocked"}`,
},
{
"op_id": "after-failure",
"server_sequence": 3,
"device_id": "remote-device",
"entity_type": syncsvc.EntityFile,
"entity_id": "Remote/after.txt",
"op_type": syncsvc.OpCreate,
"payload_json": `{"path":"Remote/after.txt","content":"must wait"}`,
},
},
})
case "/api/v1/sync/push":
_ = json.NewEncoder(w).Encode(map[string]interface{}{"accepted": []string{}, "count": 0, "conflicts": []map[string]interface{}{}})
default:
http.NotFound(w, r)
}
}))
defer server.Close()
if err := app.syncSvc.SetState(server.URL, ""); err != nil {
t.Fatal(err)
}
if err := syncsvc.SaveDeviceToken(root, "device-token"); err != nil {
t.Fatal(err)
}
if _, err := app.syncNow(); err == nil || !strings.Contains(err.Error(), "sequence 2") {
t.Fatalf("first sync error = %v, want failed sequence", err)
}
assertLastPullSequence(t, app.syncSvc, 1)
if _, errStr := app.GetVaultFileMetadata("files.plugin", "Remote/after.txt"); !strings.Contains(errStr, "not-found") {
t.Fatalf("operation after failure applied early: %q", errStr)
}
cfg := app.appSettings.Get()
if cfg.Sync.LastError == "" || !strings.Contains(cfg.Sync.LastError, "sequence 2") || !strings.Contains(cfg.Sync.LastError, "Missing/blocked.txt") {
t.Fatalf("sync status after failed apply = %#v", cfg.Sync)
}
if err := os.Mkdir(filepath.Join(root, "Missing"), 0o755); err != nil {
t.Fatal(err)
}
restarted := &App{
vault: app.vault,
files: app.files,
plugins: app.plugins,
appSettings: app.appSettings,
syncSvc: syncsvc.NewService(root, ""),
}
if _, err := restarted.syncNow(); err != nil {
t.Fatalf("retry after restart: %v", err)
}
if pullCalls < 2 {
t.Fatalf("pull calls = %d, want retry", pullCalls)
}
assertLastPullSequence(t, restarted.syncSvc, 3)
expectText(t, restarted, "Missing/blocked.txt", "blocked")
expectText(t, restarted, "Remote/after.txt", "must wait")
}
func TestSyncNowBootstrapsExistingLocalSnapshotAfterInitialPull(t *testing.T) {
app, root := newFilesTestApp(t, []string{"files.read", "files.write", "files.delete"})
if err := os.Mkdir(filepath.Join(root, "Existing"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, "Existing", "before-connect.txt"), []byte("local before connect"), 0o644); err != nil {
t.Fatal(err)
}
app.syncSvc = syncsvc.NewService(root, "local-device")
if _, err := app.syncSvc.ScanAndRecord(); err != nil {
t.Fatalf("initial baseline scan: %v", err)
}
app.appSettings = appsettings.NewManager(filepath.Join(t.TempDir(), "config.json"))
if err := app.appSettings.Load(); err != nil {
t.Fatal(err)
}
var pushed []syncsvc.PushOp
server := newLocalHTTPTestServer(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer device-token" {
http.Error(w, "missing auth", http.StatusUnauthorized)
return
}
w.Header().Set("Content-Type", "application/json")
switch r.URL.Path {
case "/api/v1/sync/pull":
_ = json.NewEncoder(w).Encode(map[string]interface{}{"server_sequence": 0, "ops": []map[string]interface{}{}})
case "/api/v1/sync/push":
var request struct {
Ops []syncsvc.PushOp `json:"ops"`
}
if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
pushed = request.Ops
accepted := make([]string, 0, len(request.Ops))
for _, op := range request.Ops {
accepted = append(accepted, op.OpID)
}
_ = json.NewEncoder(w).Encode(map[string]interface{}{"accepted": accepted, "count": len(accepted), "conflicts": []map[string]interface{}{}})
default:
http.NotFound(w, r)
}
}))
defer server.Close()
if err := app.syncSvc.SetState(server.URL, ""); err != nil {
t.Fatal(err)
}
if err := syncsvc.SaveDeviceToken(root, "device-token"); err != nil {
t.Fatal(err)
}
result, err := app.syncNow()
if err != nil {
t.Fatalf("syncNow: %v", err)
}
if result["pushed"] != len(pushed) || len(pushed) < 2 {
t.Fatalf("result=%#v pushed=%#v, want initial entries", result, pushed)
}
foundFolder, foundFile := false, false
for _, op := range pushed {
if op.EntityID == "Existing" && op.EntityType == syncsvc.EntityFolder && op.OpType == syncsvc.OpCreate {
foundFolder = true
}
if op.EntityID == "Existing/before-connect.txt" && op.EntityType == syncsvc.EntityFile && op.OpType == syncsvc.OpCreate {
foundFile = true
}
if strings.Contains(op.EntityID, "/.verstak/") {
t.Fatalf("ordinary bootstrap operation leaked internal path: %+v", op)
}
}
if !foundFolder || !foundFile {
t.Fatalf("bootstrap push = %#v, missing existing file or folder", pushed)
}
bootstrapped, err := app.syncSvc.BootstrapComplete()
if err != nil || !bootstrapped {
t.Fatalf("bootstrap complete = %v err=%v", bootstrapped, err)
}
}
func TestInitialSyncConflictDoesNotOverwriteLocalFileOrAdvanceCursor(t *testing.T) {
app, root := newFilesTestApp(t, []string{"files.read", "files.write", "files.delete"})
if err := os.WriteFile(filepath.Join(root, "same-name.txt"), []byte("local value"), 0o644); err != nil {
t.Fatal(err)
}
app.syncSvc = syncsvc.NewService(root, "local-device")
if _, err := app.syncSvc.ScanAndRecord(); err != nil {
t.Fatalf("initial baseline scan: %v", err)
}
app.appSettings = appsettings.NewManager(filepath.Join(t.TempDir(), "config.json"))
if err := app.appSettings.Load(); err != nil {
t.Fatal(err)
}
pushCalls := 0
server := newLocalHTTPTestServer(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer device-token" {
http.Error(w, "missing auth", http.StatusUnauthorized)
return
}
w.Header().Set("Content-Type", "application/json")
switch r.URL.Path {
case "/api/v1/sync/pull":
_ = json.NewEncoder(w).Encode(map[string]interface{}{
"server_sequence": 1,
"ops": []map[string]interface{}{{
"op_id": "remote-update",
"server_sequence": 1,
"device_id": "remote-device",
"entity_type": syncsvc.EntityFile,
"entity_id": "same-name.txt",
"op_type": syncsvc.OpUpdate,
"payload_json": `{"path":"same-name.txt","content":"remote value"}`,
}},
})
case "/api/v1/sync/push":
pushCalls++
_ = json.NewEncoder(w).Encode(map[string]interface{}{"accepted": []string{}, "count": 0, "conflicts": []map[string]interface{}{}})
default:
http.NotFound(w, r)
}
}))
defer server.Close()
if err := app.syncSvc.SetState(server.URL, ""); err != nil {
t.Fatal(err)
}
if err := syncsvc.SaveDeviceToken(root, "device-token"); err != nil {
t.Fatal(err)
}
if _, err := app.syncNow(); err == nil || !strings.Contains(err.Error(), "initial reconciliation") || !strings.Contains(err.Error(), "same-name.txt") {
t.Fatalf("sync error = %v, want explicit initial conflict", err)
}
expectText(t, app, "same-name.txt", "local value")
assertLastPullSequence(t, app.syncSvc, 0)
if pushCalls != 0 {
t.Fatalf("push calls = %d, conflict must not overwrite remote state", pushCalls)
}
bootstrapped, err := app.syncSvc.BootstrapComplete()
if err != nil || bootstrapped {
t.Fatalf("bootstrap state = %v err=%v, conflict must remain unresolved", bootstrapped, err)
}
}
func TestInitialSyncEmptyVaultRestoresRemoteDataWithoutDeletePush(t *testing.T) {
app, root := newFilesTestApp(t, []string{"files.read", "files.write", "files.delete"})
app.syncSvc = syncsvc.NewService(root, "empty-device")
if _, err := app.syncSvc.ScanAndRecord(); err != nil {
t.Fatalf("initial empty baseline: %v", err)
}
app.appSettings = appsettings.NewManager(filepath.Join(t.TempDir(), "config.json"))
if err := app.appSettings.Load(); err != nil {
t.Fatal(err)
}
var pushed []syncsvc.PushOp
server := newLocalHTTPTestServer(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get("Authorization") != "Bearer device-token" {
http.Error(w, "missing auth", http.StatusUnauthorized)
return
}
w.Header().Set("Content-Type", "application/json")
switch r.URL.Path {
case "/api/v1/sync/pull":
_ = json.NewEncoder(w).Encode(map[string]interface{}{
"server_sequence": 1,
"ops": []map[string]interface{}{{
"op_id": "remote-file",
"server_sequence": 1,
"device_id": "remote-device",
"entity_type": syncsvc.EntityFile,
"entity_id": "restored.txt",
"op_type": syncsvc.OpCreate,
"payload_json": `{"path":"restored.txt","content":"from remote"}`,
}},
})
case "/api/v1/sync/push":
var request struct {
Ops []syncsvc.PushOp `json:"ops"`
}
if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
pushed = append(pushed, request.Ops...)
_ = json.NewEncoder(w).Encode(map[string]interface{}{"accepted": []string{}, "count": 0, "conflicts": []map[string]interface{}{}})
default:
http.NotFound(w, r)
}
}))
defer server.Close()
if err := app.syncSvc.SetState(server.URL, ""); err != nil {
t.Fatal(err)
}
if err := syncsvc.SaveDeviceToken(root, "device-token"); err != nil {
t.Fatal(err)
}
result, err := app.syncNow()
if err != nil {
t.Fatalf("sync empty vault: %v", err)
}
expectText(t, app, "restored.txt", "from remote")
for _, op := range pushed {
if op.OpType == syncsvc.OpDelete {
t.Fatalf("empty vault published delete operation: %+v", op)
}
}
if result["pushed"] != 0 && len(pushed) == 0 {
t.Fatalf("result reports pushed operations without a push payload: %#v", result)
}
assertLastPullSequence(t, app.syncSvc, 1)
}
func assertLastPullSequence(t *testing.T, service *syncsvc.Service, want int) {
t.Helper()
_, _, got, _, err := service.GetState()
if err != nil {
t.Fatalf("GetState: %v", err)
}
if got != want {
t.Fatalf("last pull sequence = %d, want %d", got, want)
}
}
func TestSyncConfigurePairsCurrentVaultID(t *testing.T) {
app, _ := newSyncFilesTestApp(t, []string{"files.read", "files.write", "files.delete"}, "local-device")
meta := app.vault.GetVaultMeta()
@ -1911,7 +2294,7 @@ func TestSyncConfigurePairsCurrentVaultID(t *testing.T) {
}))
defer server.Close()
if err := app.syncConfigure(server.URL, "alice", "secret"); err != nil {
if err := app.syncConfigure(server.URL, "alice", "secret", ""); err != nil {
t.Fatalf("syncConfigure: %v", err)
}
if pairedVaultID != meta.VaultID {
@ -1919,6 +2302,79 @@ func TestSyncConfigurePairsCurrentVaultID(t *testing.T) {
}
}
func TestSyncConfigurePairsSpecifiedRemoteVaultID(t *testing.T) {
app, _ := newSyncFilesTestApp(t, []string{"files.read", "files.write", "files.delete"}, "local-device")
const remoteVaultID = "existing-remote-vault"
var pairedVaultID string
server := newLocalHTTPTestServer(t, http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
if r.URL.Path != "/api/client/pair" {
http.NotFound(w, r)
return
}
var request struct {
VaultID string `json:"vault_id"`
}
if err := json.NewDecoder(r.Body).Decode(&request); err != nil {
http.Error(w, err.Error(), http.StatusBadRequest)
return
}
pairedVaultID = request.VaultID
w.Header().Set("Content-Type", "application/json")
_ = json.NewEncoder(w).Encode(map[string]string{
"device_id": "paired-device",
"device_token": "paired-token",
})
}))
defer server.Close()
if err := app.syncConfigure(server.URL, "alice", "secret", remoteVaultID); err != nil {
t.Fatalf("syncConfigure: %v", err)
}
if pairedVaultID != remoteVaultID {
t.Fatalf("paired vault ID = %q, want %q", pairedVaultID, remoteVaultID)
}
storedRemoteVaultID, err := app.syncSvc.RemoteVaultID()
if err != nil || storedRemoteVaultID != remoteVaultID {
t.Fatalf("stored remote vault ID = %q err=%v, want %q", storedRemoteVaultID, err, remoteVaultID)
}
bootstrapped, err := app.syncSvc.BootstrapComplete()
if err != nil || bootstrapped {
t.Fatalf("bootstrap state after new pairing = %v err=%v, want false", bootstrapped, err)
}
}
func TestSyncConfigureRefusesRemoteScopeChangeWithUnpushedOperations(t *testing.T) {
app, _ := newSyncFilesTestApp(t, []string{"files.read", "files.write", "files.delete"}, "local-device")
if err := app.syncSvc.SetState("https://old-sync.example.test", ""); err != nil {
t.Fatal(err)
}
if err := app.syncSvc.SetRemoteVaultID("old-remote-vault"); err != nil {
t.Fatal(err)
}
if err := app.syncSvc.RecordOp(syncsvc.EntityFile, "pending.txt", syncsvc.OpCreate, map[string]string{"path": "pending.txt", "content": "local"}); err != nil {
t.Fatal(err)
}
err := app.syncConfigure("https://new-sync.example.test", "alice", "secret", "new-remote-vault")
if err == nil || !strings.Contains(err.Error(), "unpushed local operation") {
t.Fatalf("scope-change error = %v, want pending-operation refusal", err)
}
}
func TestSyncStatusExposesPersistentScannerWarning(t *testing.T) {
app, _ := newSyncFilesTestApp(t, []string{"files.read", "files.write", "files.delete"}, "local-device")
if err := app.syncSvc.SetLastWarning("file-too-large: archive.bin"); err != nil {
t.Fatal(err)
}
status, err := app.syncStatus()
if err != nil {
t.Fatalf("syncStatus: %v", err)
}
if status.LastWarning != "file-too-large: archive.bin" {
t.Fatalf("last warning = %q", status.LastWarning)
}
}
func TestSyncNowHydratesLegacyVaultDeviceID(t *testing.T) {
app, root := newSyncFilesTestApp(t, []string{"files.read", "files.write", "files.delete"}, "wrong-global-device")
var pushedDeviceID string

101
internal/api/sync_real_server_test.go
View File

@ -2,10 +2,12 @@ package api
import (
"os"
"path/filepath"
"strings"
"testing"
corefiles "github.com/verstak/verstak-desktop/internal/core/files"
"github.com/verstak/verstak-desktop/internal/core/workspace"
)
func TestSyncNowAgainstRealServerTwoVaults(t *testing.T) {
@ -18,8 +20,16 @@ func TestSyncNowAgainstRealServerTwoVaults(t *testing.T) {
t.Skip("set VERSTAK_SYNC_SMOKE_* env vars to run the real sync-server smoke test")
}
appA, _ := newSyncFilesTestApp(t, []string{"files.read", "files.write", "files.delete"}, deviceA)
appB, _ := newSyncFilesTestApp(t, []string{"files.read", "files.write", "files.delete"}, deviceB)
appA, rootA := newSyncFilesTestApp(t, []string{"files.read", "files.write", "files.delete"}, deviceA)
appB, rootB := newSyncFilesTestApp(t, []string{"files.read", "files.write", "files.delete"}, deviceB)
appA.workspace = workspace.NewManager(rootA)
appB.workspace = workspace.NewManager(rootB)
if err := appA.workspace.Load(); err != nil {
t.Fatalf("load workspace A: %v", err)
}
if err := appB.workspace.Load(); err != nil {
t.Fatalf("load workspace B: %v", err)
}
if err := appA.syncSvc.SetState(serverURL, apiKeyA); err != nil {
t.Fatalf("appA SetState: %v", err)
}
@ -43,8 +53,8 @@ func TestSyncNowAgainstRealServerTwoVaults(t *testing.T) {
if errStr := appB.MoveVaultPath("files.plugin", "Shared/one.txt", "Shared/two.txt", corefiles.MoveOptions{}); errStr != "" {
t.Fatalf("appB move: %s", errStr)
}
expectSyncCounts(t, appB, 2, 2)
expectSyncCounts(t, appA, 0, 2)
expectSyncCounts(t, appB, 3, 3)
expectSyncCounts(t, appA, 0, 3)
expectText(t, appA, "Shared/two.txt", "from B")
if _, errStr := appA.TrashVaultPath("files.plugin", "Shared/two.txt"); errStr != "" {
@ -68,8 +78,8 @@ func TestSyncNowAgainstRealServerTwoVaults(t *testing.T) {
if errStr := appA.MoveVaultPath("files.plugin", "Shared/Folder", "Shared/Archive", corefiles.MoveOptions{}); errStr != "" {
t.Fatalf("appA move folder: %s", errStr)
}
expectSyncCounts(t, appA, 1, 1)
expectSyncCounts(t, appB, 0, 1)
expectSyncCounts(t, appA, 2, 2)
expectSyncCounts(t, appB, 0, 2)
if _, errStr := appB.GetVaultFileMetadata("files.plugin", "Shared/Folder"); !strings.Contains(errStr, "not-found") {
t.Fatalf("appB moved folder old metadata err = %q, want not-found", errStr)
}
@ -85,6 +95,63 @@ func TestSyncNowAgainstRealServerTwoVaults(t *testing.T) {
if _, errStr := appA.GetVaultFileMetadata("files.plugin", "Shared/Archive"); !strings.Contains(errStr, "not-found") {
t.Fatalf("appA deleted folder metadata err = %q, want not-found", errStr)
}
if err := os.WriteFile(filepath.Join(rootA, "Shared", "external.txt"), []byte("external while running"), 0o644); err != nil {
t.Fatalf("external create: %v", err)
}
expectSyncCounts(t, appA, 1, 1)
expectSyncCounts(t, appB, 0, 1)
expectText(t, appB, "Shared/external.txt", "external while running")
assertNoUnpushedOps(t, appB)
if err := os.WriteFile(filepath.Join(rootB, "Shared", "external.txt"), []byte("external while closed"), 0o644); err != nil {
t.Fatalf("offline external update: %v", err)
}
if err := os.WriteFile(filepath.Join(rootB, "Shared", "offline-created.txt"), []byte("created while closed"), 0o644); err != nil {
t.Fatalf("offline external create: %v", err)
}
if err := os.Remove(filepath.Join(rootB, "Shared", "external.txt")); err != nil {
t.Fatalf("offline external delete: %v", err)
}
expectSyncCounts(t, appB, 2, 2)
expectSyncCounts(t, appA, 0, 2)
if _, errStr := appA.GetVaultFileMetadata("files.plugin", "Shared/external.txt"); !strings.Contains(errStr, "not-found") {
t.Fatalf("offline deleted file remained on appA: %q", errStr)
}
expectText(t, appA, "Shared/offline-created.txt", "created while closed")
assertNoUnpushedOps(t, appA)
deal, errStr := appA.CreateWorkspace("Synced Deal", "minimal")
if errStr != "" {
t.Fatalf("appA CreateWorkspace: %s", errStr)
}
expectSyncCounts(t, appA, 1, 1)
expectSyncCounts(t, appB, 0, 1)
assertWorkspaceIdentity(t, appB, "Synced Deal", deal.ID)
if errStr := appA.RenameWorkspace("Synced Deal", "Renamed Deal"); errStr != "" {
t.Fatalf("appA RenameWorkspace: %s", errStr)
}
expectSyncCounts(t, appA, 1, 1)
expectSyncCounts(t, appB, 0, 1)
assertWorkspaceIdentity(t, appB, "Renamed Deal", deal.ID)
trash, errStr := appA.TrashWorkspace("Renamed Deal")
if errStr != "" {
t.Fatalf("appA TrashWorkspace: %s", errStr)
}
expectSyncCounts(t, appA, 1, 1)
expectSyncCounts(t, appB, 0, 1)
if _, err := appB.workspace.GetWorkspaceIdentity("Renamed Deal"); err == nil {
t.Fatal("trashed workspace is still active on appB")
}
if _, errStr := appA.RestoreWorkspaceTrash(trash.TrashID, "Restored Deal"); errStr != "" {
t.Fatalf("appA RestoreWorkspaceTrash: %s", errStr)
}
expectSyncCounts(t, appA, 1, 1)
expectSyncCounts(t, appB, 0, 1)
assertWorkspaceIdentity(t, appB, "Restored Deal", deal.ID)
}
func expectSyncCounts(t *testing.T, app *App, pushed, pulled int) {
@ -108,3 +175,25 @@ func expectText(t *testing.T, app *App, path, want string) {
t.Fatalf("ReadVaultTextFile(%s) = %q, want %q", path, text, want)
}
}
func assertNoUnpushedOps(t *testing.T, app *App) {
t.Helper()
ops, err := app.syncSvc.GetUnpushedOps()
if err != nil {
t.Fatalf("GetUnpushedOps: %v", err)
}
if len(ops) != 0 {
t.Fatalf("remote operations were echoed as local operations: %#v", ops)
}
}
func assertWorkspaceIdentity(t *testing.T, app *App, name, wantID string) {
t.Helper()
identity, err := app.workspace.GetWorkspaceIdentity(name)
if err != nil {
t.Fatalf("GetWorkspaceIdentity(%s): %v", name, err)
}
if identity.WorkspaceID != wantID {
t.Fatalf("workspace %s ID = %s, want %s", name, identity.WorkspaceID, wantID)
}
}

15
internal/core/files/path_policy.go
View File

@ -25,8 +25,7 @@ func IsReservedPath(relativePath string) bool {
if cleaned == "" {
return false
}
first := strings.Split(cleaned, "/")[0]
return strings.EqualFold(first, ".verstak")
return containsReservedSegment(cleaned)
}
func normalizeRelativePath(input string, allowRoot bool) (string, error) {
@ -67,8 +66,16 @@ func IsReservedPathNoNormalize(cleaned string) bool {
if cleaned == "" {
return false
}
first := strings.Split(cleaned, "/")[0]
return strings.EqualFold(first, ".verstak")
return containsReservedSegment(cleaned)
}
func containsReservedSegment(cleaned string) bool {
for _, segment := range strings.Split(cleaned, "/") {
if strings.EqualFold(segment, ".verstak") {
return true
}
}
return false
}
func looksAbsolute(input string) bool {

4
internal/core/files/path_policy_test.go
View File

@ -43,6 +43,7 @@ func TestNormalizeRelativeFileRejectsUnsafePaths(t *testing.T) {
".verstak/vault.json",
"./.verstak",
".verstak/trash",
"Workspace/.verstak/workspace.json",
"folder/../.verstak",
".Verstak",
}
@ -70,6 +71,9 @@ func TestReservedPathPolicy(t *testing.T) {
if IsReservedPath("Notes/.verstak.md") {
t.Fatal("Notes/.verstak.md should not be reserved")
}
if !IsReservedPath("Workspace/.verstak/workspace.json") {
t.Fatal("nested .verstak should be reserved")
}
}
func TestNormalizeRelativeFileAcceptsOnlySlashSeparatedRelativePaths(t *testing.T) {

28
internal/core/filewatcher/service.go
View File

@ -40,6 +40,18 @@ type Service struct {
cancel chan struct{}
done chan struct{}
current map[string]snapshotEntry
onChange func()
}
// SetOnChange installs a lightweight notification used by core services that
// need a debounced reconciliation after the watcher has observed a change.
func (s *Service) SetOnChange(callback func()) {
if s == nil {
return
}
s.mu.Lock()
s.onChange = callback
s.mu.Unlock()
}
// NewService creates a watcher. The interval parameter is mainly for tests.
@ -133,22 +145,30 @@ func (s *Service) poll(root string) {
s.mu.Lock()
prev := s.current
s.current = next
callback := s.onChange
s.mu.Unlock()
changed := false
for path, entry := range next {
old, ok := prev[path]
if !ok {
s.publish(path, "external.create", entry.kind)
changed = true
continue
}
if entry.kind == entryFile && (entry.size != old.size || !entry.modTime.Equal(old.modTime)) {
s.publish(path, "external.update", entry.kind)
changed = true
}
}
for path, entry := range prev {
if _, ok := next[path]; !ok {
s.publish(path, "external.delete", entry.kind)
changed = true
}
}
if changed && callback != nil {
callback()
}
}
func (s *Service) publish(path, operation string, kind entryKind) {
@ -217,8 +237,12 @@ func kindFromInfo(info fs.FileInfo) entryKind {
}
func isReserved(rel string) bool {
first := strings.Split(filepath.ToSlash(rel), "/")[0]
return strings.EqualFold(first, ".verstak")
for _, segment := range strings.Split(filepath.ToSlash(rel), "/") {
if strings.EqualFold(segment, ".verstak") {
return true
}
}
return false
}
func workspaceRoot(path string) string {

20
internal/core/filewatcher/service_test.go
View File

@ -74,6 +74,26 @@ func TestServiceIgnoresReservedVerstakPaths(t *testing.T) {
}
}
func TestServiceCallsChangeCallbackForExternalChanges(t *testing.T) {
root := t.TempDir()
service := NewService(events.NewBus(), 10*time.Millisecond)
changed := make(chan struct{}, 1)
service.SetOnChange(func() { changed <- struct{}{} })
if err := service.Start(root); err != nil {
t.Fatalf("Start: %v", err)
}
t.Cleanup(service.Stop)
if err := os.WriteFile(filepath.Join(root, "external.txt"), []byte("change"), 0o644); err != nil {
t.Fatal(err)
}
select {
case <-changed:
case <-time.After(500 * time.Millisecond):
t.Fatal("timed out waiting for watcher callback")
}
}
func waitForEvent(t *testing.T, eventCh <-chan events.Event) events.Event {
t.Helper()
select {

175
internal/core/sync/service.go
View File

@ -5,6 +5,7 @@ import (
"fmt"
"os"
"path/filepath"
"strings"
"time"
"github.com/google/uuid"
@ -15,6 +16,7 @@ const (
EntityNote = "note"
EntityFile = "file"
EntityFolder = "folder"
EntityWorkspace = "workspace"
EntityAction = "action"
EntityWorklog = "worklog"
)
@ -24,6 +26,9 @@ const (
OpUpdate = "update"
OpDelete = "delete"
OpMove = "move"
OpRename = "rename"
OpTrash = "trash"
OpRestore = "restore"
)
// Op represents a sync operation.
@ -50,6 +55,9 @@ type syncState struct {
DeviceID string `json:"device_id"`
LastPullSeq int `json:"last_pull_seq"`
LastSyncAt string `json:"last_sync_at"`
BootstrapComplete bool `json:"bootstrap_complete"`
LastWarning string `json:"last_warning"`
RemoteVaultID string `json:"remote_vault_id"`
}
// Service records and manages sync operations using JSON file storage.
@ -81,6 +89,14 @@ func (s *Service) statePath() string {
return filepath.Join(s.syncDir(), "state.json")
}
func (s *Service) snapshotPath() string {
return filepath.Join(s.syncDir(), "snapshot.json")
}
func (s *Service) scanJournalPath() string {
return filepath.Join(s.syncDir(), "scan-journal.json")
}
func (s *Service) ensureDir() error {
return os.MkdirAll(s.syncDir(), 0o755)
}
@ -113,11 +129,33 @@ func (s *Service) RecordOp(entityType, entityID, opType string, payload interfac
CreatedAt: now,
}
return s.recordOps([]Op{op})
}
// recordOps is idempotent by op ID so a scanner recovery journal can safely
// resume after a crash between recording operations and replacing its snapshot.
func (s *Service) recordOps(newOps []Op) error {
if err := s.ensureDir(); err != nil {
return err
}
ops, err := s.loadOps()
if err != nil {
return err
}
existing := make(map[string]bool, len(ops))
for _, op := range ops {
existing[op.OpID] = true
}
for _, op := range newOps {
if op.OpID == "" || existing[op.OpID] {
continue
}
if op.ID == "" {
op.ID = op.OpID
}
ops = append(ops, op)
existing[op.OpID] = true
}
return s.saveOps(ops)
}
@ -160,6 +198,42 @@ func (s *Service) GetUnpushedOps() ([]Op, error) {
return unpushed, nil
}
// HasUnpushedPath reports whether a local operation still owns a path (or one
// of its descendants). Pull uses it to turn an incoming overwrite/delete into
// a visible conflict instead of silently replacing a local external edit.
func (s *Service) HasUnpushedPath(path string) (bool, error) {
ops, err := s.GetUnpushedOps()
if err != nil {
return false, err
}
for _, op := range ops {
if syncPathsOverlap(path, op.EntityID) {
return true, nil
}
var payload struct {
Path string `json:"path"`
FromPath string `json:"fromPath"`
ToPath string `json:"toPath"`
}
if op.PayloadJSON == "" || json.Unmarshal([]byte(op.PayloadJSON), &payload) != nil {
continue
}
if syncPathsOverlap(path, payload.Path) || syncPathsOverlap(path, payload.FromPath) || syncPathsOverlap(path, payload.ToPath) {
return true, nil
}
}
return false, nil
}
func syncPathsOverlap(left, right string) bool {
left = strings.Trim(left, "/")
right = strings.Trim(right, "/")
if left == "" || right == "" {
return false
}
return left == right || strings.HasPrefix(left, right+"/") || strings.HasPrefix(right, left+"/")
}
// MarkPushed marks ops as pushed to server.
func (s *Service) MarkPushed(opIDs []string) error {
ops, err := s.loadOps()
@ -244,6 +318,66 @@ func (s *Service) SetLastSyncAt(t string) error {
return s.saveState(st)
}
// BootstrapComplete reports whether the initial pull/reconcile/bootstrap cycle
// finished successfully for this vault connection.
func (s *Service) BootstrapComplete() (bool, error) {
st, err := s.loadState()
if err != nil {
return false, err
}
return st.BootstrapComplete, nil
}
// SetBootstrapComplete marks the initial reconciliation as complete only after
// all remote operations were applied and the local initial snapshot was queued.
func (s *Service) SetBootstrapComplete(done bool) error {
st, err := s.loadState()
if err != nil {
return err
}
st.BootstrapComplete = done
return s.saveState(st)
}
// LastWarning returns the persistent scanner warning shown by sync status.
func (s *Service) LastWarning() (string, error) {
st, err := s.loadState()
if err != nil {
return "", err
}
return st.LastWarning, nil
}
// SetLastWarning persists an unresolved scanner condition. An empty string
// clears the warning once a later complete scan no longer reports it.
func (s *Service) SetLastWarning(message string) error {
st, err := s.loadState()
if err != nil {
return err
}
st.LastWarning = message
return s.saveState(st)
}
// RemoteVaultID returns the optional target vault chosen while pairing a new
// local vault for restore. Empty means this vault's own durable ID was used.
func (s *Service) RemoteVaultID() (string, error) {
st, err := s.loadState()
if err != nil {
return "", err
}
return st.RemoteVaultID, nil
}
func (s *Service) SetRemoteVaultID(vaultID string) error {
st, err := s.loadState()
if err != nil {
return err
}
st.RemoteVaultID = vaultID
return s.saveState(st)
}
// GetDeviceID returns the device ID used by this service.
func (s *Service) GetDeviceID() string {
return s.deviceID
@ -288,7 +422,7 @@ func (s *Service) saveOps(ops []Op) error {
if err != nil {
return fmt.Errorf("marshal ops: %w", err)
}
return os.WriteFile(s.opsPath(), data, 0o644)
return atomicWriteFile(s.opsPath(), data, 0o600)
}
func (s *Service) loadState() (*syncState, error) {
@ -311,5 +445,42 @@ func (s *Service) saveState(st *syncState) error {
if err != nil {
return fmt.Errorf("marshal state: %w", err)
}
return os.WriteFile(s.statePath(), data, 0o644)
return atomicWriteFile(s.statePath(), data, 0o600)
}
func atomicWriteFile(path string, data []byte, perm os.FileMode) error {
if err := os.MkdirAll(filepath.Dir(path), 0o750); err != nil {
return err
}
tmp, err := os.CreateTemp(filepath.Dir(path), ".verstak-sync-*")
if err != nil {
return err
}
tmpPath := tmp.Name()
cleanup := true
defer func() {
if cleanup {
_ = os.Remove(tmpPath)
}
}()
if err := tmp.Chmod(perm); err != nil {
_ = tmp.Close()
return err
}
if _, err := tmp.Write(data); err != nil {
_ = tmp.Close()
return err
}
if err := tmp.Sync(); err != nil {
_ = tmp.Close()
return err
}
if err := tmp.Close(); err != nil {
return err
}
if err := os.Rename(tmpPath, path); err != nil {
return err
}
cleanup = false
return nil
}

789
internal/core/sync/snapshot.go Normal file
View File

@ -0,0 +1,789 @@
package sync
import (
"crypto/sha256"
"encoding/base64"
"encoding/hex"
"encoding/json"
"fmt"
"io"
"io/fs"
"os"
"path/filepath"
"sort"
"strings"
"time"
"unicode"
"unicode/utf8"
"github.com/google/uuid"
corefiles "github.com/verstak/verstak-desktop/internal/core/files"
)
const snapshotVersion = 1
const maxOperationFileBytes = corefiles.MaxBinaryReadBytes
// Snapshot is the durable local view of the synchronizable part of a vault.
// Its entries are only files and folders whose latest state was successfully
// represented by an operation or intentionally accepted as an initial baseline.
type Snapshot struct {
Version int `json:"version"`
Entries map[string]SnapshotEntry `json:"entries"`
Workspaces map[string]WorkspaceSnapshot `json:"workspaces,omitempty"`
TrashedWorkspaces map[string]WorkspaceSnapshot `json:"trashedWorkspaces,omitempty"`
WorkspacesInitialized bool `json:"workspacesInitialized,omitempty"`
Unresolved map[string]string `json:"unresolved,omitempty"`
}
// SnapshotEntry stores only stable filesystem facts needed for reconciliation.
type SnapshotEntry struct {
Path string `json:"path"`
Type string `json:"type"`
Size int64 `json:"size"`
ModifiedAt string `json:"modifiedAt"`
Hash string `json:"hash,omitempty"`
}
// WorkspaceSnapshot keeps the core-owned identity and creation metadata of a
// top-level workspace. The marker itself remains excluded from normal file
// sync and is never exposed through the Files API.
type WorkspaceSnapshot struct {
WorkspaceID string `json:"workspaceId"`
Path string `json:"path"`
Metadata json.RawMessage `json:"metadata,omitempty"`
Entries map[string]SnapshotEntry `json:"entries,omitempty"`
}
type scanJournal struct {
Snapshot Snapshot `json:"snapshot"`
Ops []Op `json:"ops"`
}
type scannedVault struct {
Entries map[string]SnapshotEntry
Workspaces map[string]WorkspaceSnapshot
Unresolved map[string]string
}
// LoadSnapshot returns the current durable scanner snapshot. A missing
// snapshot is represented by an empty snapshot, which is useful to callers
// that only need to inspect it.
func (s *Service) LoadSnapshot() (Snapshot, error) {
snapshot, exists, err := s.loadSnapshot()
if err != nil {
return Snapshot{}, err
}
if !exists {
return newSnapshot(), nil
}
return snapshot, nil
}
// ScanAndRecord scans the whole synchronizable vault and records exactly the
// detected local changes. On its first run it writes a baseline and deliberately
// produces no operations; bootstrap decides what can safely be published.
func (s *Service) ScanAndRecord() ([]string, error) {
if err := s.recoverScanJournal(); err != nil {
return nil, err
}
previous, exists, err := s.loadSnapshot()
if err != nil {
return nil, err
}
current, warnings, err := scanVault(s.vaultRoot, previous)
if err != nil {
return nil, err
}
next := snapshotFromScan(current, previous, exists)
if !exists {
if err := s.saveSnapshot(next); err != nil {
return nil, err
}
return warnings, nil
}
ops, next, err := diffSnapshots(previous, next, s.deviceID, s.vaultRoot)
if err != nil {
return nil, err
}
if len(ops) == 0 {
if err := s.saveSnapshot(next); err != nil {
return nil, err
}
return warnings, nil
}
if err := s.commitScanTransaction(next, ops); err != nil {
return nil, err
}
return warnings, nil
}
// RecordBootstrapOps records creates for a pre-pull local snapshot. It is used
// only after a successful initial pull, so an empty new vault never turns into
// remote delete operations. Callers may pass the snapshot captured before the
// pull; remote-only entries added during reconciliation are therefore not
// reflected back to the server as local creates.
func (s *Service) RecordBootstrapOps(initial Snapshot) error {
if err := s.recoverScanJournal(); err != nil {
return err
}
current, exists, err := s.loadSnapshot()
if err != nil {
return err
}
if !exists {
return fmt.Errorf("bootstrap requires an initial snapshot")
}
ops, _, err := diffSnapshots(newSnapshot(), initial, s.deviceID, s.vaultRoot)
if err != nil {
return err
}
existing, err := s.GetUnpushedOps()
if err != nil {
return err
}
existingCreates := make(map[string]bool, len(existing))
for _, op := range existing {
if op.OpType == OpCreate {
existingCreates[op.EntityType+"\x00"+op.EntityID] = true
}
}
filtered := ops[:0]
for _, op := range ops {
if op.OpType == OpCreate && existingCreates[op.EntityType+"\x00"+op.EntityID] {
continue
}
filtered = append(filtered, op)
}
ops = filtered
if len(ops) == 0 {
return nil
}
return s.commitScanTransaction(current, ops)
}
// RebaseSnapshot accepts filesystem changes that were applied from a remote
// operation without producing any outgoing operation for them.
func (s *Service) RebaseSnapshot() ([]string, error) {
if err := s.recoverScanJournal(); err != nil {
return nil, err
}
previous, exists, err := s.loadSnapshot()
if err != nil {
return nil, err
}
current, warnings, err := scanVault(s.vaultRoot, previous)
if err != nil {
return nil, err
}
next := snapshotFromScan(current, previous, exists)
acceptWorkspaceLifecycle(previous, &next)
if err := s.saveSnapshot(next); err != nil {
return nil, err
}
return warnings, nil
}
func newSnapshot() Snapshot {
return Snapshot{
Version: snapshotVersion,
Entries: make(map[string]SnapshotEntry),
Workspaces: make(map[string]WorkspaceSnapshot),
TrashedWorkspaces: make(map[string]WorkspaceSnapshot),
WorkspacesInitialized: true,
Unresolved: make(map[string]string),
}
}
func (s *Service) loadSnapshot() (Snapshot, bool, error) {
data, err := os.ReadFile(s.snapshotPath())
if err != nil {
if os.IsNotExist(err) {
return Snapshot{}, false, nil
}
return Snapshot{}, false, fmt.Errorf("read snapshot: %w", err)
}
var snapshot Snapshot
if err := json.Unmarshal(data, &snapshot); err != nil {
return Snapshot{}, false, fmt.Errorf("parse snapshot: %w", err)
}
if snapshot.Version != snapshotVersion {
return Snapshot{}, false, fmt.Errorf("unsupported snapshot version: %d", snapshot.Version)
}
if snapshot.Entries == nil {
snapshot.Entries = make(map[string]SnapshotEntry)
}
if snapshot.Workspaces == nil {
snapshot.Workspaces = make(map[string]WorkspaceSnapshot)
}
if snapshot.TrashedWorkspaces == nil {
snapshot.TrashedWorkspaces = make(map[string]WorkspaceSnapshot)
}
if snapshot.Unresolved == nil {
snapshot.Unresolved = make(map[string]string)
}
return snapshot, true, nil
}
func (s *Service) saveSnapshot(snapshot Snapshot) error {
data, err := json.MarshalIndent(snapshot, "", " ")
if err != nil {
return fmt.Errorf("marshal snapshot: %w", err)
}
return atomicWriteFile(s.snapshotPath(), data, 0o600)
}
func (s *Service) loadScanJournal() (scanJournal, bool, error) {
data, err := os.ReadFile(s.scanJournalPath())
if err != nil {
if os.IsNotExist(err) {
return scanJournal{}, false, nil
}
return scanJournal{}, false, fmt.Errorf("read scan journal: %w", err)
}
var journal scanJournal
if err := json.Unmarshal(data, &journal); err != nil {
return scanJournal{}, false, fmt.Errorf("parse scan journal: %w", err)
}
return journal, true, nil
}
func (s *Service) saveScanJournal(journal scanJournal) error {
data, err := json.MarshalIndent(journal, "", " ")
if err != nil {
return fmt.Errorf("marshal scan journal: %w", err)
}
return atomicWriteFile(s.scanJournalPath(), data, 0o600)
}
func (s *Service) recoverScanJournal() error {
journal, exists, err := s.loadScanJournal()
if err != nil || !exists {
return err
}
if err := s.recordOps(journal.Ops); err != nil {
return err
}
if err := s.saveSnapshot(journal.Snapshot); err != nil {
return err
}
if err := os.Remove(s.scanJournalPath()); err != nil && !os.IsNotExist(err) {
return err
}
return nil
}
func (s *Service) commitScanTransaction(snapshot Snapshot, ops []Op) error {
journal := scanJournal{Snapshot: snapshot, Ops: ops}
if err := s.saveScanJournal(journal); err != nil {
return err
}
if err := s.recordOps(ops); err != nil {
return err
}
if err := s.saveSnapshot(snapshot); err != nil {
return err
}
if err := os.Remove(s.scanJournalPath()); err != nil && !os.IsNotExist(err) {
return err
}
return nil
}
func scanVault(root string, previous Snapshot) (scannedVault, []string, error) {
result := scannedVault{
Entries: make(map[string]SnapshotEntry),
Workspaces: make(map[string]WorkspaceSnapshot),
Unresolved: make(map[string]string),
}
var warnings []string
err := filepath.WalkDir(root, func(path string, entry fs.DirEntry, walkErr error) error {
if walkErr != nil {
return walkErr
}
if path == root {
return nil
}
rel, err := filepath.Rel(root, path)
if err != nil {
return err
}
rel = filepath.ToSlash(rel)
if excludedFromSync(rel) {
if entry.IsDir() {
return filepath.SkipDir
}
return nil
}
if entry.Type()&os.ModeSymlink != 0 {
if entry.IsDir() {
return filepath.SkipDir
}
return nil
}
info, err := entry.Info()
if err != nil {
return err
}
if info.IsDir() {
result.Entries[rel] = SnapshotEntry{
Path: rel,
Type: EntityFolder,
ModifiedAt: info.ModTime().UTC().Format(time.RFC3339Nano),
}
return nil
}
if !info.Mode().IsRegular() {
return nil
}
if info.Size() > maxOperationFileBytes {
message := fmt.Sprintf("file-too-large: %s (%d bytes exceeds %d bytes)", rel, info.Size(), maxOperationFileBytes)
result.Unresolved[rel] = message
warnings = append(warnings, message)
return nil
}
hash, err := sha256File(path)
if err != nil {
return fmt.Errorf("hash %s: %w", rel, err)
}
result.Entries[rel] = SnapshotEntry{
Path: rel,
Type: EntityFile,
Size: info.Size(),
ModifiedAt: info.ModTime().UTC().Format(time.RFC3339Nano),
Hash: hash,
}
return nil
})
if err != nil {
return scannedVault{}, nil, err
}
workspaces, workspaceWarnings, err := scanWorkspaceSnapshots(root, previous.Workspaces)
if err != nil {
return scannedVault{}, nil, err
}
for workspaceID, workspace := range workspaces {
result.Workspaces[workspaceID] = workspace
}
for _, warning := range workspaceWarnings {
warnings = append(warnings, warning)
if strings.HasPrefix(warning, "duplicate-workspace-id: ") {
path := strings.TrimPrefix(warning, "duplicate-workspace-id: ")
result.Unresolved[path] = warning
removeEntriesUnder(result.Entries, path)
}
}
sort.Strings(warnings)
return result, warnings, nil
}
func scanWorkspaceSnapshots(root string, preferred map[string]WorkspaceSnapshot) (map[string]WorkspaceSnapshot, []string, error) {
entries, err := os.ReadDir(root)
if err != nil {
return nil, nil, err
}
candidates := make(map[string][]WorkspaceSnapshot)
var warnings []string
for _, entry := range entries {
if !entry.IsDir() || entry.Type()&os.ModeSymlink != 0 || strings.EqualFold(entry.Name(), ".verstak") {
continue
}
path := filepath.Join(root, entry.Name(), ".verstak", "workspace.json")
data, err := os.ReadFile(path)
if err != nil {
if os.IsNotExist(err) {
continue
}
return nil, nil, fmt.Errorf("read workspace marker %s: %w", entry.Name(), err)
}
var marker struct {
WorkspaceID string `json:"workspaceId"`
}
if err := json.Unmarshal(data, &marker); err != nil {
warnings = append(warnings, fmt.Sprintf("invalid-workspace-id: %s", entry.Name()))
continue
}
if _, err := uuid.Parse(marker.WorkspaceID); err != nil {
warnings = append(warnings, fmt.Sprintf("invalid-workspace-id: %s", entry.Name()))
continue
}
metadata, err := readWorkspaceMetadataSnapshot(root, entry.Name())
if err != nil {
return nil, nil, err
}
candidates[marker.WorkspaceID] = append(candidates[marker.WorkspaceID], WorkspaceSnapshot{
WorkspaceID: marker.WorkspaceID,
Path: entry.Name(),
Metadata: metadata,
})
}
workspaces := make(map[string]WorkspaceSnapshot, len(candidates))
for workspaceID, choices := range candidates {
sort.Slice(choices, func(i, j int) bool { return choices[i].Path < choices[j].Path })
selected := choices[0]
if old, ok := preferred[workspaceID]; ok {
for _, choice := range choices {
if choice.Path == old.Path {
selected = choice
break
}
}
}
workspaces[workspaceID] = selected
for _, choice := range choices {
if choice.Path != selected.Path {
warnings = append(warnings, "duplicate-workspace-id: "+choice.Path)
}
}
}
return workspaces, warnings, nil
}
func readWorkspaceMetadataSnapshot(root, name string) (json.RawMessage, error) {
path := filepath.Join(root, ".verstak", "workspaces", name, "metadata.json")
data, err := os.ReadFile(path)
if err != nil {
if os.IsNotExist(err) {
return nil, nil
}
return nil, fmt.Errorf("read workspace metadata %s: %w", name, err)
}
if !json.Valid(data) {
return nil, fmt.Errorf("invalid workspace metadata: %s", name)
}
return json.RawMessage(append([]byte(nil), data...)), nil
}
func excludedFromSync(rel string) bool {
rel = filepath.ToSlash(rel)
for _, segment := range strings.Split(rel, "/") {
if strings.EqualFold(segment, ".verstak") {
return true
}
}
base := filepath.Base(rel)
return strings.HasPrefix(base, ".verstak-write-") || strings.HasSuffix(base, ".tmp") || strings.HasSuffix(base, ".swp") || strings.HasSuffix(base, "~")
}
func sha256File(path string) (string, error) {
file, err := os.Open(path)
if err != nil {
return "", err
}
defer file.Close()
hash := sha256.New()
if _, err := io.Copy(hash, file); err != nil {
return "", err
}
return hex.EncodeToString(hash.Sum(nil)), nil
}
func snapshotFromScan(current scannedVault, previous Snapshot, previousExists bool) Snapshot {
next := newSnapshot()
for path, entry := range current.Entries {
next.Entries[path] = entry
}
for workspaceID, workspace := range current.Workspaces {
next.Workspaces[workspaceID] = workspace
}
for workspaceID, workspace := range previous.TrashedWorkspaces {
next.TrashedWorkspaces[workspaceID] = workspace
}
for path, message := range current.Unresolved {
next.Unresolved[path] = message
copyEntriesUnder(next.Entries, previous.Entries, path)
}
if !previousExists {
return next
}
for path, message := range previous.Unresolved {
if _, supported := current.Entries[path]; supported {
continue
}
if _, stillUnsupported := current.Unresolved[path]; stillUnsupported {
continue
}
next.Unresolved[path] = "unresolved sync file disappeared before it could be synchronized: " + message
copyEntriesUnder(next.Entries, previous.Entries, path)
}
return next
}
func diffSnapshots(previous, next Snapshot, deviceID, vaultRoot string) ([]Op, Snapshot, error) {
workspaceOps, err := diffWorkspaceSnapshots(&previous, &next, deviceID)
if err != nil {
return nil, next, err
}
var createsOrUpdates []Op
var deletes []Op
for path, entry := range next.Entries {
old, existed := previous.Entries[path]
if existed && entriesEqual(old, entry) {
continue
}
if unresolvedPath(next.Unresolved, path) {
continue
}
opType := OpCreate
if existed {
opType = OpUpdate
}
payload, err := payloadForEntry(vaultRoot, entry)
if err != nil {
return nil, next, err
}
createsOrUpdates = append(createsOrUpdates, newSnapshotOp(deviceID, entry.Type, path, opType, payload))
}
for path, old := range previous.Entries {
if _, exists := next.Entries[path]; exists {
continue
}
if unresolvedPath(previous.Unresolved, path) {
continue
}
deletes = append(deletes, newSnapshotOp(deviceID, old.Type, path, OpDelete, map[string]string{"path": path}))
}
sort.Slice(createsOrUpdates, func(i, j int) bool {
left, right := createsOrUpdates[i], createsOrUpdates[j]
leftDepth, rightDepth := pathDepth(left.EntityID), pathDepth(right.EntityID)
if leftDepth != rightDepth {
return leftDepth < rightDepth
}
if left.EntityType != right.EntityType {
return left.EntityType == EntityFolder
}
return left.EntityID < right.EntityID
})
sort.Slice(deletes, func(i, j int) bool {
left, right := deletes[i], deletes[j]
leftDepth, rightDepth := pathDepth(left.EntityID), pathDepth(right.EntityID)
if leftDepth != rightDepth {
return leftDepth > rightDepth
}
if left.EntityType != right.EntityType {
return left.EntityType == EntityFile
}
return left.EntityID < right.EntityID
})
return append(workspaceOps, append(createsOrUpdates, deletes...)...), next, nil
}
func diffWorkspaceSnapshots(previous, next *Snapshot, deviceID string) ([]Op, error) {
var ops []Op
if !previous.WorkspacesInitialized {
return ops, nil
}
for workspaceID, oldWorkspace := range previous.Workspaces {
currentWorkspace, active := next.Workspaces[workspaceID]
if active {
if currentWorkspace.Path != oldWorkspace.Path {
ops = append(ops, newWorkspaceSnapshotOp(deviceID, workspaceID, OpRename, currentWorkspace, oldWorkspace.Path))
remapEntriesPrefix(previous.Entries, oldWorkspace.Path, currentWorkspace.Path)
}
continue
}
ops = append(ops, newWorkspaceSnapshotOp(deviceID, workspaceID, OpTrash, oldWorkspace, ""))
oldWorkspace.Entries = entriesUnder(previous.Entries, oldWorkspace.Path)
removeEntriesUnder(previous.Entries, oldWorkspace.Path)
next.TrashedWorkspaces[workspaceID] = oldWorkspace
}
for workspaceID, currentWorkspace := range next.Workspaces {
if _, alreadyActive := previous.Workspaces[workspaceID]; alreadyActive {
continue
}
if trashedWorkspace, wasTrashed := previous.TrashedWorkspaces[workspaceID]; wasTrashed {
ops = append(ops, newWorkspaceSnapshotOp(deviceID, workspaceID, OpRestore, currentWorkspace, ""))
copyRemappedEntries(previous.Entries, trashedWorkspace.Entries, trashedWorkspace.Path, currentWorkspace.Path)
delete(next.TrashedWorkspaces, workspaceID)
continue
}
ops = append(ops, newWorkspaceSnapshotOp(deviceID, workspaceID, OpCreate, currentWorkspace, ""))
delete(next.Entries, currentWorkspace.Path)
}
sort.Slice(ops, func(i, j int) bool {
if ops[i].OpType != ops[j].OpType {
return workspaceOpOrder(ops[i].OpType) < workspaceOpOrder(ops[j].OpType)
}
return ops[i].EntityID < ops[j].EntityID
})
return ops, nil
}
func workspaceOpOrder(opType string) int {
switch opType {
case OpCreate:
return 0
case OpRename:
return 1
case OpRestore:
return 2
case OpTrash:
return 3
default:
return 4
}
}
type snapshotWorkspacePayload struct {
WorkspaceID string `json:"workspaceId"`
Path string `json:"path"`
PreviousPath string `json:"previousPath,omitempty"`
Name string `json:"name"`
Metadata json.RawMessage `json:"metadata,omitempty"`
}
func newWorkspaceSnapshotOp(deviceID, workspaceID, opType string, workspace WorkspaceSnapshot, previousPath string) Op {
payload, _ := json.Marshal(snapshotWorkspacePayload{
WorkspaceID: workspaceID,
Path: workspace.Path,
PreviousPath: previousPath,
Name: workspace.Path,
Metadata: workspace.Metadata,
})
return newSnapshotOp(deviceID, EntityWorkspace, workspaceID, opType, json.RawMessage(payload))
}
func acceptWorkspaceLifecycle(previous Snapshot, next *Snapshot) {
for workspaceID, oldWorkspace := range previous.Workspaces {
if _, stillActive := next.Workspaces[workspaceID]; !stillActive {
oldWorkspace.Entries = entriesUnder(previous.Entries, oldWorkspace.Path)
next.TrashedWorkspaces[workspaceID] = oldWorkspace
}
}
for workspaceID := range next.Workspaces {
delete(next.TrashedWorkspaces, workspaceID)
}
}
func unresolvedPath(unresolved map[string]string, path string) bool {
for unresolvedPath := range unresolved {
if path == unresolvedPath || strings.HasPrefix(path, unresolvedPath+"/") {
return true
}
}
return false
}
func copyEntriesUnder(destination, source map[string]SnapshotEntry, root string) {
for path, entry := range source {
if path == root || strings.HasPrefix(path, root+"/") {
destination[path] = entry
}
}
}
func removeEntriesUnder(entries map[string]SnapshotEntry, root string) {
for path := range entries {
if path == root || strings.HasPrefix(path, root+"/") {
delete(entries, path)
}
}
}
func remapEntriesPrefix(entries map[string]SnapshotEntry, oldPrefix, newPrefix string) {
type remappedEntry struct {
oldPath string
entry SnapshotEntry
}
var remapped []remappedEntry
for path, entry := range entries {
if path != oldPrefix && !strings.HasPrefix(path, oldPrefix+"/") {
continue
}
suffix := strings.TrimPrefix(path, oldPrefix)
entry.Path = newPrefix + suffix
remapped = append(remapped, remappedEntry{oldPath: path, entry: entry})
}
for _, item := range remapped {
delete(entries, item.oldPath)
entries[item.entry.Path] = item.entry
}
}
func entriesUnder(entries map[string]SnapshotEntry, root string) map[string]SnapshotEntry {
result := make(map[string]SnapshotEntry)
copyEntriesUnder(result, entries, root)
return result
}
func copyRemappedEntries(destination, source map[string]SnapshotEntry, oldPrefix, newPrefix string) {
for path, entry := range source {
if path != oldPrefix && !strings.HasPrefix(path, oldPrefix+"/") {
continue
}
suffix := strings.TrimPrefix(path, oldPrefix)
entry.Path = newPrefix + suffix
destination[entry.Path] = entry
}
}
func entriesEqual(left, right SnapshotEntry) bool {
return left.Type == right.Type && left.Size == right.Size && left.Hash == right.Hash
}
func payloadForEntry(vaultRoot string, entry SnapshotEntry) (map[string]string, error) {
payload := map[string]string{"path": entry.Path, "contentHash": entry.Hash}
if entry.Type == EntityFolder {
return payload, nil
}
data, err := os.ReadFile(filepath.Join(vaultRoot, filepath.FromSlash(entry.Path)))
if err != nil {
return nil, err
}
if int64(len(data)) > maxOperationFileBytes {
return nil, fmt.Errorf("file-too-large: %s", entry.Path)
}
if hash, err := sha256File(filepath.Join(vaultRoot, filepath.FromSlash(entry.Path))); err != nil {
return nil, err
} else if hash != entry.Hash {
return nil, fmt.Errorf("file changed during scan: %s", entry.Path)
}
return filePayload(entry.Path, data, entry.Hash), nil
}
func newSnapshotOp(deviceID, entityType, entityID, opType string, payload interface{}) Op {
data, _ := json.Marshal(payload)
id := uuid.NewString()
return Op{
ID: id,
OpID: id,
DeviceID: deviceID,
EntityType: entityType,
EntityID: entityID,
OpType: opType,
PayloadJSON: string(data),
CreatedAt: time.Now().UTC().Format(time.RFC3339Nano),
}
}
func pathDepth(path string) int {
if path == "" {
return 0
}
return strings.Count(path, "/") + 1
}
// filePayload encodes an already bounded file without changing its content
// representation. UTF-8 files use plain text up to the existing text limit;
// other supported files use the current bounded base64 transport.
func filePayload(path string, data []byte, hash string) map[string]string {
payload := map[string]string{"path": path, "contentHash": hash}
if int64(len(data)) <= corefiles.MaxTextFileBytes && isSyncText(data) {
payload["content"] = string(data)
return payload
}
payload["dataBase64"] = base64.StdEncoding.EncodeToString(data)
return payload
}
func isSyncText(data []byte) bool {
if !utf8.Valid(data) {
return false
}
for _, r := range string(data) {
if unicode.IsControl(r) && r != '\n' && r != '\r' && r != '\t' {
return false
}
}
return true
}

406
internal/core/sync/snapshot_test.go Normal file
View File

@ -0,0 +1,406 @@
package sync
import (
"encoding/json"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
"github.com/google/uuid"
)
func TestScanAndRecordTracksExternalWorkspaceLifecycleByIdentity(t *testing.T) {
root := t.TempDir()
workspaceID := uuid.NewString()
createSnapshotWorkspace(t, root, "Project", workspaceID)
if err := os.Mkdir(filepath.Join(root, "Project", "Files"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, "Project", "Files", "note.txt"), []byte("one"), 0o644); err != nil {
t.Fatal(err)
}
service := NewService(root, "device-a")
if _, err := service.ScanAndRecord(); err != nil {
t.Fatalf("baseline: %v", err)
}
if err := os.Rename(filepath.Join(root, "Project"), filepath.Join(root, "Renamed")); err != nil {
t.Fatal(err)
}
if _, err := service.ScanAndRecord(); err != nil {
t.Fatalf("scan rename: %v", err)
}
assertWorkspaceSnapshotOp(t, unpushedOps(t, service), 0, OpRename, workspaceID, "Renamed", "Project")
if _, err := service.ScanAndRecord(); err != nil {
t.Fatalf("unchanged renamed scan: %v", err)
}
if got := len(unpushedOps(t, service)); got != 1 {
t.Fatalf("unchanged rename produced %d operations, want 1", got)
}
trashPath := filepath.Join(root, ".verstak", "trash", "workspaces", "external-trash", "Renamed")
if err := os.MkdirAll(filepath.Dir(trashPath), 0o755); err != nil {
t.Fatal(err)
}
if err := os.Rename(filepath.Join(root, "Renamed"), trashPath); err != nil {
t.Fatal(err)
}
if _, err := service.ScanAndRecord(); err != nil {
t.Fatalf("scan trash: %v", err)
}
assertWorkspaceSnapshotOp(t, unpushedOps(t, service), 1, OpTrash, workspaceID, "Renamed", "")
if err := os.Rename(trashPath, filepath.Join(root, "Restored")); err != nil {
t.Fatal(err)
}
if _, err := service.ScanAndRecord(); err != nil {
t.Fatalf("scan restore: %v", err)
}
assertWorkspaceSnapshotOp(t, unpushedOps(t, service), 2, OpRestore, workspaceID, "Restored", "")
createSnapshotWorkspace(t, root, "Copied", workspaceID)
warnings, err := service.ScanAndRecord()
if err != nil {
t.Fatalf("scan duplicate identity: %v", err)
}
if len(warnings) != 1 || !strings.Contains(warnings[0], "duplicate-workspace-id: Copied") {
t.Fatalf("duplicate identity warnings = %v", warnings)
}
if got := len(unpushedOps(t, service)); got != 3 {
t.Fatalf("duplicate identity created operations: %d, want 3", got)
}
}
func createSnapshotWorkspace(t *testing.T, root, name, workspaceID string) {
t.Helper()
if err := os.MkdirAll(filepath.Join(root, name, ".verstak"), 0o755); err != nil {
t.Fatal(err)
}
marker, err := json.Marshal(map[string]string{"workspaceId": workspaceID})
if err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, name, ".verstak", "workspace.json"), marker, 0o600); err != nil {
t.Fatal(err)
}
metadataPath := filepath.Join(root, ".verstak", "workspaces", name, "metadata.json")
if err := os.MkdirAll(filepath.Dir(metadataPath), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(metadataPath, []byte(`{"workspaceId":"`+workspaceID+`","workspaceName":"`+name+`"}`), 0o600); err != nil {
t.Fatal(err)
}
}
func assertWorkspaceSnapshotOp(t *testing.T, ops []Op, index int, opType, workspaceID, path, previousPath string) {
t.Helper()
if len(ops) <= index {
t.Fatalf("operations = %#v, want index %d", ops, index)
}
op := ops[index]
if op.EntityType != EntityWorkspace || op.EntityID != workspaceID || op.OpType != opType {
t.Fatalf("workspace operation = %+v", op)
}
var payload snapshotWorkspacePayload
if err := json.Unmarshal([]byte(op.PayloadJSON), &payload); err != nil {
t.Fatalf("decode workspace payload: %v", err)
}
if payload.Path != path || payload.PreviousPath != previousPath || payload.WorkspaceID != workspaceID {
t.Fatalf("workspace payload = %+v", payload)
}
}
func TestScanAndRecordBaselinesThenRecordsExternalChanges(t *testing.T) {
root := t.TempDir()
service := NewService(root, "device-a")
warnings, err := service.ScanAndRecord()
if err != nil {
t.Fatalf("initial ScanAndRecord: %v", err)
}
if len(warnings) != 0 {
t.Fatalf("initial warnings = %v, want none", warnings)
}
assertUnpushedCount(t, service, 0)
if err := os.Mkdir(filepath.Join(root, "Docs"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, "Docs", "note.txt"), []byte("one"), 0o644); err != nil {
t.Fatal(err)
}
warnings, err = service.ScanAndRecord()
if err != nil {
t.Fatalf("scan create: %v", err)
}
if len(warnings) != 0 {
t.Fatalf("create warnings = %v", warnings)
}
ops := unpushedOps(t, service)
if len(ops) != 2 {
t.Fatalf("create ops = %#v, want folder and file", ops)
}
if ops[0].EntityType != EntityFolder || ops[0].EntityID != "Docs" || ops[0].OpType != OpCreate {
t.Fatalf("folder op = %+v", ops[0])
}
if ops[1].EntityType != EntityFile || ops[1].EntityID != "Docs/note.txt" || ops[1].OpType != OpCreate {
t.Fatalf("file op = %+v", ops[1])
}
var createPayload map[string]interface{}
if err := json.Unmarshal([]byte(ops[1].PayloadJSON), &createPayload); err != nil {
t.Fatalf("decode file payload: %v", err)
}
if createPayload["content"] != "one" || createPayload["contentHash"] == "" {
t.Fatalf("file payload = %#v", createPayload)
}
if _, err := service.ScanAndRecord(); err != nil {
t.Fatalf("unchanged scan: %v", err)
}
assertUnpushedCount(t, service, 2)
if err := os.WriteFile(filepath.Join(root, "Docs", "note.txt"), []byte("two"), 0o644); err != nil {
t.Fatal(err)
}
if _, err := service.ScanAndRecord(); err != nil {
t.Fatalf("scan update: %v", err)
}
ops = unpushedOps(t, service)
if len(ops) != 3 || ops[2].EntityType != EntityFile || ops[2].OpType != OpUpdate {
t.Fatalf("update ops = %#v", ops)
}
if err := os.Remove(filepath.Join(root, "Docs", "note.txt")); err != nil {
t.Fatal(err)
}
if err := os.Remove(filepath.Join(root, "Docs")); err != nil {
t.Fatal(err)
}
if _, err := service.ScanAndRecord(); err != nil {
t.Fatalf("scan delete: %v", err)
}
ops = unpushedOps(t, service)
if len(ops) != 5 {
t.Fatalf("delete ops = %#v", ops)
}
if ops[3].EntityType != EntityFile || ops[3].OpType != OpDelete || ops[4].EntityType != EntityFolder || ops[4].OpType != OpDelete {
t.Fatalf("delete ordering = %#v", ops[3:])
}
}
func TestScanAndRecordNeverTreatsInitialFilesAsDeletes(t *testing.T) {
root := t.TempDir()
if err := os.Mkdir(filepath.Join(root, "Existing"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, "Existing", "before-sync.txt"), []byte("local"), 0o644); err != nil {
t.Fatal(err)
}
service := NewService(root, "device-a")
if _, err := service.ScanAndRecord(); err != nil {
t.Fatalf("initial ScanAndRecord: %v", err)
}
assertUnpushedCount(t, service, 0)
snapshot, err := service.LoadSnapshot()
if err != nil {
t.Fatalf("LoadSnapshot: %v", err)
}
if snapshot.Entries["Existing/before-sync.txt"].Hash == "" {
t.Fatalf("snapshot = %#v, expected content hash", snapshot.Entries)
}
}
func TestScanAndRecordFindsChangesMadeWhileDesktopWasClosed(t *testing.T) {
root := t.TempDir()
initial := NewService(root, "device-a")
if _, err := initial.ScanAndRecord(); err != nil {
t.Fatalf("initial baseline: %v", err)
}
if err := os.WriteFile(filepath.Join(root, "offline.txt"), []byte("created while closed"), 0o644); err != nil {
t.Fatal(err)
}
restarted := NewService(root, "device-a")
if _, err := restarted.ScanAndRecord(); err != nil {
t.Fatalf("scan after offline create: %v", err)
}
ops := unpushedOps(t, restarted)
if len(ops) != 1 || ops[0].OpType != OpCreate || ops[0].EntityID != "offline.txt" {
t.Fatalf("offline create operations = %#v", ops)
}
if err := os.WriteFile(filepath.Join(root, "offline.txt"), []byte("updated while closed"), 0o644); err != nil {
t.Fatal(err)
}
restarted = NewService(root, "device-a")
if _, err := restarted.ScanAndRecord(); err != nil {
t.Fatalf("scan after offline update: %v", err)
}
if err := os.Remove(filepath.Join(root, "offline.txt")); err != nil {
t.Fatal(err)
}
restarted = NewService(root, "device-a")
if _, err := restarted.ScanAndRecord(); err != nil {
t.Fatalf("scan after offline delete: %v", err)
}
ops = unpushedOps(t, restarted)
if len(ops) != 3 || ops[1].OpType != OpUpdate || ops[2].OpType != OpDelete {
t.Fatalf("offline lifecycle operations = %#v", ops)
}
}
func TestRecordBootstrapOpsPublishesExistingFilesWithoutDeletes(t *testing.T) {
root := t.TempDir()
if err := os.Mkdir(filepath.Join(root, "Existing"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, "Existing", "before-sync.txt"), []byte("local"), 0o644); err != nil {
t.Fatal(err)
}
service := NewService(root, "device-a")
if _, err := service.ScanAndRecord(); err != nil {
t.Fatal(err)
}
initial, err := service.LoadSnapshot()
if err != nil {
t.Fatal(err)
}
if err := service.RecordBootstrapOps(initial); err != nil {
t.Fatalf("RecordBootstrapOps: %v", err)
}
ops := unpushedOps(t, service)
if len(ops) != 2 {
t.Fatalf("bootstrap ops = %#v, want create folder and file", ops)
}
for _, op := range ops {
if op.OpType != OpCreate {
t.Fatalf("bootstrap op = %+v, initial scan must not create delete", op)
}
}
empty := newSnapshot()
if err := service.RecordBootstrapOps(empty); err != nil {
t.Fatalf("empty bootstrap: %v", err)
}
if got := len(unpushedOps(t, service)); got != 2 {
t.Fatalf("empty bootstrap added operations = %d, want 2", got)
}
}
func TestSyncBootstrapAndWarningStateSurviveRestart(t *testing.T) {
root := t.TempDir()
service := NewService(root, "device-a")
if err := service.SetBootstrapComplete(true); err != nil {
t.Fatalf("SetBootstrapComplete: %v", err)
}
if err := service.SetLastWarning("file-too-large: archive.bin"); err != nil {
t.Fatalf("SetLastWarning: %v", err)
}
restarted := NewService(root, "")
bootstrapped, err := restarted.BootstrapComplete()
if err != nil {
t.Fatalf("BootstrapComplete: %v", err)
}
if !bootstrapped {
t.Fatal("bootstrap state was lost after restart")
}
warning, err := restarted.LastWarning()
if err != nil {
t.Fatalf("LastWarning: %v", err)
}
if warning != "file-too-large: archive.bin" {
t.Fatalf("warning = %q", warning)
}
}
func TestScanAndRecordSkipsReservedTemporaryAndSymlinkPaths(t *testing.T) {
root := t.TempDir()
if err := os.MkdirAll(filepath.Join(root, ".verstak", "sync"), 0o755); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, ".verstak", "sync", "state.json"), []byte("{}"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, ".verstak-write-local"), []byte("temporary"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, "draft.tmp"), []byte("temporary"), 0o600); err != nil {
t.Fatal(err)
}
if err := os.WriteFile(filepath.Join(root, "normal.txt"), []byte("normal"), 0o600); err != nil {
t.Fatal(err)
}
if runtime.GOOS != "windows" {
if err := os.Symlink(filepath.Join(root, "normal.txt"), filepath.Join(root, "normal-link.txt")); err != nil {
t.Skipf("symlink unavailable: %v", err)
}
}
service := NewService(root, "device-a")
if _, err := service.ScanAndRecord(); err != nil {
t.Fatalf("baseline: %v", err)
}
snapshot, err := service.LoadSnapshot()
if err != nil {
t.Fatal(err)
}
for _, path := range []string{".verstak/sync/state.json", ".verstak-write-local", "draft.tmp", "normal-link.txt"} {
if _, ok := snapshot.Entries[path]; ok {
t.Fatalf("reserved path %q was included in snapshot %#v", path, snapshot.Entries)
}
}
if _, ok := snapshot.Entries["normal.txt"]; !ok {
t.Fatalf("normal file missing from snapshot %#v", snapshot.Entries)
}
}
func TestScanAndRecordKeepsUnsupportedFileUnresolved(t *testing.T) {
root := t.TempDir()
path := filepath.Join(root, "too-large.bin")
if err := os.WriteFile(path, make([]byte, maxOperationFileBytes+1), 0o600); err != nil {
t.Fatal(err)
}
service := NewService(root, "device-a")
warnings, err := service.ScanAndRecord()
if err != nil {
t.Fatalf("scan unsupported file: %v", err)
}
if len(warnings) != 1 || !strings.Contains(warnings[0], "too-large.bin") || !strings.Contains(warnings[0], "file-too-large") {
t.Fatalf("warnings = %v", warnings)
}
assertUnpushedCount(t, service, 0)
snapshot, err := service.LoadSnapshot()
if err != nil {
t.Fatal(err)
}
if _, ok := snapshot.Entries["too-large.bin"]; ok {
t.Fatalf("unsupported file was marked synchronized: %#v", snapshot.Entries)
}
warnings, err = service.ScanAndRecord()
if err != nil || len(warnings) != 1 {
t.Fatalf("second scan warnings=%v err=%v, unresolved file must remain visible", warnings, err)
}
}
func unpushedOps(t *testing.T, service *Service) []Op {
t.Helper()
ops, err := service.GetUnpushedOps()
if err != nil {
t.Fatalf("GetUnpushedOps: %v", err)
}
return ops
}
func assertUnpushedCount(t *testing.T, service *Service, want int) {
t.Helper()
if got := len(unpushedOps(t, service)); got != want {
t.Fatalf("unpushed ops = %d, want %d", got, want)
}
}

245
internal/core/workspace/manager.go
View File

@ -470,25 +470,35 @@ func (m *Manager) CreateWorkspace(name, templateID string) (Workspace, error) {
func writeWorkspaceIdentity(workspacePath string) (string, error) {
workspaceID := uuid.NewString()
data, err := json.Marshal(workspaceIdentityMarker{WorkspaceID: workspaceID})
if err != nil {
return "", err
}
markerPath := filepath.Join(workspacePath, filepath.FromSlash(workspaceIdentityRelativePath))
if err := os.MkdirAll(filepath.Dir(markerPath), 0o755); err != nil {
return "", err
}
tmpPath := markerPath + ".tmp"
if err := os.WriteFile(tmpPath, data, 0o644); err != nil {
return "", err
}
if err := os.Rename(tmpPath, markerPath); err != nil {
_ = os.Remove(tmpPath)
if err := writeWorkspaceIdentityValue(workspacePath, workspaceID); err != nil {
return "", err
}
return workspaceID, nil
}
func writeWorkspaceIdentityValue(workspacePath, workspaceID string) error {
if _, err := uuid.Parse(workspaceID); err != nil {
return fmt.Errorf("invalid workspace identity: %w", err)
}
data, err := json.Marshal(workspaceIdentityMarker{WorkspaceID: workspaceID})
if err != nil {
return err
}
markerPath := filepath.Join(workspacePath, filepath.FromSlash(workspaceIdentityRelativePath))
if err := os.MkdirAll(filepath.Dir(markerPath), 0o755); err != nil {
return err
}
tmpPath := markerPath + ".tmp"
if err := os.WriteFile(tmpPath, data, 0o644); err != nil {
return err
}
if err := os.Rename(tmpPath, markerPath); err != nil {
_ = os.Remove(tmpPath)
return err
}
return nil
}
func ensureWorkspaceIdentity(workspacePath string) (string, error) {
workspaceID, err := readWorkspaceIdentity(workspacePath)
if os.IsNotExist(err) {
@ -739,6 +749,213 @@ func (m *Manager) RestoreWorkspaceTrash(trashID, targetName string) (Workspace,
return Workspace{ID: workspaceID, Name: targetName, RootPath: targetName}, nil
}
// CreateWorkspaceFromSync creates a workspace from the core-only sync contract.
// It deliberately does not apply a live template: child folders and files are
// represented by normal file operations, while the captured metadata preserves
// the historical template snapshot that matters for restoration.
func (m *Manager) CreateWorkspaceFromSync(name, workspaceID string, meta Metadata) (Workspace, error) {
name = strings.TrimSpace(name)
if err := validateWorkspaceName(name); err != nil {
return Workspace{}, err
}
if _, err := uuid.Parse(workspaceID); err != nil {
return Workspace{}, fmt.Errorf("invalid workspace identity: %w", err)
}
if existing, found, err := m.findActiveWorkspaceByID(workspaceID); err != nil {
return Workspace{}, err
} else if found {
if existing.Name == name {
return existing, nil
}
return Workspace{}, fmt.Errorf("conflict: workspace identity %s already belongs to %s", workspaceID, existing.Name)
}
full := filepath.Join(m.vaultDir, name)
if _, err := os.Lstat(full); err == nil {
return Workspace{}, fmt.Errorf("conflict: %s", name)
} else if !os.IsNotExist(err) {
return Workspace{}, err
}
if err := os.Mkdir(full, 0o755); err != nil {
return Workspace{}, err
}
created := true
defer func() {
if created {
_ = os.RemoveAll(full)
}
}()
if err := writeWorkspaceIdentityValue(full, workspaceID); err != nil {
return Workspace{}, err
}
meta.WorkspaceID = workspaceID
meta.WorkspaceName = name
if meta.Features == nil {
meta.Features = map[string]bool{"files": true}
}
if meta.Folders == nil {
meta.Folders = defaultFolders()
}
if meta.UpdatedAt == "" {
meta.UpdatedAt = time.Now().UTC().Format(time.RFC3339Nano)
}
for _, folder := range meta.Folders {
if strings.TrimSpace(folder) == "" {
continue
}
if err := validateWorkspaceFolderPath(folder); err != nil {
return Workspace{}, err
}
if err := os.MkdirAll(filepath.Join(full, folder), 0o755); err != nil {
return Workspace{}, err
}
}
if err := m.writeMetadata(name, meta); err != nil {
return Workspace{}, err
}
created = false
return Workspace{ID: workspaceID, Name: name, RootPath: name}, nil
}
func validateWorkspaceFolderPath(folder string) error {
folder = strings.TrimSpace(folder)
if folder == "" || filepath.IsAbs(folder) || strings.Contains(folder, `\`) {
return fmt.Errorf("invalid workspace folder path")
}
cleaned := filepath.ToSlash(filepath.Clean(filepath.FromSlash(folder)))
if cleaned == "." || cleaned == ".." || strings.HasPrefix(cleaned, "../") {
return fmt.Errorf("invalid workspace folder path")
}
for _, segment := range strings.Split(cleaned, "/") {
if strings.EqualFold(segment, ".verstak") {
return fmt.Errorf("invalid workspace folder path")
}
}
return nil
}
// RenameWorkspaceFromSync applies an idempotent rename only to the durable
// identity named by the operation. A different local path is a visible conflict.
func (m *Manager) RenameWorkspaceFromSync(workspaceID, oldName, newName string) error {
newName = strings.TrimSpace(newName)
if err := validateWorkspaceName(newName); err != nil {
return err
}
workspace, found, err := m.findActiveWorkspaceByID(workspaceID)
if err != nil {
return err
}
if !found {
return fmt.Errorf("not-found: workspace identity %s", workspaceID)
}
if workspace.Name == newName {
return nil
}
if strings.TrimSpace(oldName) != "" && workspace.Name != oldName {
return fmt.Errorf("conflict: workspace identity %s is at %s, expected %s", workspaceID, workspace.Name, oldName)
}
return m.RenameWorkspace(workspace.Name, newName)
}
// TrashWorkspaceFromSync moves the identified active workspace into this
// device's local trash. Local trash IDs are intentionally not synchronized.
func (m *Manager) TrashWorkspaceFromSync(workspaceID, name string) (TrashResult, error) {
workspace, found, err := m.findActiveWorkspaceByID(workspaceID)
if err != nil {
return TrashResult{}, err
}
if !found {
if _, trashed, err := m.findTrashedWorkspaceByID(workspaceID); err != nil {
return TrashResult{}, err
} else if trashed {
return TrashResult{WorkspaceID: workspaceID}, nil
}
return TrashResult{}, fmt.Errorf("not-found: workspace identity %s", workspaceID)
}
if strings.TrimSpace(name) != "" && workspace.Name != name {
return TrashResult{}, fmt.Errorf("conflict: workspace identity %s is at %s, expected %s", workspaceID, workspace.Name, name)
}
return m.TrashWorkspace(workspace.Name)
}
// RestoreWorkspaceFromSync restores a workspace by durable identity. It is
// idempotent at the requested target and rejects a workspace of another ID.
func (m *Manager) RestoreWorkspaceFromSync(workspaceID, targetName string) (Workspace, error) {
targetName = strings.TrimSpace(targetName)
if err := validateWorkspaceName(targetName); err != nil {
return Workspace{}, err
}
if active, found, err := m.findActiveWorkspaceByID(workspaceID); err != nil {
return Workspace{}, err
} else if found {
if active.Name == targetName {
return active, nil
}
return Workspace{}, fmt.Errorf("conflict: workspace identity %s is already active at %s", workspaceID, active.Name)
}
trashID, found, err := m.findTrashedWorkspaceByID(workspaceID)
if err != nil {
return Workspace{}, err
}
if !found {
return Workspace{}, fmt.Errorf("not-found: trashed workspace identity %s", workspaceID)
}
return m.RestoreWorkspaceTrash(trashID, targetName)
}
func (m *Manager) findActiveWorkspaceByID(workspaceID string) (Workspace, bool, error) {
if _, err := uuid.Parse(workspaceID); err != nil {
return Workspace{}, false, fmt.Errorf("invalid workspace identity: %w", err)
}
workspaces, err := m.ListWorkspaces()
if err != nil {
return Workspace{}, false, err
}
var match Workspace
for _, workspace := range workspaces {
if workspace.ID != workspaceID {
continue
}
if match.ID != "" {
return Workspace{}, false, fmt.Errorf("conflict: duplicated workspace identity %s", workspaceID)
}
match = workspace
}
return match, match.ID != "", nil
}
func (m *Manager) findTrashedWorkspaceByID(workspaceID string) (string, bool, error) {
if _, err := uuid.Parse(workspaceID); err != nil {
return "", false, fmt.Errorf("invalid workspace identity: %w", err)
}
trashRoot := filepath.Join(m.vaultDir, ".verstak", "trash", "workspaces")
entries, err := os.ReadDir(trashRoot)
if err != nil {
if os.IsNotExist(err) {
return "", false, nil
}
return "", false, err
}
var match string
for _, entry := range entries {
if !entry.IsDir() {
continue
}
identity, err := m.GetWorkspaceTrashIdentity(entry.Name())
if err != nil {
continue
}
if identity.WorkspaceID != workspaceID {
continue
}
if match != "" {
return "", false, fmt.Errorf("conflict: duplicated trashed workspace identity %s", workspaceID)
}
match = entry.Name()
}
return match, match != "", nil
}
func validateWorkspaceTrashID(trashID string) error {
if trashID == "" || strings.ContainsAny(trashID, `/\\`) || filepath.Clean(trashID) != trashID {
return fmt.Errorf("invalid workspace trash ID")

89
internal/core/workspace/manager_test.go
View File

@ -7,6 +7,8 @@ import (
"runtime"
"strings"
"testing"
"github.com/google/uuid"
)
func TestListWorkspacesReadsTopLevelPhysicalFolders(t *testing.T) {
@ -435,6 +437,93 @@ func TestRestoreWorkspaceTrashPreservesIdentity(t *testing.T) {
}
}
func TestApplySyncedWorkspaceLifecycleKeepsIdentityAndRejectsNameConflict(t *testing.T) {
vaultDir := newVaultDir(t)
m := NewManager(vaultDir)
workspaceID := "5f0f96d9-61c8-4b6b-8c3a-a1b9a0f40001"
meta := Metadata{
WorkspaceID: workspaceID,
WorkspaceName: "Remote",
CreatedFromTemplate: &TemplateSnapshot{
TemplateID: "minimal",
TemplateName: "Minimal",
TemplateVersion: 1,
AppliedAt: "2026-07-17T00:00:00Z",
},
Features: map[string]bool{"files": true},
Folders: map[string]string{"notes": "Notes"},
}
created, err := m.CreateWorkspaceFromSync("Remote", workspaceID, meta)
if err != nil {
t.Fatalf("CreateWorkspaceFromSync: %v", err)
}
if created.ID != workspaceID || created.Name != "Remote" {
t.Fatalf("created = %+v", created)
}
if _, err := m.CreateWorkspaceFromSync("Remote", workspaceID, meta); err != nil {
t.Fatalf("replayed create: %v", err)
}
if err := m.RenameWorkspaceFromSync(workspaceID, "Remote", "Remote-Renamed"); err != nil {
t.Fatalf("RenameWorkspaceFromSync: %v", err)
}
if err := m.RenameWorkspaceFromSync(workspaceID, "Remote", "Remote-Renamed"); err != nil {
t.Fatalf("replayed rename: %v", err)
}
trashed, err := m.TrashWorkspaceFromSync(workspaceID, "Remote-Renamed")
if err != nil {
t.Fatalf("TrashWorkspaceFromSync: %v", err)
}
if trashed.WorkspaceID != workspaceID {
t.Fatalf("trashed = %+v", trashed)
}
if _, err := m.TrashWorkspaceFromSync(workspaceID, "Remote-Renamed"); err != nil {
t.Fatalf("replayed trash: %v", err)
}
restored, err := m.RestoreWorkspaceFromSync(workspaceID, "Remote-Restored")
if err != nil {
t.Fatalf("RestoreWorkspaceFromSync: %v", err)
}
if restored.ID != workspaceID || restored.Name != "Remote-Restored" {
t.Fatalf("restored = %+v", restored)
}
if _, err := m.RestoreWorkspaceFromSync(workspaceID, "Remote-Restored"); err != nil {
t.Fatalf("replayed restore: %v", err)
}
if _, err := m.CreateWorkspace("Taken", "minimal"); err != nil {
t.Fatalf("CreateWorkspace Taken: %v", err)
}
if err := m.RenameWorkspaceFromSync(workspaceID, "Remote-Restored", "Taken"); err == nil || !strings.Contains(err.Error(), "conflict") {
t.Fatalf("rename conflict error = %v, want conflict", err)
}
if _, err := m.CreateWorkspaceFromSync("Copied", workspaceID, meta); err == nil || !strings.Contains(err.Error(), "conflict") {
t.Fatalf("duplicate workspace ID error = %v, want conflict", err)
}
stored, err := m.GetWorkspaceMetadata("Remote-Restored")
if err != nil {
t.Fatalf("GetWorkspaceMetadata: %v", err)
}
if stored.WorkspaceID != workspaceID || stored.CreatedFromTemplate == nil || stored.CreatedFromTemplate.TemplateID != "minimal" {
t.Fatalf("stored metadata = %+v", stored)
}
}
func TestCreateWorkspaceFromSyncRejectsUnsafeMetadataFolder(t *testing.T) {
vaultDir := newVaultDir(t)
m := NewManager(vaultDir)
if _, err := m.CreateWorkspaceFromSync("Remote", uuid.NewString(), Metadata{
Folders: map[string]string{"files": "../escaped"},
}); err == nil || !strings.Contains(err.Error(), "invalid workspace folder") {
t.Fatalf("unsafe synced metadata error = %v", err)
}
if _, err := os.Stat(filepath.Join(vaultDir, "escaped")); !os.IsNotExist(err) {
t.Fatalf("unsafe synced metadata created path outside workspace: %v", err)
}
}
func TestPurgeWorkspaceTrashRemovesPayload(t *testing.T) {
vaultDir := newVaultDir(t)
m := NewManager(vaultDir)

15
scripts/smoke-real-sync.sh
View File

@ -30,7 +30,10 @@ fi
) >"$LOG_FILE" 2>&1 &
SERVER_PID="$!"
for _ in $(seq 1 80); do
# A cold Go cache can take longer than 20 seconds to compile `go run`; wait
# for the real listener rather than treating a still-running compiler as a
# healthy server.
for _ in $(seq 1 300); do
if curl -fsS "http://127.0.0.1:$PORT/api/v1/health" >/dev/null 2>&1; then
break
fi
@ -45,10 +48,12 @@ curl -fsS "http://127.0.0.1:$PORT/api/v1/health" >/dev/null
NOW="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
sqlite3 "$DATA_DIR/server.db" "
INSERT INTO server_devices (id, name, api_key, last_seen, created_at)
VALUES ('smoke-device-a', 'Smoke Device A', 'smoke-key-a', '$NOW', '$NOW');
INSERT INTO server_devices (id, name, api_key, last_seen, created_at)
VALUES ('smoke-device-b', 'Smoke Device B', 'smoke-key-b', '$NOW', '$NOW');
INSERT INTO server_users (id, username, email, password_hash, confirmed, created_at)
VALUES ('smoke-user', 'smoke-user', 'smoke@example.test', 'unused', 1, '$NOW');
INSERT INTO server_devices (id, name, api_key, user_id, vault_id, last_seen, created_at)
VALUES ('smoke-device-a', 'Smoke Device A', 'smoke-key-a', 'smoke-user', 'smoke-vault', '$NOW', '$NOW');
INSERT INTO server_devices (id, name, api_key, user_id, vault_id, last_seen, created_at)
VALUES ('smoke-device-b', 'Smoke Device B', 'smoke-key-b', 'smoke-user', 'smoke-vault', '$NOW', '$NOW');
"
(