docs: design persistent search runtime hosting
This commit is contained in:
parent
bcedbc6143
commit
f241dffd0b
|
|
@ -0,0 +1,319 @@
|
|||
# Persistent Search Index And Runtime Provider Hosting Design
|
||||
|
||||
## Context
|
||||
|
||||
Phase 4 requires `verstak.search` to move beyond live recursive scans:
|
||||
|
||||
- keep search as an official plugin, not a core feature;
|
||||
- persist a workspace-scoped search index;
|
||||
- host `searchProviders` contributed by other plugins at runtime;
|
||||
- preserve the local-first, readable vault model;
|
||||
- avoid copying code or architecture from the old Verstak repository.
|
||||
|
||||
Current implementation status:
|
||||
|
||||
- `verstak.search` is a workspace item and contributes `searchProviders`;
|
||||
- it searches while typing by walking files through `api.files.list`;
|
||||
- it reads text-like files through `api.files.readText`;
|
||||
- it opens file results through `api.workbench.openResource`;
|
||||
- the contribution registry already exposes `searchProviders`;
|
||||
- `api.commands.executeFor(pluginId, commandId, args)` already hosts frontend
|
||||
provider handlers for other contribution types;
|
||||
- desktop backend already has plugin-scoped JSON data methods
|
||||
`ReadPluginDataJSON` and `WritePluginDataJSON`, but the frontend plugin API
|
||||
currently exposes only `settings`.
|
||||
|
||||
No visual companion is needed for this design because the decision is about
|
||||
runtime contracts and data flow, not layout.
|
||||
|
||||
## Assumptions
|
||||
|
||||
- The next implementation should be a small reversible platform step.
|
||||
- Search remains replaceable plugin functionality; core should expose generic
|
||||
storage and contribution execution only.
|
||||
- `searchProviders[].handler` is treated as a command id. A provider plugin
|
||||
must declare that command in `contributes.commands` and register its frontend
|
||||
handler with `api.commands.register`.
|
||||
- The initial persistent index is JSON-backed plugin data, not SQLite FTS and
|
||||
not a background sidecar.
|
||||
- The index is an optimization and discovery layer. User files remain the
|
||||
source of truth.
|
||||
|
||||
## Alternatives Considered
|
||||
|
||||
### Recommended: plugin-owned index plus command-backed provider hosting
|
||||
|
||||
`verstak.search` owns its local index in plugin storage, uses public Files API
|
||||
and events to keep it fresh, and fans out to contributed providers with
|
||||
`api.commands.executeFor`.
|
||||
|
||||
Trade-offs:
|
||||
|
||||
- smallest core/runtime change;
|
||||
- matches existing Files/Notes contribution execution pattern;
|
||||
- keeps user-facing search outside core;
|
||||
- JSON index is simpler than full text search but sufficient for the current
|
||||
roadmap item.
|
||||
|
||||
### Core search service
|
||||
|
||||
Desktop core would own indexing and execute provider searches.
|
||||
|
||||
Trade-offs:
|
||||
|
||||
- easier to centralize later ranking and indexing;
|
||||
- violates the current direction that search is a plugin-level user feature;
|
||||
- makes core understand official search semantics too early.
|
||||
|
||||
### Sidecar or SQLite FTS indexer now
|
||||
|
||||
Introduce a dedicated indexer process or SQLite FTS schema immediately.
|
||||
|
||||
Trade-offs:
|
||||
|
||||
- better long-term scalability;
|
||||
- too large for the current milestone;
|
||||
- adds migration, lifecycle, and sync/cache policy decisions before the basic
|
||||
provider runtime contract is proven.
|
||||
|
||||
## Chosen Design
|
||||
|
||||
Use the recommended approach.
|
||||
|
||||
`verstak.search` becomes both:
|
||||
|
||||
- the workspace search UI;
|
||||
- the runtime host for all enabled `searchProviders`.
|
||||
|
||||
The desktop and SDK expose a generic frontend storage surface:
|
||||
|
||||
```ts
|
||||
api.storage.data.read(name: string): Promise<Record<string, unknown>>
|
||||
api.storage.data.write(name: string, data: Record<string, unknown>): Promise<void>
|
||||
```
|
||||
|
||||
This maps to existing plugin-scoped backend methods. It requires
|
||||
`storage.namespace`, follows current plugin ownership rules, and does not let a
|
||||
plugin read another plugin's namespace.
|
||||
|
||||
The Search plugin stores its persistent index as plugin data named
|
||||
`search-index`. Settings remain for user preferences only.
|
||||
|
||||
## Provider Runtime Contract
|
||||
|
||||
`searchProviders` keep their current manifest shape:
|
||||
|
||||
```json
|
||||
{
|
||||
"id": "verstak.search.vault-text",
|
||||
"label": "Vault Text Search",
|
||||
"handler": "verstak.search.searchVaultText"
|
||||
}
|
||||
```
|
||||
|
||||
The `handler` value must name a command declared by the same plugin:
|
||||
|
||||
```json
|
||||
{
|
||||
"contributes": {
|
||||
"commands": [
|
||||
{
|
||||
"id": "verstak.search.searchVaultText",
|
||||
"title": "Search Vault Text",
|
||||
"handler": "searchVaultText"
|
||||
}
|
||||
],
|
||||
"searchProviders": [
|
||||
{
|
||||
"id": "verstak.search.vault-text",
|
||||
"label": "Vault Text Search",
|
||||
"handler": "verstak.search.searchVaultText"
|
||||
}
|
||||
]
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The provider registers the command handler at mount time:
|
||||
|
||||
```js
|
||||
api.commands.register('verstak.search.searchVaultText', searchVaultText)
|
||||
```
|
||||
|
||||
Runtime availability:
|
||||
|
||||
- this milestone does not auto-start unloaded frontend bundles or sidecars;
|
||||
- a provider is executable only after its command handler is registered in the
|
||||
current frontend runtime;
|
||||
- declared but unregistered providers are skipped and reported as unavailable;
|
||||
- later sidecar/background activation can extend this without changing the
|
||||
provider manifest shape.
|
||||
|
||||
Provider input:
|
||||
|
||||
```ts
|
||||
{
|
||||
source: 'search',
|
||||
providerId: string,
|
||||
query: string,
|
||||
workspaceRootPath: string,
|
||||
limit: number
|
||||
}
|
||||
```
|
||||
|
||||
Provider output:
|
||||
|
||||
```ts
|
||||
{
|
||||
results: SearchResult[]
|
||||
}
|
||||
```
|
||||
|
||||
`SearchResult`:
|
||||
|
||||
```ts
|
||||
{
|
||||
id?: string,
|
||||
path?: string,
|
||||
title?: string,
|
||||
snippet?: string,
|
||||
matchType?: string,
|
||||
providerId?: string,
|
||||
providerLabel?: string,
|
||||
type?: 'file' | 'folder' | 'activity' | 'worklog' | string,
|
||||
openable?: boolean,
|
||||
line?: number,
|
||||
score?: number,
|
||||
resource?: {
|
||||
kind: 'vault-file',
|
||||
path: string,
|
||||
mode?: 'view' | 'edit'
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
The Search host normalizes missing optional fields. Invalid provider responses
|
||||
are ignored with a visible status warning; they do not fail the whole search.
|
||||
|
||||
## Persistent Index Shape
|
||||
|
||||
The first index version is intentionally small:
|
||||
|
||||
```json
|
||||
{
|
||||
"version": 1,
|
||||
"workspaceRootPath": "Project",
|
||||
"builtAt": "2026-06-29T00:00:00Z",
|
||||
"entries": [
|
||||
{
|
||||
"path": "Project/Docs/case.md",
|
||||
"name": "case.md",
|
||||
"type": "file",
|
||||
"extension": "md",
|
||||
"size": 1234,
|
||||
"modifiedAt": "2026-06-29T00:00:00Z",
|
||||
"text": "short normalized searchable text or snippet"
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Rules:
|
||||
|
||||
- index only the current workspace root;
|
||||
- store vault-relative slash paths;
|
||||
- index folders and regular files;
|
||||
- read content only for text-like files already handled by current Search;
|
||||
- rely on the host `readText` limit for large text files;
|
||||
- store short normalized text, not full arbitrary binary data;
|
||||
- rebuild when version or workspace root differs.
|
||||
|
||||
## Index Lifecycle
|
||||
|
||||
On mount:
|
||||
|
||||
1. Read `api.storage.data.read('search-index')`.
|
||||
2. If version/root matches, use it immediately.
|
||||
3. If missing or stale, build an index from `api.files.list` and text reads.
|
||||
4. Write the built index with `api.storage.data.write`.
|
||||
5. Subscribe to `file.changed`.
|
||||
|
||||
On `file.changed`:
|
||||
|
||||
- if event path is outside the workspace, ignore it;
|
||||
- for create/update, refresh that path's metadata/content if readable;
|
||||
- for delete/trash, remove the path and descendants;
|
||||
- for move, remove `fromPath` when present and refresh the new path;
|
||||
- write the updated index after the change is applied.
|
||||
|
||||
If an incremental update fails, mark the index stale in UI and continue serving
|
||||
the last usable index until rebuild succeeds.
|
||||
|
||||
## Search Flow
|
||||
|
||||
For a query shorter than two characters, return no results.
|
||||
|
||||
For a valid query:
|
||||
|
||||
1. Query local persistent index for path, name, folder, and text matches.
|
||||
2. List enabled `searchProviders` through `api.contributions.list`.
|
||||
3. Call each provider except duplicates that would recurse into the same
|
||||
in-flight command.
|
||||
4. Merge and normalize results.
|
||||
5. Sort by score, then provider order, then path/title.
|
||||
6. Render provider label, match type, path/title, snippet, and open action when
|
||||
the result has a supported resource.
|
||||
|
||||
Provider failures are isolated. The status line reports how many providers
|
||||
failed without hiding successful results.
|
||||
|
||||
## Error Handling
|
||||
|
||||
- Missing `storage` API: fall back to live scanning and show a degraded status.
|
||||
- Missing `storage.namespace` permission: Search runs without persistence and
|
||||
reports degraded persistence.
|
||||
- Corrupt index JSON: ignore it, rebuild, and overwrite only after a successful
|
||||
rebuild.
|
||||
- Provider command not declared or not registered: skip that provider and show a
|
||||
warning count.
|
||||
- File read errors: skip that file, matching the current Search behavior.
|
||||
- Event subscription errors: keep search usable, mark index refresh as manual.
|
||||
|
||||
## Testing And Verification
|
||||
|
||||
Expected test coverage:
|
||||
|
||||
- SDK types include `api.storage.data.read/write` and search result/provider
|
||||
contracts.
|
||||
- SDK mock API stores plugin data separately from settings.
|
||||
- Desktop frontend bridge exposes `api.storage.data.read/write`.
|
||||
- Desktop bridge smoke test verifies plugin data round-trip through Wails mock.
|
||||
- Search plugin smoke test verifies:
|
||||
- persisted index is read before scanning;
|
||||
- missing/stale index triggers build and write;
|
||||
- query uses persisted entries;
|
||||
- `file.changed` updates or removes entries;
|
||||
- provider fan-out uses `contributions.list('searchProviders')` and
|
||||
`commands.executeFor`;
|
||||
- provider failure does not hide local results.
|
||||
- Roadmap and official plugin docs are updated after implementation.
|
||||
|
||||
Manual smoke after implementation:
|
||||
|
||||
- start desktop frontend/dev flow used by the repository;
|
||||
- open Search workspace item;
|
||||
- search a known text file;
|
||||
- reload/remount Search and confirm results come from stored index;
|
||||
- enable a test provider and confirm its results appear with provider label.
|
||||
|
||||
## Out Of Scope
|
||||
|
||||
- SQLite FTS;
|
||||
- typo/layout tolerant search;
|
||||
- binary OCR or PDF extraction;
|
||||
- background sidecar indexing;
|
||||
- cross-workspace global search;
|
||||
- sync policy for index cache;
|
||||
- journal/worklog/activity reconstruction implementation.
|
||||
|
||||
Those remain later roadmap items.
|
||||
Loading…
Reference in New Issue