docs: plan persistent search runtime hosting
This commit is contained in:
parent
f241dffd0b
commit
16eb932be2
|
|
@ -0,0 +1,963 @@
|
|||
# Persistent Search Runtime Hosting Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Build a persistent workspace search index for `verstak.search` and host runtime `searchProviders` through the existing plugin command bridge.
|
||||
|
||||
**Architecture:** Keep search as an official plugin. Add a generic plugin data storage surface to the SDK and desktop frontend bridge, then make `verstak.search` persist its own JSON index and fan out to registered provider command handlers with `api.commands.executeFor`.
|
||||
|
||||
**Tech Stack:** TypeScript SDK types/tests with Vitest, Wails desktop frontend bridge JavaScript smoke tests, official plugin browserless Node smoke tests, Markdown docs.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Do not copy code or architecture from `~/git/verstak`; it is feature reference only.
|
||||
- Search remains plugin functionality; do not move search semantics into desktop core.
|
||||
- User files remain the source of truth; the index is an optimization and discovery layer.
|
||||
- Use JSON plugin data named `search-index`; do not use SQLite FTS or a sidecar indexer.
|
||||
- `searchProviders[].handler` is a command id declared by the same plugin.
|
||||
- A provider is executable only when its command handler is registered in the current frontend runtime.
|
||||
- Missing storage, corrupt index data, provider failures, and unreadable files must degrade Search without failing the whole view.
|
||||
- Use TDD for production code changes: write the failing test, run it red, implement the minimal code, run it green.
|
||||
- After each meaningful change, commit and push the affected repository.
|
||||
|
||||
---
|
||||
|
||||
## File Structure
|
||||
|
||||
### `verstak-sdk`
|
||||
|
||||
- Modify `src/plugin-api.ts`
|
||||
- Add `PluginDataJSON` type.
|
||||
- Add `storage.data.read/write` to `VerstakPluginAPI`.
|
||||
- Modify `src/test-utils.ts`
|
||||
- Store mock plugin data separately from mock settings.
|
||||
- Expose `api.storage.data.read/write`.
|
||||
- Modify `src/plugin-api.test.ts`
|
||||
- Add a focused mock API test for plugin data storage isolation.
|
||||
- Generated by build: `dist/*`.
|
||||
|
||||
### `verstak-desktop`
|
||||
|
||||
- Modify `frontend/src/lib/plugin-host/VerstakPluginAPI.js`
|
||||
- Expose `api.storage.data.read/write`.
|
||||
- Route to existing Wails methods `ReadPluginDataJSON` and `WritePluginDataJSON`.
|
||||
- Modify `frontend/tests/plugin-api-contributions-test.mjs`
|
||||
- Extend the smoke mock and assertions to cover plugin data round-trip.
|
||||
- Modify `frontend/src/lib/test/wails-mock.js`
|
||||
- Add in-memory plugin data behavior for browser/e2e mocks.
|
||||
|
||||
### `verstak-official-plugins`
|
||||
|
||||
- Modify `plugins/search/plugin.json`
|
||||
- Add `storage.namespace`, `events.subscribe`, and `commands.register`.
|
||||
- Add a command contribution whose id is used by `searchProviders[].handler`.
|
||||
- Modify `plugins/search/frontend/src/index.js`
|
||||
- Add small index helpers.
|
||||
- Register the local vault-text search provider command.
|
||||
- Load/build/write `search-index`.
|
||||
- Subscribe to `file.changed`.
|
||||
- Fan out to other `searchProviders`.
|
||||
- Modify `scripts/smoke-search-plugin.js`
|
||||
- Cover persisted index reads/writes, provider fan-out, provider failures, and file event updates.
|
||||
|
||||
### `verstak-docs`
|
||||
|
||||
- Modify `05_Official_Plugins.md`
|
||||
- Replace the future-work sentence with current behavior after implementation.
|
||||
- Modify `07_Full_Implementation_Roadmap.md`
|
||||
- Mark persistent search index and runtime hosting complete after code is verified.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: SDK Plugin Data Storage Contract
|
||||
|
||||
**Files:**
|
||||
- Modify: `/home/mirivlad/git/verstak2/verstak-sdk/src/plugin-api.ts`
|
||||
- Modify: `/home/mirivlad/git/verstak2/verstak-sdk/src/test-utils.ts`
|
||||
- Modify: `/home/mirivlad/git/verstak2/verstak-sdk/src/plugin-api.test.ts`
|
||||
- Generated by build: `/home/mirivlad/git/verstak2/verstak-sdk/dist/*`
|
||||
|
||||
**Interfaces:**
|
||||
- Produces:
|
||||
- `export type PluginDataJSON = Record<string, unknown>;`
|
||||
- `VerstakPluginAPI.storage.data.read(name: string): Promise<PluginDataJSON>`
|
||||
- `VerstakPluginAPI.storage.data.write(name: string, data: PluginDataJSON): Promise<void>`
|
||||
- Consumes:
|
||||
- Existing `VerstakPluginAPI` and `createMockPluginAPI`.
|
||||
|
||||
- [ ] **Step 1: Write the failing SDK test**
|
||||
|
||||
Add this test after `settings persist in the mock API namespace` in `src/plugin-api.test.ts`:
|
||||
|
||||
```ts
|
||||
test('plugin data persists separately from settings in the mock API namespace', async () => {
|
||||
const api = createMockPluginAPI('storage.plugin');
|
||||
|
||||
await api.settings.write('search-index', { source: 'settings' });
|
||||
await api.storage.data.write('search-index', {
|
||||
version: 1,
|
||||
workspaceRootPath: 'Project',
|
||||
entries: [{ path: 'Project/Docs/case.md' }],
|
||||
});
|
||||
|
||||
await expect(api.storage.data.read('search-index')).resolves.toEqual({
|
||||
version: 1,
|
||||
workspaceRootPath: 'Project',
|
||||
entries: [{ path: 'Project/Docs/case.md' }],
|
||||
});
|
||||
await expect(api.settings.read('search-index')).resolves.toEqual({ source: 'settings' });
|
||||
await expect(api.storage.data.read('missing')).resolves.toEqual({});
|
||||
});
|
||||
```
|
||||
|
||||
Also extend `mock API exposes the bundled runtime shape` with:
|
||||
|
||||
```ts
|
||||
expect(typeof api.storage.data.read).toBe('function');
|
||||
expect(typeof api.storage.data.write).toBe('function');
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-sdk
|
||||
npx vitest run src/plugin-api.test.ts -t "plugin data persists separately from settings in the mock API namespace"
|
||||
```
|
||||
|
||||
Expected: TypeScript/Vitest fails because `storage` does not exist on `VerstakPluginAPI`.
|
||||
|
||||
- [ ] **Step 3: Add the SDK type**
|
||||
|
||||
In `src/plugin-api.ts`, add this type near `PluginCommandArgs`:
|
||||
|
||||
```ts
|
||||
export type PluginDataJSON = Record<string, unknown>;
|
||||
```
|
||||
|
||||
Then add this block to `VerstakPluginAPI` immediately after `settings`:
|
||||
|
||||
```ts
|
||||
storage: {
|
||||
data: {
|
||||
read(name: string): Promise<PluginDataJSON>;
|
||||
write(name: string, data: PluginDataJSON): Promise<void>;
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Add mock storage data**
|
||||
|
||||
In `src/test-utils.ts`, add an in-memory data map near the existing `settings` map:
|
||||
|
||||
```ts
|
||||
const pluginData = new Map<string, Record<string, unknown>>();
|
||||
```
|
||||
|
||||
Then add this `storage` object immediately after the returned `settings` object:
|
||||
|
||||
```ts
|
||||
storage: {
|
||||
data: {
|
||||
read: vi.fn(async (name: string) => ({ ...(pluginData.get(name) || {}) })),
|
||||
write: vi.fn(async (name: string, data: Record<string, unknown>) => {
|
||||
pluginData.set(name, { ...(data || {}) });
|
||||
}),
|
||||
},
|
||||
},
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Run GREEN**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-sdk
|
||||
npx vitest run src/plugin-api.test.ts -t "plugin data persists separately from settings in the mock API namespace"
|
||||
```
|
||||
|
||||
Expected: one matching test passes.
|
||||
|
||||
- [ ] **Step 6: Run SDK verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-sdk
|
||||
npm test
|
||||
npm run build
|
||||
```
|
||||
|
||||
Expected: Vitest exits 0; `tsc` exits 0 and updates `dist/`.
|
||||
|
||||
- [ ] **Step 7: Commit and push SDK**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-sdk
|
||||
git status --short
|
||||
git add src/plugin-api.ts src/test-utils.ts src/plugin-api.test.ts dist
|
||||
git commit -m "feat: expose plugin data storage API"
|
||||
git push
|
||||
```
|
||||
|
||||
Expected: commit is created and pushed; `git status --short --branch` reports `## main...origin/main`.
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Desktop Frontend Storage Bridge
|
||||
|
||||
**Files:**
|
||||
- Modify: `/home/mirivlad/git/verstak2/verstak-desktop/frontend/src/lib/plugin-host/VerstakPluginAPI.js`
|
||||
- Modify: `/home/mirivlad/git/verstak2/verstak-desktop/frontend/tests/plugin-api-contributions-test.mjs`
|
||||
- Modify: `/home/mirivlad/git/verstak2/verstak-desktop/frontend/src/lib/test/wails-mock.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes:
|
||||
- Existing Wails methods `App.ReadPluginDataJSON(pluginId, name)` and `App.WritePluginDataJSON(pluginId, name, data)`.
|
||||
- Produces:
|
||||
- Runtime `api.storage.data.read(name)` and `api.storage.data.write(name, data)`.
|
||||
|
||||
- [ ] **Step 1: Write the failing bridge smoke test**
|
||||
|
||||
In `frontend/tests/plugin-api-contributions-test.mjs`, extend `window.go.api.App.App` with plugin data storage:
|
||||
|
||||
```js
|
||||
const pluginData = {};
|
||||
```
|
||||
|
||||
Place it before `globalThis.window = { ... }`.
|
||||
|
||||
Inside `App`, add:
|
||||
|
||||
```js
|
||||
ReadPluginDataJSON: (pluginId, name) => Promise.resolve([
|
||||
Object.assign({}, (pluginData[pluginId] && pluginData[pluginId][name]) || {}),
|
||||
'',
|
||||
]),
|
||||
WritePluginDataJSON: (pluginId, name, data) => {
|
||||
pluginData[pluginId] = pluginData[pluginId] || {};
|
||||
pluginData[pluginId][name] = Object.assign({}, data || {});
|
||||
return Promise.resolve('');
|
||||
},
|
||||
```
|
||||
|
||||
After the existing `executeFor` assertion, add:
|
||||
|
||||
```js
|
||||
if (!api.storage || !api.storage.data || typeof api.storage.data.read !== 'function' || typeof api.storage.data.write !== 'function') {
|
||||
throw new Error('api.storage.data read/write is missing');
|
||||
}
|
||||
|
||||
await api.storage.data.write('search-index', { version: 1, workspaceRootPath: 'Project' });
|
||||
const stored = await api.storage.data.read('search-index');
|
||||
if (stored.version !== 1 || stored.workspaceRootPath !== 'Project') {
|
||||
throw new Error(`unexpected storage data: ${JSON.stringify(stored)}`);
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-desktop
|
||||
node frontend/tests/plugin-api-contributions-test.mjs
|
||||
```
|
||||
|
||||
Expected: fails with `api.storage.data read/write is missing`.
|
||||
|
||||
- [ ] **Step 3: Implement bridge storage**
|
||||
|
||||
In `frontend/src/lib/plugin-host/VerstakPluginAPI.js`, add this block immediately after `settings` and before `files`:
|
||||
|
||||
```js
|
||||
storage: {
|
||||
data: {
|
||||
read: function(name) {
|
||||
assertActive('storage.data.read(' + name + ')');
|
||||
if (!name) {
|
||||
throw new Error('storage.data.read requires a name');
|
||||
}
|
||||
return callBackend(pluginId, 'storage.data.read(' + name + ')', function() {
|
||||
return App.ReadPluginDataJSON(pluginId, name);
|
||||
}).then(function(data) {
|
||||
return data || {};
|
||||
});
|
||||
},
|
||||
write: function(name, data) {
|
||||
assertActive('storage.data.write(' + name + ')');
|
||||
if (!name) {
|
||||
throw new Error('storage.data.write requires a name');
|
||||
}
|
||||
return callBackendErrorString(pluginId, 'storage.data.write(' + name + ')', function() {
|
||||
return App.WritePluginDataJSON(pluginId, name, data || {});
|
||||
});
|
||||
}
|
||||
}
|
||||
},
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Implement browser mock storage**
|
||||
|
||||
In `frontend/src/lib/test/wails-mock.js`, add a module-scope object near existing plugin mock state:
|
||||
|
||||
```js
|
||||
var pluginData = {};
|
||||
```
|
||||
|
||||
Replace the current `ReadPluginDataJSON` and `WritePluginDataJSON` stubs with:
|
||||
|
||||
```js
|
||||
ReadPluginDataJSON: function (pluginId, name) {
|
||||
var data = (pluginData[pluginId] && pluginData[pluginId][name]) || {};
|
||||
return Promise.resolve([Object.assign({}, data), '']);
|
||||
},
|
||||
WritePluginDataJSON: function (pluginId, name, data) {
|
||||
pluginData[pluginId] = pluginData[pluginId] || {};
|
||||
pluginData[pluginId][name] = Object.assign({}, data || {});
|
||||
return Promise.resolve('');
|
||||
},
|
||||
```
|
||||
|
||||
If `pluginData` already exists in another scope, reuse that exact object and do not create a duplicate.
|
||||
|
||||
- [ ] **Step 5: Run GREEN**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-desktop
|
||||
node frontend/tests/plugin-api-contributions-test.mjs
|
||||
```
|
||||
|
||||
Expected: `plugin api contributions smoke passed`.
|
||||
|
||||
- [ ] **Step 6: Run desktop frontend verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-desktop/frontend
|
||||
npm run build
|
||||
```
|
||||
|
||||
Expected: Vite build exits 0.
|
||||
|
||||
- [ ] **Step 7: Commit and push desktop**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-desktop
|
||||
git status --short
|
||||
git add frontend/src/lib/plugin-host/VerstakPluginAPI.js frontend/tests/plugin-api-contributions-test.mjs frontend/src/lib/test/wails-mock.js
|
||||
git commit -m "feat: expose plugin data storage bridge"
|
||||
git push
|
||||
```
|
||||
|
||||
Expected: commit is created and pushed; `git status --short --branch` reports `## main...origin/main`.
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Search Plugin Persistent Index And Provider Hosting
|
||||
|
||||
**Files:**
|
||||
- Modify: `/home/mirivlad/git/verstak2/verstak-official-plugins/plugins/search/plugin.json`
|
||||
- Modify: `/home/mirivlad/git/verstak2/verstak-official-plugins/plugins/search/frontend/src/index.js`
|
||||
- Modify: `/home/mirivlad/git/verstak2/verstak-official-plugins/scripts/smoke-search-plugin.js`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes:
|
||||
- `api.storage.data.read('search-index')`
|
||||
- `api.storage.data.write('search-index', data)`
|
||||
- `api.events.subscribe('file.changed', handler)`
|
||||
- `api.contributions.list('searchProviders')`
|
||||
- `api.commands.register(commandId, handler)`
|
||||
- `api.commands.executeFor(pluginId, commandId, args)`
|
||||
- Produces:
|
||||
- Command `verstak.search.searchVaultText`.
|
||||
- Search provider host behavior that merges local indexed results with external provider results.
|
||||
|
||||
- [ ] **Step 1: Write failing smoke coverage for storage and providers**
|
||||
|
||||
In `scripts/smoke-search-plugin.js`, extend the fake API with:
|
||||
|
||||
```js
|
||||
const eventHandlers = {};
|
||||
const storageWrites = [];
|
||||
const providerCalls = [];
|
||||
let caseText = '# Case\nTarget phrase is here.\n';
|
||||
```
|
||||
|
||||
Add to `api`:
|
||||
|
||||
```js
|
||||
storage: {
|
||||
data: {
|
||||
read: async (name) => {
|
||||
if (name !== 'search-index') throw new Error('unexpected storage read ' + name);
|
||||
return {};
|
||||
},
|
||||
write: async (name, data) => {
|
||||
if (name !== 'search-index') throw new Error('unexpected storage write ' + name);
|
||||
storageWrites.push(data);
|
||||
},
|
||||
},
|
||||
},
|
||||
events: {
|
||||
subscribe: async (name, handler) => {
|
||||
eventHandlers[name] = eventHandlers[name] || [];
|
||||
eventHandlers[name].push(handler);
|
||||
return () => {
|
||||
eventHandlers[name] = (eventHandlers[name] || []).filter((item) => item !== handler);
|
||||
};
|
||||
},
|
||||
},
|
||||
contributions: {
|
||||
list: async (point) => point === 'searchProviders' ? [{
|
||||
pluginId: 'provider.search',
|
||||
id: 'provider.search.activity',
|
||||
label: 'Activity Provider',
|
||||
handler: 'provider.search.activity',
|
||||
}, {
|
||||
pluginId: 'provider.search',
|
||||
id: 'provider.search.fail',
|
||||
label: 'Failing Provider',
|
||||
handler: 'provider.search.fail',
|
||||
}] : [],
|
||||
},
|
||||
commands: {
|
||||
register: async () => () => {},
|
||||
executeFor: async (pluginId, commandId, args) => {
|
||||
providerCalls.push({ pluginId, commandId, args });
|
||||
if (commandId === 'provider.search.fail') throw new Error('provider unavailable');
|
||||
return {
|
||||
status: 'handled',
|
||||
pluginId,
|
||||
commandId,
|
||||
result: {
|
||||
results: [{
|
||||
title: 'Activity hit',
|
||||
snippet: 'Target activity result',
|
||||
matchType: 'Activity',
|
||||
providerId: 'provider.search.activity',
|
||||
providerLabel: 'Activity Provider',
|
||||
type: 'activity',
|
||||
openable: false,
|
||||
score: 75,
|
||||
}],
|
||||
},
|
||||
};
|
||||
},
|
||||
},
|
||||
```
|
||||
|
||||
Change the `readText` branch for `Project/Docs/case.md` to:
|
||||
|
||||
```js
|
||||
if (relativePath === 'Project/Docs/case.md') return caseText;
|
||||
```
|
||||
|
||||
After the first `target` search assertions, add:
|
||||
|
||||
```js
|
||||
if (storageWrites.length === 0) throw new Error('search index should be persisted after initial build');
|
||||
if (!container.textContent.includes('Activity hit')) throw new Error('provider result should be rendered');
|
||||
if (!container.textContent.includes('Activity Provider')) throw new Error('provider label should be rendered');
|
||||
if (!container.textContent.includes('provider unavailable')) throw new Error('provider failure should be reported without hiding results');
|
||||
if (providerCalls.length === 0 || providerCalls[0].args.query !== 'target') {
|
||||
throw new Error(`provider was not called with search query: ${JSON.stringify(providerCalls)}`);
|
||||
}
|
||||
```
|
||||
|
||||
Then add this event refresh check before the `image` search:
|
||||
|
||||
```js
|
||||
caseText = '# Case\nFresh event phrase is here.\n';
|
||||
(eventHandlers['file.changed'] || []).forEach((handler) => handler({
|
||||
name: 'file.changed',
|
||||
payload: { path: 'Project/Docs/case.md', operation: 'update' },
|
||||
timestamp: new Date().toISOString(),
|
||||
}));
|
||||
await flush();
|
||||
|
||||
input = queryInput();
|
||||
input.value = 'fresh';
|
||||
input.dispatchEvent('input');
|
||||
await wait(360);
|
||||
|
||||
if (!container.textContent.includes('Fresh event phrase is here')) {
|
||||
throw new Error('file.changed should refresh persisted search index content');
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run RED**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-official-plugins
|
||||
node scripts/smoke-search-plugin.js
|
||||
```
|
||||
|
||||
Expected: fails because `verstak.search` does not call storage or provider APIs.
|
||||
|
||||
- [ ] **Step 3: Update the Search manifest**
|
||||
|
||||
Change `plugins/search/plugin.json`:
|
||||
|
||||
```json
|
||||
"permissions": [
|
||||
"files.read",
|
||||
"workbench.open",
|
||||
"storage.namespace",
|
||||
"events.subscribe",
|
||||
"commands.register",
|
||||
"ui.register"
|
||||
],
|
||||
```
|
||||
|
||||
Add `commands` before `workspaceItems`:
|
||||
|
||||
```json
|
||||
"commands": [
|
||||
{
|
||||
"id": "verstak.search.searchVaultText",
|
||||
"title": "Search Vault Text",
|
||||
"handler": "searchVaultText"
|
||||
}
|
||||
],
|
||||
```
|
||||
|
||||
Change the local search provider handler:
|
||||
|
||||
```json
|
||||
"handler": "verstak.search.searchVaultText"
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Add index helpers in Search frontend**
|
||||
|
||||
In `plugins/search/frontend/src/index.js`, add constants near existing limits:
|
||||
|
||||
```js
|
||||
var INDEX_NAME = 'search-index';
|
||||
var INDEX_VERSION = 1;
|
||||
```
|
||||
|
||||
Add helper functions after `scanPath`:
|
||||
|
||||
```js
|
||||
function entryToIndexEntry(entry, text) {
|
||||
return {
|
||||
path: entry.relativePath,
|
||||
name: entry.name || entry.relativePath,
|
||||
type: entry.type,
|
||||
extension: extension(entry),
|
||||
size: Number(entry.size || 0),
|
||||
modifiedAt: String(entry.modifiedAt || ''),
|
||||
text: String(text || '').replace(/\s+/g, ' ').slice(0, 4000)
|
||||
};
|
||||
}
|
||||
|
||||
function validIndex(index, rootPath) {
|
||||
return index && index.version === INDEX_VERSION &&
|
||||
cleanPath(index.workspaceRootPath) === cleanPath(rootPath) &&
|
||||
Array.isArray(index.entries);
|
||||
}
|
||||
|
||||
async function buildIndex(api, rootPath) {
|
||||
var entries = await collectEntries(api, rootPath);
|
||||
var indexed = [];
|
||||
for (var i = 0; i < entries.length; i++) {
|
||||
var entry = entries[i];
|
||||
var text = '';
|
||||
if (isTextFile(entry)) {
|
||||
try {
|
||||
text = await api.files.readText(entry.relativePath);
|
||||
} catch (err) {
|
||||
text = '';
|
||||
}
|
||||
}
|
||||
indexed.push(entryToIndexEntry(entry, text));
|
||||
}
|
||||
return {
|
||||
version: INDEX_VERSION,
|
||||
workspaceRootPath: cleanPath(rootPath),
|
||||
builtAt: new Date().toISOString(),
|
||||
entries: indexed
|
||||
};
|
||||
}
|
||||
|
||||
function searchIndex(index, query) {
|
||||
var results = [];
|
||||
var needle = String(query || '').toLowerCase();
|
||||
if (!validIndex(index, index && index.workspaceRootPath) || needle.length < 2) return results;
|
||||
for (var i = 0; i < index.entries.length && results.length < MAX_RESULTS; i++) {
|
||||
var entry = index.entries[i];
|
||||
var path = String(entry.path || '');
|
||||
var name = String(entry.name || '');
|
||||
var text = String(entry.text || '');
|
||||
var pathHit = path.toLowerCase().indexOf(needle) !== -1 || name.toLowerCase().indexOf(needle) !== -1;
|
||||
var textIndex = text.toLowerCase().indexOf(needle);
|
||||
if (pathHit) {
|
||||
results.push({
|
||||
path: path,
|
||||
type: entry.type,
|
||||
matchType: entry.type === 'folder' ? 'Folder name' : 'File name',
|
||||
openable: entry.type !== 'folder',
|
||||
line: 0,
|
||||
snippet: entry.type === 'folder' ? 'Folder name/path match' : 'File name/path match',
|
||||
providerLabel: 'Vault Text Search',
|
||||
score: 100
|
||||
});
|
||||
} else if (textIndex !== -1) {
|
||||
results.push({
|
||||
path: path,
|
||||
type: entry.type,
|
||||
matchType: 'Content match',
|
||||
openable: true,
|
||||
line: 0,
|
||||
snippet: snippet(text, textIndex, needle.length),
|
||||
providerLabel: 'Vault Text Search',
|
||||
score: 90
|
||||
});
|
||||
}
|
||||
}
|
||||
return results;
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 5: Add provider fan-out helper**
|
||||
|
||||
Add this helper after `searchIndex`:
|
||||
|
||||
```js
|
||||
async function runExternalProviders(api, rootPath, query) {
|
||||
if (!api.contributions || typeof api.contributions.list !== 'function' ||
|
||||
!api.commands || typeof api.commands.executeFor !== 'function') {
|
||||
return { results: [], failed: 0 };
|
||||
}
|
||||
var providers = await api.contributions.list('searchProviders');
|
||||
providers = Array.isArray(providers) ? providers : [];
|
||||
var results = [];
|
||||
var failed = 0;
|
||||
for (var i = 0; i < providers.length; i++) {
|
||||
var provider = providers[i];
|
||||
if (!provider || !provider.pluginId || !provider.handler) continue;
|
||||
if (provider.pluginId === 'verstak.search' && provider.handler === 'verstak.search.searchVaultText') continue;
|
||||
try {
|
||||
var response = await api.commands.executeFor(provider.pluginId, provider.handler, {
|
||||
source: 'search',
|
||||
providerId: provider.id,
|
||||
query: query,
|
||||
workspaceRootPath: cleanPath(rootPath),
|
||||
limit: MAX_RESULTS
|
||||
});
|
||||
var payload = response && response.result;
|
||||
var providerResults = payload && Array.isArray(payload.results) ? payload.results : [];
|
||||
providerResults.forEach(function (result) {
|
||||
results.push(Object.assign({}, result, {
|
||||
providerId: result.providerId || provider.id,
|
||||
providerLabel: result.providerLabel || provider.label || provider.id
|
||||
}));
|
||||
});
|
||||
} catch (err) {
|
||||
failed += 1;
|
||||
}
|
||||
}
|
||||
return { results: results, failed: failed };
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 6: Wire index lifecycle in SearchView**
|
||||
|
||||
Inside `SearchView.mount`, extend state:
|
||||
|
||||
```js
|
||||
var state = { query: '', searching: false, results: [], status: 'Enter at least 2 characters.', error: '', providerFailures: 0 };
|
||||
var index = null;
|
||||
var cleanupFns = [];
|
||||
```
|
||||
|
||||
Add these functions inside `mount` before `render`:
|
||||
|
||||
```js
|
||||
async function loadOrBuildIndex() {
|
||||
if (api.storage && api.storage.data && typeof api.storage.data.read === 'function') {
|
||||
try {
|
||||
var stored = await api.storage.data.read(INDEX_NAME);
|
||||
if (validIndex(stored, rootPath)) {
|
||||
index = stored;
|
||||
return;
|
||||
}
|
||||
} catch (err) {
|
||||
state.status = 'Search index unavailable; using live scan.';
|
||||
}
|
||||
}
|
||||
index = await buildIndex(api, rootPath);
|
||||
if (api.storage && api.storage.data && typeof api.storage.data.write === 'function') {
|
||||
try {
|
||||
await api.storage.data.write(INDEX_NAME, index);
|
||||
} catch (err) {
|
||||
state.status = 'Search index could not be saved.';
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async function refreshIndexPath(path) {
|
||||
var clean = cleanPath(path);
|
||||
if (!clean || (rootPath && clean !== rootPath && clean.indexOf(rootPath + '/') !== 0)) return;
|
||||
index = await buildIndex(api, rootPath);
|
||||
if (api.storage && api.storage.data && typeof api.storage.data.write === 'function') {
|
||||
await api.storage.data.write(INDEX_NAME, index);
|
||||
}
|
||||
}
|
||||
|
||||
function subscribeIndexEvents() {
|
||||
if (!api.events || typeof api.events.subscribe !== 'function') return;
|
||||
api.events.subscribe('file.changed', function (event) {
|
||||
var payload = (event && event.payload) || {};
|
||||
refreshIndexPath(payload.path || payload.toPath || '').catch(function (err) {
|
||||
console.error('[search] index refresh:', err);
|
||||
});
|
||||
}).then(function (unsubscribe) {
|
||||
if (typeof unsubscribe === 'function') cleanupFns.push(unsubscribe);
|
||||
}).catch(function (err) {
|
||||
console.error('[search] file.changed subscription:', err);
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
At the end of `mount`, before `render();`, add:
|
||||
|
||||
```js
|
||||
if (api.commands && typeof api.commands.register === 'function') {
|
||||
api.commands.register('verstak.search.searchVaultText', function (args) {
|
||||
var q = String((args && args.query) || '');
|
||||
return { results: searchIndex(index, q) };
|
||||
}).then(function (unsubscribe) {
|
||||
if (typeof unsubscribe === 'function') cleanupFns.push(unsubscribe);
|
||||
}).catch(function (err) {
|
||||
console.error('[search] command register:', err);
|
||||
});
|
||||
}
|
||||
loadOrBuildIndex().then(function () {
|
||||
subscribeIndexEvents();
|
||||
}).catch(function (err) {
|
||||
console.error('[search] initial index:', err);
|
||||
});
|
||||
```
|
||||
|
||||
In cleanup, add:
|
||||
|
||||
```js
|
||||
while (cleanupFns.length) {
|
||||
try { cleanupFns.pop()(); } catch (err) { console.error('[search] cleanup:', err); }
|
||||
}
|
||||
```
|
||||
|
||||
- [ ] **Step 7: Replace search execution with index plus provider results**
|
||||
|
||||
In `search()`, replace:
|
||||
|
||||
```js
|
||||
var results = await runSearch(api, rootPath, state.query);
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```js
|
||||
if (!index) await loadOrBuildIndex();
|
||||
var localResults = searchIndex(index, state.query);
|
||||
var providerResponse = await runExternalProviders(api, rootPath, state.query);
|
||||
var results = localResults.concat(providerResponse.results)
|
||||
.sort(function (a, b) {
|
||||
return Number(b.score || 0) - Number(a.score || 0) ||
|
||||
String(a.path || a.title || '').localeCompare(String(b.path || b.title || ''));
|
||||
})
|
||||
.slice(0, MAX_RESULTS);
|
||||
state.providerFailures = providerResponse.failed;
|
||||
```
|
||||
|
||||
Change status assignment to:
|
||||
|
||||
```js
|
||||
state.status = state.results.length + ' result' + (state.results.length === 1 ? '' : 's') +
|
||||
(state.providerFailures ? ' - ' + state.providerFailures + ' provider unavailable' + (state.providerFailures === 1 ? '' : 's') : '');
|
||||
```
|
||||
|
||||
In `render`, change result title rendering to support provider results:
|
||||
|
||||
```js
|
||||
el('div', { className: 'search-path' }, [result.path || result.title || 'Untitled result']),
|
||||
el('div', { className: 'search-snippet' }, [result.snippet || '']),
|
||||
el('div', { className: 'search-meta' }, [
|
||||
(result.providerLabel ? result.providerLabel + ' - ' : '') +
|
||||
(result.matchType || result.type || 'Search result') +
|
||||
(result.line ? ' - Line ' + result.line : '')
|
||||
])
|
||||
```
|
||||
|
||||
Change openResource payload to use `result.resource` when present:
|
||||
|
||||
```js
|
||||
var resource = result.resource || {
|
||||
kind: 'vault-file',
|
||||
path: result.path,
|
||||
mode: 'view'
|
||||
};
|
||||
api.workbench.openResource(resource).catch(function (err) { console.error('[search] openResource:', err); });
|
||||
```
|
||||
|
||||
- [ ] **Step 8: Run GREEN for focused Search smoke**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-official-plugins
|
||||
node scripts/smoke-search-plugin.js
|
||||
```
|
||||
|
||||
Expected: `search plugin smoke passed`.
|
||||
|
||||
- [ ] **Step 9: Run official plugin verification**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-official-plugins
|
||||
./scripts/check.sh
|
||||
```
|
||||
|
||||
Expected: check exits 0 and search frontend smoke passes.
|
||||
|
||||
- [ ] **Step 10: Commit and push official plugins**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-official-plugins
|
||||
git status --short
|
||||
git add plugins/search/plugin.json plugins/search/frontend/src/index.js scripts/smoke-search-plugin.js
|
||||
git commit -m "feat: persist search index and host providers"
|
||||
git push
|
||||
```
|
||||
|
||||
Expected: commit is created and pushed; `git status --short --branch` reports `## main...origin/main`.
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Documentation And Roadmap Update
|
||||
|
||||
**Files:**
|
||||
- Modify: `/home/mirivlad/git/verstak2/verstak-docs/05_Official_Plugins.md`
|
||||
- Modify: `/home/mirivlad/git/verstak2/verstak-docs/07_Full_Implementation_Roadmap.md`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes:
|
||||
- Verified commits from SDK, desktop, and official plugins.
|
||||
- Produces:
|
||||
- Docs that match implemented Search behavior.
|
||||
|
||||
- [ ] **Step 1: Update official plugin status text**
|
||||
|
||||
In `05_Official_Plugins.md`, replace the final Search status sentence:
|
||||
|
||||
```md
|
||||
Persistent indexing and cross-provider runtime hosting are still future work.
|
||||
```
|
||||
|
||||
with:
|
||||
|
||||
```md
|
||||
It persists a workspace-scoped JSON search index in the plugin data namespace,
|
||||
refreshes it from public file events, registers its own vault-text provider as
|
||||
a command-backed `searchProviders` handler, and fans out to other registered
|
||||
provider commands at runtime. Full-text ranking, typo/layout tolerant search,
|
||||
and sidecar indexing remain later work.
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Update roadmap checkbox**
|
||||
|
||||
In `07_Full_Implementation_Roadmap.md`, change:
|
||||
|
||||
```md
|
||||
- implement persistent search index and cross-provider runtime hosting;
|
||||
```
|
||||
|
||||
to:
|
||||
|
||||
```md
|
||||
- [x] implement persistent search index and cross-provider runtime hosting;
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Verify docs diff**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-docs
|
||||
git diff --check
|
||||
rg -n "Persistent indexing and cross-provider runtime hosting are still future work|implement persistent search index and cross-provider runtime hosting" 05_Official_Plugins.md 07_Full_Implementation_Roadmap.md
|
||||
```
|
||||
|
||||
Expected: `git diff --check` exits 0. The `rg` command shows the checked roadmap line and does not show the old future-work sentence.
|
||||
|
||||
- [ ] **Step 4: Commit and push docs**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-docs
|
||||
git status --short
|
||||
git add 05_Official_Plugins.md 07_Full_Implementation_Roadmap.md
|
||||
git commit -m "docs: mark persistent search hosting complete"
|
||||
git push
|
||||
```
|
||||
|
||||
Expected: commit is created and pushed; `git status --short --branch` reports `## main...origin/main`.
|
||||
|
||||
---
|
||||
|
||||
## Final Verification Checklist
|
||||
|
||||
Run after all tasks are implemented:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-sdk
|
||||
npm test
|
||||
npm run build
|
||||
```
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-desktop
|
||||
node frontend/tests/plugin-api-contributions-test.mjs
|
||||
cd frontend
|
||||
npm run build
|
||||
```
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-official-plugins
|
||||
node scripts/smoke-search-plugin.js
|
||||
./scripts/check.sh
|
||||
```
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2/verstak-docs
|
||||
git diff --check
|
||||
```
|
||||
|
||||
Then inspect:
|
||||
|
||||
```bash
|
||||
cd /home/mirivlad/git/verstak2
|
||||
git -C verstak-sdk status --short --branch
|
||||
git -C verstak-desktop status --short --branch
|
||||
git -C verstak-official-plugins status --short --branch
|
||||
git -C verstak-docs status --short --branch
|
||||
```
|
||||
|
||||
Expected: each changed repository reports `## main...origin/main`.
|
||||
Loading…
Reference in New Issue