102 lines
2.1 KiB
Markdown
102 lines
2.1 KiB
Markdown
# How To Run
|
|
|
|
1. Install dependencies:
|
|
|
|
```bash
|
|
python3 -m venv .venv
|
|
. .venv/bin/activate
|
|
python -m pip install -e ".[dev]"
|
|
```
|
|
|
|
2. Configure:
|
|
|
|
```bash
|
|
cp .env.example .env
|
|
```
|
|
|
|
The default `DUCK_MAIN_MODEL_PATH` points to `./models/Qwen3.6/nonMTP/Qwen3.6-35B-A3B-UD-Q4_K_M.gguf`.
|
|
|
|
`DUCK_ENABLE_REFLECTION=0` is the recommended default for the local single-slot
|
|
stack. Set it to `1` only when you explicitly want critic reflection after each
|
|
chat and accept that it can slow down the next request.
|
|
|
|
3. Start DuckLM:
|
|
|
|
```bash
|
|
bash scripts/duck.sh start
|
|
```
|
|
|
|
This starts the local stack:
|
|
|
|
- Qdrant vector memory on `http://127.0.0.1:6333/`
|
|
- `llama-server` on `http://127.0.0.1:8081/v1`
|
|
- DuckLM API/WebChat on `http://127.0.0.1:8000/`
|
|
|
|
Useful process commands:
|
|
|
|
```bash
|
|
bash scripts/duck.sh status
|
|
bash scripts/duck.sh logs --follow
|
|
bash scripts/duck.sh restart
|
|
bash scripts/duck.sh stop
|
|
```
|
|
|
|
Use live probes when you need backend diagnostics, not just process status:
|
|
|
|
```bash
|
|
bash scripts/duck.sh status --probe
|
|
curl --noproxy '*' 'http://127.0.0.1:8000/v1/status?probe=true'
|
|
```
|
|
|
|
4. Open WebChat:
|
|
|
|
```text
|
|
http://127.0.0.1:8000/
|
|
```
|
|
|
|
Low-level llama-only commands are still available when needed:
|
|
|
|
```bash
|
|
bash scripts/llama/start_main.sh status
|
|
bash scripts/llama/start_main.sh logs --follow
|
|
```
|
|
|
|
MTP/speculative variant:
|
|
|
|
```bash
|
|
bash scripts/duck.sh stop
|
|
bash scripts/duck-mtp.sh start
|
|
bash scripts/duck-mtp.sh status
|
|
bash scripts/duck-mtp.sh logs --follow
|
|
```
|
|
|
|
`duck-mtp.sh` keeps DuckLM on `http://127.0.0.1:8000/` and starts the MTP-backed
|
|
`llama-server` on the normal role endpoint `http://127.0.0.1:8081/v1`, so
|
|
`config/models.yaml` does not need to change.
|
|
|
|
5. Send a task:
|
|
|
|
```bash
|
|
curl -X POST http://127.0.0.1:8000/v1/chat \
|
|
-H "Content-Type: application/json" \
|
|
-d '{"message":"Скажи коротко, что ты DuckLM","workspace":"./workspace","debug":true}'
|
|
```
|
|
|
|
6. Inspect events:
|
|
|
|
```bash
|
|
curl http://127.0.0.1:8000/v1/tasks/<task_id>/events
|
|
```
|
|
|
|
7. Approvals:
|
|
|
|
```bash
|
|
curl http://127.0.0.1:8000/v1/approvals/pending
|
|
```
|
|
|
|
8. Stop services:
|
|
|
|
```bash
|
|
bash scripts/duck.sh stop
|
|
```
|