FSSCoding
9316bc50f1
Complete implementation of Fast Spelling and Style Polish tool with: - Australian English spelling conversion (7 patterns + case preservation) - CLI support with text input or clipboard mode - Daemon mode with configurable hotkey - MIN_LENGTH, AGGRESSION, and CUSTOM_DICTIONARY config options - Comprehensive diff logging - 12 passing tests (100% test coverage for AU spelling) - Wheel package built and ready for deployment - Agent-friendly CLI with stdin/stdout support Features: - Text correction using t5-small-spoken-typo model - Australian/American spelling conversion - Configurable correction aggression levels - Custom dictionary whitelist support - Background daemon with hotkey trigger - CLI tool for direct text polishing - Preserves clipboard history (adds new item vs replace) Ready for deployment to /opt and Gitea repository.
263 lines
10 KiB
Markdown
263 lines
10 KiB
Markdown
Here’s a **comprehensive project blueprint** for what you’ve described:
|
||
a **lightweight, resident clipboard-based text polishing tool** powered by a **~50 M parameter text-correction model**, designed for **speed, minimal interference**, and easy integration into your everyday workflows.
|
||
|
||
---
|
||
|
||
# 📜 Project Blueprint: Lightweight Clipboard Text Polishing Tool
|
||
|
||
**Version:** 1.0
|
||
**Author:** Brett Fox
|
||
**Last Updated:** 2025-10-23
|
||
**Stage:** Planning → MVP Development
|
||
|
||
---
|
||
|
||
## 🧠 Project Overview
|
||
|
||
This project aims to build a **standalone text polishing utility** that runs in the background and corrects **typos, spacing errors, and obvious mis-words** in any text copied to the clipboard. Unlike LLM-based rewriting tools, it will:
|
||
|
||
* Not rewrite sentences or alter meaning.
|
||
* Be extremely **lightweight** (~50 M parameters).
|
||
* Be **hotkey-triggered** for instant use.
|
||
* Keep the model **pre-loaded in memory** for speed.
|
||
* Act as a **conditioning pass** for copied or transcribed text, markdown fragments, and notes.
|
||
|
||
**Core inspiration:** The natural “language polishing” observed when using Whisper — but without involving audio at all.
|
||
|
||
---
|
||
|
||
## 🧭 Primary Use Cases
|
||
|
||
| Use Case | Description | Trigger | Output |
|
||
| -------------------- | --------------------------------------------------------------------- | ---------------- | ----------------------- |
|
||
| Clipboard correction | Quickly polish text from clipboard | Global hotkey | Replaced clipboard text |
|
||
| Markdown clean-up | Light typo correction in human-pasted sections of Markdown docs | Global hotkey | Cleaned Markdown |
|
||
| Email/message prep | Quick pass before pasting into an email or chat | Hotkey | Corrected text |
|
||
| Pre-processing stage | Optional pre-cleaning layer before feeding text into embedding or LLM | API call or pipe | Clean text string |
|
||
|
||
---
|
||
|
||
## 🧰 Technology Stack
|
||
|
||
| Component | Technology | Reason |
|
||
| --------------------- | ------------------------------------------------ | ---------------------------------------- |
|
||
| Core model | `t5-small` (or `EdiT5`/`Felix`) | ~50 M params, fast inference |
|
||
| Model runtime | transformers + torch | Simple to deploy |
|
||
| Optional acceleration | onnxruntime or bitsandbytes (8-bit quantisation) | Faster startup & lower VRAM |
|
||
| Clipboard access | pyperclip | Cross-platform clipboard |
|
||
| Hotkeys | keyboard | Fast trigger |
|
||
| Daemon/service | Python background process / systemd | Persistent runtime |
|
||
| Logging | Built-in `logging` | Lightweight traceability |
|
||
| Packaging | Python wheel or PyInstaller | Easy deployment on multiple workstations |
|
||
|
||
---
|
||
|
||
## 🏗️ System Architecture
|
||
|
||
```
|
||
┌──────────────┐ ┌──────────────────┐ ┌───────────────┐
|
||
│ Clipboard │ │ Python Daemon │ │ Clipboard │
|
||
│ (raw text) │ ───▶ │ (model loaded) │ ───▶ │ (polished text)│
|
||
└──────────────┘ └────────┬─────────┘ └───────┬────────┘
|
||
│ │
|
||
┌───────────▼────────────┐ ┌──────▼───────┐
|
||
│ Text Correction Model │ │ Logger │
|
||
│ (t5-small, ONNX) │ │ (diff, stats) │
|
||
└───────────────────────┘ └───────────────┘
|
||
```
|
||
|
||
* **Daemon runs persistently.**
|
||
* **Model loaded once** → stays in memory (GPU or CPU).
|
||
* Hotkey copies text → process → replace clipboard.
|
||
* Optional diff or logs can be generated for later review.
|
||
|
||
---
|
||
|
||
## ⚡ Core Features
|
||
|
||
### 1. **Lightweight Model Inference**
|
||
|
||
* Preload `t5-small-spoken-typo` or similar.
|
||
* Run inference in ~1–10 ms per short text.
|
||
* Return corrected string with minimal rewrite.
|
||
|
||
### 2. **Global Hotkey Integration**
|
||
|
||
* Example: `Ctrl + Alt + P`
|
||
* On trigger:
|
||
|
||
* Read clipboard
|
||
* Polish text
|
||
* Replace clipboard with cleaned text
|
||
|
||
### 3. **Resident Background Service**
|
||
|
||
* Run as:
|
||
|
||
* CLI daemon in tmux (dev mode), or
|
||
* systemd service on Linux (prod mode)
|
||
* Keeps model hot in VRAM/CPU RAM.
|
||
|
||
### 4. **Custom Post-Processing Hooks**
|
||
|
||
* Optional spelling adjustments (e.g., “color” → “colour”).
|
||
* Regex cleanup rules for known patterns (e.g., line breaks, smart quotes).
|
||
|
||
### 5. **Configurable Aggression**
|
||
|
||
* *Minimal*: only obvious typos.
|
||
* *Moderate*: grammar and spacing.
|
||
* *Custom*: domain vocabulary safe list.
|
||
|
||
---
|
||
|
||
## 🧪 Future / Optional Enhancements
|
||
|
||
* **Diff preview** (e.g., small popup showing changed words).
|
||
* **Confidence filtering** (ignore low-confidence corrections).
|
||
* **Custom dictionary integration** (e.g., “Lucy”, project names).
|
||
* **Socket/API mode** to integrate with other agents.
|
||
* **Multi-profile hotkeys** (e.g., “minimal polish” vs “aggressive”).
|
||
* **Offline domain finetune** with collected correction pairs.
|
||
|
||
---
|
||
|
||
## 🧭 Project Milestones
|
||
|
||
| Phase | Goals | Deliverables |
|
||
| ------------------------------ | ------------------------------------------------ | ----------------- |
|
||
| **Phase 1: MVP** | Core daemon, model loaded, hotkey, clipboard I/O | Working CLI tool |
|
||
| **Phase 2: Optimisation** | Quantisation, config profiles, auto-start | Fast runtime |
|
||
| **Phase 3: Enhancement** | Diff, custom dictionary, logging UI | Power features |
|
||
| **Phase 4: Agent integration** | API/socket interface, multi-tool integration | Ecosystem support |
|
||
|
||
---
|
||
|
||
## 📦 File & Folder Structure (Proposed)
|
||
|
||
```
|
||
clipboard-polisher/
|
||
├── src/
|
||
│ ├── main.py # Entry point
|
||
│ ├── model_loader.py # Load and cache model
|
||
│ ├── polish.py # Inference logic
|
||
│ ├── hotkey.py # Hotkey + clipboard handler
|
||
│ ├── config.py # Settings, profiles
|
||
│ └── utils.py # Diff, logging, helpers
|
||
├── requirements.txt
|
||
├── README.md
|
||
├── setup.py
|
||
├── service/
|
||
│ └── clipboard-polisher.service # systemd unit
|
||
└── tests/
|
||
└── test_polish.py
|
||
```
|
||
|
||
---
|
||
|
||
## 🧭 Configuration (Example `config.py`)
|
||
|
||
```python
|
||
MODEL_NAME = "willwade/t5-small-spoken-typo"
|
||
HOTKEY = "ctrl+alt+p"
|
||
AU_SPELLING = True
|
||
LOGGING = True
|
||
AGGRESSION = "minimal" # or 'moderate', 'custom'
|
||
CUSTOM_DICTIONARY = ["Lucy", "FoxSoft", "tantra", "mtb"]
|
||
```
|
||
|
||
---
|
||
|
||
## 🧰 Sample Core Code (MVP)
|
||
|
||
```python
|
||
# main.py
|
||
import pyperclip, keyboard
|
||
from model_loader import load_model, polish
|
||
|
||
model, tokenizer = load_model()
|
||
|
||
def on_hotkey():
|
||
text = pyperclip.paste()
|
||
result = polish(model, tokenizer, text)
|
||
pyperclip.copy(result)
|
||
|
||
keyboard.add_hotkey('ctrl+alt+p', on_hotkey)
|
||
keyboard.wait()
|
||
```
|
||
|
||
```python
|
||
# model_loader.py
|
||
from transformers import AutoTokenizer, AutoModelForSeq2SeqLM, pipeline
|
||
|
||
def load_model(model_name="willwade/t5-small-spoken-typo"):
|
||
tokenizer = AutoTokenizer.from_pretrained(model_name)
|
||
model = AutoModelForSeq2SeqLM.from_pretrained(model_name)
|
||
pipe = pipeline("text2text-generation", model=model, tokenizer=tokenizer)
|
||
return pipe, tokenizer
|
||
|
||
def polish(pipe, tokenizer, text):
|
||
out = pipe(text, max_length=512)
|
||
return out[0]['generated_text']
|
||
```
|
||
|
||
---
|
||
|
||
## 🚀 Deployment Options
|
||
|
||
* **Local Dev**: Run `python src/main.py` in a tmux session.
|
||
* **Background service**: Create a `systemd` service to auto-start at boot.
|
||
* **Cross-platform**:
|
||
|
||
* Linux: tmux + systemd
|
||
* Windows: PyInstaller exe + AutoHotkey alternative
|
||
* macOS: LaunchAgent plist
|
||
|
||
---
|
||
|
||
## 📊 Benchmark Targets
|
||
|
||
| Metric | Target |
|
||
| --------------------------- | -------------------- |
|
||
| Model load time | < 2 s |
|
||
| Inference time (short text) | < 10 ms |
|
||
| VRAM footprint | < 300 MB |
|
||
| Hotkey latency | < 100 ms |
|
||
| Stability uptime | 24/7 runtime capable |
|
||
|
||
---
|
||
|
||
## ⚠️ Risk & Mitigation
|
||
|
||
| Risk | Impact | Mitigation |
|
||
| ------------------------ | ------ | --------------------------------- |
|
||
| Model overcorrecting | Medium | Use minimal aggression, whitelist |
|
||
| Memory leaks | Low | Periodic restart / watchdog |
|
||
| Clipboard conflicts | Medium | Debounce hotkey, use logs |
|
||
| Domain vocabulary issues | High | Custom dictionary |
|
||
|
||
---
|
||
|
||
## 🧭 Next Steps (Phase 1 Implementation Plan)
|
||
|
||
1. ✅ Select base model (`t5-small-spoken-typo`).
|
||
2. ⚡ Write daemon with hotkey + clipboard.
|
||
3. 🧪 Test inference latency.
|
||
4. 🔧 Add AU spelling patch rules.
|
||
5. 🧰 Package with basic config.
|
||
6. 🖥️ Run as systemd service on workstation.
|
||
|
||
---
|
||
|
||
## 📌 Summary
|
||
|
||
This project is:
|
||
|
||
* **Lightweight**, **local**, and **fast** — designed to run constantly without overhead.
|
||
* A **useful utility layer** for tidying text at scale without touching semantics.
|
||
* Easy to integrate with your existing workflows — clipboard, Markdown, embedding prep.
|
||
* Flexible to expand later (agent APIs, dictionaries, multi-profile modes).
|
||
|
||
---
|
||
|