> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nelieo.com/llms.txt
> Use this file to discover all available pages before exploring further.
# NSP: The Live Application Substrate for AI Agent Automation
> Learn how NSP attaches to live applications, exposes structured JSON state, and lets AI agents operate software at >99% accuracy without screenshots.
NSP (Nelieo Substrate Protocol) is a runtime substrate daemon that attaches to running applications and exposes their internal state and actions as typed JSON. Instead of taking screenshots and guessing pixel coordinates, your AI agent reads live data directly from the application's runtime and calls typed functions to act on it — the same way a human developer would if they had direct access to the app's memory.
## Why Agents Fail Without NSP
Every vision-based AI agent today operates the same way: capture a screenshot, send it to a vision model, infer where to click, hope the click lands correctly, and repeat.
```text theme={null}
Screenshot (2 MB PNG)
→ Vision model inference (~8 seconds)
→ "Click at pixel (847, 392)"
→ Miss by 3 pixels
→ Screenshot again
→ Repeat forever
```
At 85% per-action accuracy — an optimistic figure for state-of-the-art vision agents — a 50-step task has a **0.03% end-to-end success rate**. Agents don't fail because the underlying AI is weak. They fail because the interface is fundamentally broken: screenshots are lossy compressions of structured data that already existed in memory.
| Metric | NSP | Vision-Based Agent |
| ------------------------- | ----------- | ------------------ |
| State read latency | **\~50 ms** | 8,000–15,000 ms |
| Action accuracy (JS apps) | **>99%** | 66–82% |
| 50-step task completion | **\~95%** | \~0.03% |
| Actions per minute | **\~120** | \~4–7 |
| State verification | **Always** | Never |
| Works offline | **Yes** | Depends on model |
## How NSP Solves It
NSP attaches directly to a running application's runtime and reads its live state as structured JSON. Your agent calls typed functions. No screenshots, no pixel coordinates, no guessing.
```bash theme={null}
GET /substrate/v1/state/12847
```
```json theme={null}
{
"app_name": "Gmail",
"runtime": "v8",
"confidence": 0.97,
"objects": {
"inbox.unread_count": 47,
"inbox.emails[0].subject": "Q3 Performance Report",
"inbox.emails[0].from": "alice@example.com",
"inbox.emails[0].is_read": false
},
"actions": {
"send_reply": {
"signature": "fn(thread_id: string, body: string) -> bool",
"reversibility": "irreversible_write"
},
"archive": {
"signature": "fn(email_id: string) -> bool",
"reversibility": "reversible_write"
}
}
}
```
The result is >99% accuracy on JavaScript apps, 120 actions per minute, and continuous 24/7 operation with state verification after every action.
## The Four Layers
The NSP daemon (`axon-daemon`) runs on the host machine alongside your applications. It has four layers that work together to turn live process memory into an agent-callable API.
Continuously scans all running OS processes every 500 ms, classifies each process's runtime (V8, JVM, CLR, or Native), and manages the full probe attachment lifecycle.
Language-specific probes attach to live processes and extract semantic state — no source code, no UI automation, and no prior instrumentation required.
Converts raw runtime data extracted by each probe into structured SSF JSON with stable, human-readable dot-notation keys.
A REST and WebSocket server on `localhost:7842` that any AI agent can call to read state, subscribe to real-time changes, and execute actions.
## Runtime Coverage
NSP supports four runtime families, each with a dedicated probe strategy and measured accuracy baseline.
| Runtime | Applications | Probe Method | Accuracy |
| ------------------- | ------------------------------------------------------------------------------------- | ------------------------------------------------- | -------- |
| **JavaScript / V8** | Gmail, Slack, VS Code, Notion, GitHub Desktop, Salesforce, WhatsApp, Discord, Shopify | CDP remote debugging + fast-path state extraction | >99% |
| **Java / JVM** | SAP, Eclipse, IntelliJ, Jenkins, Elasticsearch, ArduPilot GCS | Native JVM instrumentation + reflection | >97% |
| **.NET / CLR** | Excel, Word, PowerPoint, Visual Studio, Mission Planner, Azure tools | CLR heap inspection + managed bootstrapper | >97% |
| **Native C++** | Bloomberg Terminal, Photoshop, AutoCAD, Chrome internals | Process memory reading + Visual-Memory Grounding | >75% |
Native C++ accuracy is lower because compiled binaries have no metadata for semantic labeling. NSP uses Visual-Memory Grounding — a hybrid of memory layout analysis and a local vision model — to close the gap. Expect V8 and JVM accuracy on the applications your agents will use most.
## Next Steps
Install the daemon, configure Chrome, and call your first state endpoint in under 5 minutes.
Understand SubstrateState, SSF, action schemas, confidence scores, and reversibility tiers.
Install `nelieo-nsp`, attach to apps by name, and use typed helpers with full async support.
Complete reference for every REST and WebSocket endpoint exposed by the daemon.