NSP: AI Agent Runtime Substrate for Live Applications

NSP (Nelieo State Protocol) is a runtime substrate that attaches to live applications and exposes their internal state as structured JSON. Instead of having your agent guess pixel coordinates from a screenshot, NSP lets it call typed functions directly against the running process — the same way a developer would call an API. The result is dramatically higher accuracy, lower latency, and reliable 24/7 task completion.

The Problem with Vision-Based Agents

Every vision-based agent today follows the same brittle loop: capture a screenshot, send it to a vision model, infer a pixel coordinate, click, and repeat. At 85% per-action accuracy, a 50-step task has a 0.03% end-to-end success rate. The agent isn’t the weak link — the interface is.

Screenshot (2MB PNG)
  → Vision model inference (8 seconds)
  → "Click at pixel (847, 392)"
  → Miss by 3 pixels
  → Screenshot again
  → Repeat forever

NSP replaces this loop entirely. It attaches directly to a running application’s runtime and reads its live state as structured JSON, then exposes typed action functions your agent can call in a single request.

GET /substrate/v1/state/12847

{
  "app_name": "Gmail",
  "runtime": "v8",
  "confidence": 0.97,
  "objects": {
    "inbox.unread_count": 47,
    "inbox.emails[0].subject": "Q3 Performance Report",
    "inbox.emails[0].from": "alice@example.com",
    "inbox.emails[0].is_read": false
  },
  "actions": {
    "send_reply": {
      "signature": "fn(thread_id: string, body: string) -> bool",
      "reversibility": "irreversible_write"
    },
    "archive": {
      "signature": "fn(email_id: string) -> bool",
      "reversibility": "reversible_write"
    }
  }
}

Performance Comparison

NSP and vision-based agents are not in the same category. The table below uses real benchmarks from supported JavaScript applications.

Metric	NSP	Vision-Based Agent
State read latency	~50ms	8,000–15,000ms
Action accuracy (JS apps)	>99%	66–82%
50-step task completion	~95%	~0.03%
Actions per minute	~120	~4–7
State verification	Always	Never
Works offline	Yes	Depends on model

How It Works

The NSP daemon runs on the host machine alongside your applications. It has four layers that work together to deliver structured state to your agent.

Process Watcher

Continuously scans all running OS processes, classifies their runtime (V8 / JVM / CLR), and manages probe attachment lifecycle. New processes are detected within 500ms of launch.

Runtime Probes

Language-specific probes attach to live processes and extract semantic state — without source code, UI automation, or any developer instrumentation of the target app.

Semantic Engine

Converts raw runtime data (heap nodes, JVMTI fields, CLR metadata) into structured SSF JSON with stable, human-readable dot-notation keys your agent can reference reliably.

Agent API

A hardened REST and WebSocket server on localhost:7842 that any AI agent can call to read state and execute actions. Auth, rate limiting, and safety validation are built in.

Runtime Coverage

NSP supports the three major managed runtimes used by enterprise and consumer applications today. Support for native C/C++ processes is coming next month.

Runtime	Applications	Semantic Method	Accuracy
JavaScript/V8	Gmail, Slack, VS Code, Notion, GitHub, Salesforce, WhatsApp, Discord, Shopify	V8 Inspector Protocol (CDP) + ASO fast-path	>99%
Java/JVM	SAP, Eclipse, IntelliJ, Jenkins, Elasticsearch, ArduPilot GCS	JVMTI + JNI reflection	>97%
.NET/CLR	Excel, Word, PowerPoint, Visual Studio, Mission Planner, Azure tools	Agent-native MMF + CLR heap walk + JIT hooks	>97%

Next Steps

Quickstart

Up and running in 5 minutes. Install the daemon, launch Chrome, and call your first state endpoint.

Architecture

Deep dive into how the four layers fit together and how each request flows through the system.

Python SDK

The official SDK. pip install nelieo-nsp. Full async support, typed models, and WebSocket streaming.

API Reference

Complete reference for every REST and WebSocket endpoint the daemon exposes.

Get Started

The Daemon

Runtime Probes

Security

Guides

The Problem with Vision-Based Agents

Performance Comparison

How It Works

Process Watcher

Runtime Probes

Semantic Engine

Agent API

Runtime Coverage

Next Steps

Quickstart

Architecture

Python SDK

API Reference

​The Problem with Vision-Based Agents

​Performance Comparison

​How It Works

Process Watcher

Runtime Probes

Semantic Engine

Agent API

​Runtime Coverage

​Next Steps

Quickstart

Architecture

Python SDK

API Reference

The Problem with Vision-Based Agents

Performance Comparison

How It Works

Runtime Coverage

Next Steps