Native / C++ Probe: Memory Reading and Semantic Labeling

The Native probe handles processes that do not use a managed runtime — C and C++ applications, Rust binaries, Go programs, and any process that runs without a JVM, CLR, or V8 engine. It reads process memory directly using ptrace on Linux or ReadProcessMemory on Windows, then applies Visual-Memory Grounding to assign human-readable semantic labels to the raw memory values it finds. This is the most technically challenging probe in NSP’s suite, and it produces lower confidence than managed-runtime probes — but it unlocks state extraction for applications that no other probe can reach.

Technical Approach

Native probing is harder than managed runtimes because there is no reflection metadata to lean on. No type names, field names, or method signatures are stored in the binary at runtime (unless debug symbols are present). Memory layout is determined by the compiler, optimization flags, and application version. NSP solves this with a combination of symbol resolution and visual correlation:

Process memory scan
  │  ReadProcessMemory (Windows) / ptrace (Linux)
  ▼
Symbol resolution
  │  PDB (Windows) or DWARF (Linux) debug symbols, if available
  ▼
Visual-Memory Grounding
  ├─ Screenshot capture → Vision model → semantic UI regions
  ├─ Memory pattern matching → candidate values at candidate addresses
  └─ Cross-correlation → semantic labels for matched regions
  │
  ▼
SubstrateState (confidence 0.40–0.75)

Symbol Resolution

When the target process ships public debug symbols (PDB files on Windows, DWARF sections on Linux), NSP loads them to resolve type names and field offsets directly. This significantly improves both coverage and confidence. When symbols are stripped — as is common in production builds — NSP falls back entirely to Visual-Memory Grounding.

Visual-Memory Grounding

Visual-Memory Grounding is NSP’s technique for labeling raw memory without reflection metadata:

Screenshot capture — A screenshot of the application window is taken.
Vision model inference — A vision model segments the screenshot into semantic regions (e.g. “price field”, “account balance”, “instrument panel”).
Memory pattern matching — The probe scans process memory for values consistent with the numbers or strings visible in each region (e.g. a float matching the displayed price to within floating-point rounding).
Cross-correlation — Candidate memory addresses are cross-correlated with visual regions to assign semantic labels. Surviving matches are written as dot-notation SSF keys.

Visual-Memory Grounding requires that the application window is visible on screen during the cold probe. Minimized or off-screen windows will produce fewer labeled fields and lower confidence scores.

Supported Applications

Native probing is best-effort. Applications with public debug symbols achieve the highest coverage. The following have validated entries in the app signature database:

Application	Symbol Availability	Coverage
Bloomberg Terminal	Partial (public PDB)	Moderate — key financial data fields, price feeds
Adobe Photoshop	Stripped	Moderate — document state, active tool, layer count
AutoCAD	Stripped	Moderate — drawing state, active command, object count
Chrome (C++ layer)	Partial (public symbols)	Good — combined with V8 probe for full coverage

For Electron and Chrome applications, NSP runs both the V8 probe (for JavaScript state) and the Native probe (for C++ layer state) simultaneously. The results are merged into a single SubstrateState. The V8 confidence score dominates, so the merged state retains the >99% accuracy of the JavaScript layer while also surfacing C++ process-level fields.

Confidence Range and Action Gating

The Native probe produces a confidence score in the range 0.40–0.75 under normal conditions. This is lower than managed-runtime probes because semantic labeling is probabilistic rather than derived from verified reflection metadata.

Scenario	Confidence Range
Public PDB / DWARF symbols available	0.60–0.75
Stripped symbols (typical production build)	0.40–0.60
Obfuscated binary or custom allocators	0.30–0.50

These confidence values map directly to NSP’s action gating tiers:

Confidence	Action Gating
`0.40–0.79`	Read-only actions only
`0.80–0.94`	`reversible_write` actions (rarely reached for native apps)
`0.95–1.00`	`irreversible_write` actions (not achievable for native apps)

Because native app confidence sits in the 0.40–0.75 range, only read actions are allowed by default. Write actions are gated until confidence exceeds 0.80, which requires debug symbols and a well-correlated Visual-Memory Grounding pass. Plan your agent workflows around read operations for native targets, and use write actions only after verifying the confidence score in SubstrateState.confidence.

Confidence in SubstrateState

A typical SubstrateState from a native app will look like this:

{
  "app_name": "Bloomberg Terminal",
  "runtime": "native",
  "confidence": 0.61,
  "objects": {
    "market.AAPL.last_price": 213.47,
    "market.AAPL.bid": 213.45,
    "market.AAPL.ask": 213.49,
    "market.AAPL.volume": 4821093,
    "session.is_authenticated": true,
    "session.user_id": "jdoe@example.com"
  },
  "probe_metadata": {
    "capture_latency_ms": 180,
    "total_latency_ms": 310,
    "symbol_resolution": "partial_pdb",
    "vmg_regions_matched": 14,
    "vmg_regions_total": 22
  }
}

The vmg_regions_matched and vmg_regions_total fields in probe_metadata tell you how many visual regions were successfully correlated to memory addresses. A higher ratio produces higher confidence.

Python SDK Example

import asyncio
from nelieo_nsp import NSPClient

async def main():
    async with NSPClient() as client:
        session = await client.session("bloomberg-terminal-8821")
        state = await session.state()

        # Check confidence before acting
        if state.confidence < 0.40:
            print("Probe still initializing — waiting for state")
            return

        # Read financial data (read actions always allowed at >= 0.40)
        last_price = state.get_float("market.AAPL.last_price")
        volume = state.get_int("market.AAPL.volume")
        print(f"AAPL: ${last_price:.2f} | Vol: {volume:,}")

asyncio.run(main())

Subscribe to state-change events via WebSocket for continuous native app monitoring. The native probe fires updates whenever Visual-Memory Grounding detects a value change in a labeled memory region — typically within 50–500ms of the UI updating.

Limitations

Why is native confidence lower than managed runtimes?

Managed runtimes (V8, JVM, CLR) retain reflection metadata — type names, field names, and method signatures — that allow NSP to label state with certainty. Native binaries strip this information at compile time. Visual-Memory Grounding assigns labels probabilistically based on visual correlation, which is accurate but not provably correct the way reflection is. The confidence score reflects this fundamental difference.

Why are write actions blocked at low confidence?

The action gating system is a safety contract. At confidence below 0.80, NSP cannot be certain that a labeled field is what it appears to be. Allowing write actions on misidentified fields could cause silent data corruption or unintended application behavior. Read actions are safe at any confidence level — reading a misidentified field produces a wrong value, but causes no side effects.

Can I improve confidence for a specific native app?

Yes. If public debug symbols are available (PDB or DWARF), place them in the path specified by axon.toml under [native] symbol_paths. NSP will load them automatically on the next cold probe. Additionally, ensuring the application window is fully visible and not occluded during the cold probe maximizes the Visual-Memory Grounding match rate.

Does the native probe support 32-bit processes?

Yes. Both 32-bit and 64-bit native processes are supported on Windows. On Linux, ptrace works for both process address space widths. The probe automatically adjusts pointer sizes and memory alignment for the target process architecture.

Get Started

The Daemon

Runtime Probes

Security

Guides

Technical Approach

Symbol Resolution

Visual-Memory Grounding

Supported Applications

Confidence Range and Action Gating

Confidence in SubstrateState

Python SDK Example

Limitations

​Technical Approach

​Symbol Resolution

​Visual-Memory Grounding

​Supported Applications

​Confidence Range and Action Gating

​Confidence in SubstrateState

​Python SDK Example

​Limitations

Technical Approach

Symbol Resolution

Visual-Memory Grounding

Supported Applications

Confidence Range and Action Gating

Confidence in SubstrateState

Python SDK Example

Limitations