> ## Documentation Index
> Fetch the complete documentation index at: https://docs.nelieo.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Native / C++ Probe: Memory Reading and Semantic Labeling

> How NSP reads state from native C/C++ apps via ptrace or ReadProcessMemory, then applies Visual-Memory Grounding to assign semantic labels to raw memory.

The Native probe handles processes that do not use a managed runtime — C and C++ applications, Rust binaries, Go programs, and any process that runs without a JVM, CLR, or V8 engine. It reads process memory directly using `ptrace` on Linux or `ReadProcessMemory` on Windows, then applies **Visual-Memory Grounding** to assign human-readable semantic labels to the raw memory values it finds. This is the most technically challenging probe in NSP's suite, and it produces lower confidence than managed-runtime probes — but it unlocks state extraction for applications that no other probe can reach.

***

## Technical Approach

Native probing is harder than managed runtimes because there is no reflection metadata to lean on. No type names, field names, or method signatures are stored in the binary at runtime (unless debug symbols are present). Memory layout is determined by the compiler, optimization flags, and application version. NSP solves this with a combination of symbol resolution and visual correlation:

```text theme={null}
Process memory scan
  │  ReadProcessMemory (Windows) / ptrace (Linux)
  ▼
Symbol resolution
  │  PDB (Windows) or DWARF (Linux) debug symbols, if available
  ▼
Visual-Memory Grounding
  ├─ Screenshot capture → Vision model → semantic UI regions
  ├─ Memory pattern matching → candidate values at candidate addresses
  └─ Cross-correlation → semantic labels for matched regions
  │
  ▼
SubstrateState (confidence 0.40–0.75)
```

### Symbol Resolution

When the target process ships public debug symbols (PDB files on Windows, DWARF sections on Linux), NSP loads them to resolve type names and field offsets directly. This significantly improves both coverage and confidence. When symbols are stripped — as is common in production builds — NSP falls back entirely to Visual-Memory Grounding.

### Visual-Memory Grounding

Visual-Memory Grounding is NSP's technique for labeling raw memory without reflection metadata:

1. **Screenshot capture** — A screenshot of the application window is taken.
2. **Vision model inference** — A vision model segments the screenshot into semantic regions (e.g. "price field", "account balance", "instrument panel").
3. **Memory pattern matching** — The probe scans process memory for values consistent with the numbers or strings visible in each region (e.g. a float matching the displayed price to within floating-point rounding).
4. **Cross-correlation** — Candidate memory addresses are cross-correlated with visual regions to assign semantic labels. Surviving matches are written as dot-notation SSF keys.

<Warning>
  Visual-Memory Grounding requires that the application window is visible on screen during the cold probe. Minimized or off-screen windows will produce fewer labeled fields and lower confidence scores.
</Warning>

***

## Supported Applications

Native probing is best-effort. Applications with public debug symbols achieve the highest coverage. The following have validated entries in the app signature database:

| Application        | Symbol Availability      | Coverage                                               |
| ------------------ | ------------------------ | ------------------------------------------------------ |
| Bloomberg Terminal | Partial (public PDB)     | Moderate — key financial data fields, price feeds      |
| Adobe Photoshop    | Stripped                 | Moderate — document state, active tool, layer count    |
| AutoCAD            | Stripped                 | Moderate — drawing state, active command, object count |
| Chrome (C++ layer) | Partial (public symbols) | Good — combined with V8 probe for full coverage        |

<Info>
  For Electron and Chrome applications, NSP runs **both** the V8 probe (for JavaScript state) and the Native probe (for C++ layer state) simultaneously. The results are merged into a single `SubstrateState`. The V8 confidence score dominates, so the merged state retains the >99% accuracy of the JavaScript layer while also surfacing C++ process-level fields.
</Info>

***

## Confidence Range and Action Gating

The Native probe produces a `confidence` score in the range **0.40–0.75** under normal conditions. This is lower than managed-runtime probes because semantic labeling is probabilistic rather than derived from verified reflection metadata.

| Scenario                                    | Confidence Range |
| ------------------------------------------- | ---------------- |
| Public PDB / DWARF symbols available        | 0.60–0.75        |
| Stripped symbols (typical production build) | 0.40–0.60        |
| Obfuscated binary or custom allocators      | 0.30–0.50        |

These confidence values map directly to NSP's action gating tiers:

| Confidence  | Action Gating                                                 |
| ----------- | ------------------------------------------------------------- |
| `0.40–0.79` | Read-only actions only                                        |
| `0.80–0.94` | `reversible_write` actions (rarely reached for native apps)   |
| `0.95–1.00` | `irreversible_write` actions (not achievable for native apps) |

<Note>
  Because native app confidence sits in the **0.40–0.75** range, only **read actions** are allowed by default. Write actions are gated until confidence exceeds 0.80, which requires debug symbols and a well-correlated Visual-Memory Grounding pass. Plan your agent workflows around read operations for native targets, and use write actions only after verifying the confidence score in `SubstrateState.confidence`.
</Note>

***

## Confidence in SubstrateState

A typical `SubstrateState` from a native app will look like this:

```json theme={null}
{
  "app_name": "Bloomberg Terminal",
  "runtime": "native",
  "confidence": 0.61,
  "objects": {
    "market.AAPL.last_price": 213.47,
    "market.AAPL.bid": 213.45,
    "market.AAPL.ask": 213.49,
    "market.AAPL.volume": 4821093,
    "session.is_authenticated": true,
    "session.user_id": "jdoe@example.com"
  },
  "probe_metadata": {
    "capture_latency_ms": 180,
    "total_latency_ms": 310,
    "symbol_resolution": "partial_pdb",
    "vmg_regions_matched": 14,
    "vmg_regions_total": 22
  }
}
```

The `vmg_regions_matched` and `vmg_regions_total` fields in `probe_metadata` tell you how many visual regions were successfully correlated to memory addresses. A higher ratio produces higher confidence.

***

## Python SDK Example

```python theme={null}
import asyncio
from nelieo_nsp import NSPClient

async def main():
    async with NSPClient() as client:
        session = await client.session("bloomberg-terminal-8821")
        state = await session.state()

        # Check confidence before acting
        if state.confidence < 0.40:
            print("Probe still initializing — waiting for state")
            return

        # Read financial data (read actions always allowed at >= 0.40)
        last_price = state.get_float("market.AAPL.last_price")
        volume = state.get_int("market.AAPL.volume")
        print(f"AAPL: ${last_price:.2f} | Vol: {volume:,}")

asyncio.run(main())
```

<Tip>
  Subscribe to state-change events via WebSocket for continuous native app monitoring. The native probe fires updates whenever Visual-Memory Grounding detects a value change in a labeled memory region — typically within 50–500ms of the UI updating.
</Tip>

***

## Limitations

<AccordionGroup>
  <Accordion title="Why is native confidence lower than managed runtimes?">
    Managed runtimes (V8, JVM, CLR) retain reflection metadata — type names, field names, and method signatures — that allow NSP to label state with certainty. Native binaries strip this information at compile time. Visual-Memory Grounding assigns labels probabilistically based on visual correlation, which is accurate but not provably correct the way reflection is. The confidence score reflects this fundamental difference.
  </Accordion>

  <Accordion title="Why are write actions blocked at low confidence?">
    The action gating system is a safety contract. At confidence below 0.80, NSP cannot be certain that a labeled field is what it appears to be. Allowing write actions on misidentified fields could cause silent data corruption or unintended application behavior. Read actions are safe at any confidence level — reading a misidentified field produces a wrong value, but causes no side effects.
  </Accordion>

  <Accordion title="Can I improve confidence for a specific native app?">
    Yes. If public debug symbols are available (PDB or DWARF), place them in the path specified by `axon.toml` under `[native] symbol_paths`. NSP will load them automatically on the next cold probe. Additionally, ensuring the application window is fully visible and not occluded during the cold probe maximizes the Visual-Memory Grounding match rate.
  </Accordion>

  <Accordion title="Does the native probe support 32-bit processes?">
    Yes. Both 32-bit and 64-bit native processes are supported on Windows. On Linux, ptrace works for both process address space widths. The probe automatically adjusts pointer sizes and memory alignment for the target process architecture.
  </Accordion>
</AccordionGroup>