[sdcb]

Switches the inference engine to the shared Sdcb.SimdOnnx (pinned 0.1.0-preview.3, ProGet test feed). The OCR pipeline, public API, preprocessing, CTC head and post-processing stay in this repo; the engine-internal parts move to SimdOnnx: the ONNX interpreter, kernels, the Vulkan/Metal backends and their tests and benches.

Draft. CPU meets the "not slower" bar. Vulkan is now within a few percent of the old backend (noisy, see below).

Performance (interleaved A/B, same machine: Ryzen 9 5950X + Arc B580)

Median ms per image over the 100-image dataset. A ratio below 1 means faster. Stage columns are old/new means.

case	main	this PR	ratio	det	rec	peak WS MB	exact lines
tiny cpu 1w	121.5	116.8	0.96	30.9/28.7	65.8/62.5	500/528	742/742
tiny cpu 4w	64.8	62.0	0.96	30.7/29.0	82.4/76.9	518/559	742/742
tiny cpu 16w	62.0	60.6	0.98	30.6/29.1	142.6/132.8	541/593	742/742
small cpu 1w	423.3	341.4	0.81	99.1/55.4	280.4/246.8	599/666	950/950
small cpu 4w	212.0	148.4	0.70	99.8/52.0	326.1/294.2	650/750	950/950
small cpu 16w	225.0	177.7	0.79	104.0/56.8	636.0/629.1	717/885	950/950
medium cpu 1w	1137.6	1148.2	1.01	234.0/214.2	863.7/875.6	1105/1189	1004/1004
medium cpu 4w	556.3	522.2	0.94	232.0/203.7	1124.0/1046.3	1173/1427	1004/1004
medium cpu 16w	686.9	671.8	0.98	249.0/220.8	2496.3/2548.1	1269/1786	1004/1004
tiny vulkan 4w	20.3	22.0	1.09	4.6/4.9	5.8/6.1	578/664	740/741
small vulkan 4w	37.9	41.3	1.09	8.2/9.0	14.8/17.0	658/785	950/955
medium vulkan 4w	58.6	61.0	1.04	22.2/15.9	27.4/32.3	953/1175	1003/1003

Script: SimdOnnx/tools/ab-paddle.ps1. Absolute speed on this machine drifted by up to 35% between sessions, so compare only within this table. The Vulkan rows are from the last of five interleaved runs. The other four gave ratios of 1.14–1.35 (tiny), 1.09–1.43 (small) and 0.97–1.07 (medium). Stage means are at parity; what remains is per-new-shape setup (plan compile + graph + session).

What closed the Vulkan gap (details in SimdOnnx docs/perf.md):

• per-model GPU const pool, which fixed the memory doubling (each CTC graph used to re-upload the weights);
• two-pass GAP;
• zero-copy staging;
• LRU graph cache;
• here in Paddle: device graphs compile on the thread pool at Reshape/ReshapeMany, and the CTC head runs on the thread pool in batches while the GPU continues.

Device loss now recovers: the failing call finishes on the CPU and the next call gets a fresh device. Fallback is permanent only after 3 losses. Each fallback logs one stderr line.

Breaking / behaviour changes

• OnnxSharp.InferenceSession, the in-repo kernels and Backends.* are gone. They were internal; Model and CompiledModel are kept as thin adapters.
• GPU sessions fall back to CPU permanently after a device error, as before.
• test/Sdcb.SimdPaddleOCR.GpuBench is removed. Removed from the AndroidBench GPU probes: caps, gemm, layers, prof, cmp. Removed engine kernel tests: NhwcKernelTests, the conv/erf/gelu parts of KernelCorrectnessTests, GpuSessionShapeTests and InferenceSessionRunContractTests. Equivalent coverage lives in SimdOnnx.
• WasmBench no longer reports the per-operator breakdown. The JSON keeps the same shape, with empty maps.

Before merge (TODO)

• Vulkan latency. Now 0.97–1.43× depending on the run. Next step is cheaper per-shape setup: fusion passes, compile allocations, or a shape-polymorphic plan.
• Peak working set. Vulkan is +15–23% (medium 953 → 1175 MB; was 1946). CPU is +5–40% at high worker counts.
• Metal. Compiled but not run on Apple hardware in this change. Verify on an M-series Mac.
• Android. AndroidBench was not built locally (no android workload installed).
• NuGet. Publish Sdcb.SimdOnnx to nuget.org, drop the simdonnx-preview source from nuget.config, and bump the pin.
• B580 stability. Validation and GPU-AV are clean, and 46 Vulkan e2e stress runs had no crash. Earlier, one cluster of 3 driver access violations in vkQueueSubmit happened while VS Code was updating, and could not be reproduced. See SimdOnnx docs/decisions.md.

Unit tests: 76/76 on net10 (and on the ns2.0 library via the local engine).