TensorRT-LLM: rejected as a runtime, not as a kernel source
research
runtime
kernels
Its kernels are open and buildable for sm_121, so the kernel question is a benchmark, not an assumption. The runtime still loses on what it contributes.
Question. Same question as the superseded entry, reopened because that answer used precedent as evidence. An engine premised on the easy roads already being travelled cannot reject its dependencies for being unsupported.
| setup | |
|---|---|
| node | head |
| target | GB10, compute capability 12.1 (sm_121) |
| method | source and packaging inspection, nothing installed |
What was wrong:
| earlier objection | actual status |
|---|---|
| not in the model zoo | work, not a barrier. Plugins and custom definitions exist. |
sm_121 unproven |
a build flag: build_wheel.py --cuda_architectures=121-real |
| closed kernels | false. 167 open entries under cpp/tensorrt_llm/kernels; TensorRT the graph library is the closed piece, not TensorRT-LLM |
What survives. We write KDA, DSA and the 4-bit grouped expert GEMM either way, so:
| it offers | we would |
|---|---|
| tuned kernels for supported layers | write our hot layers regardless |
| batch manager and scheduler | replace it (scheduler) |
| graph runtime | shapes are static; CUDA graphs over our own launcher is small |
| KV cache manager | replace it. The atomic (KV, KDA) entry is the point |
Verdict. Runtime rejected on contribution, not difficulty. Kernels explicitly not rejected.
Next. Open experiment: build from source with --cuda_architectures=121-real, microbenchmark its NVFP4 grouped-expert GEMM and sparse-MLA kernels against ours on GB10, link or port whatever wins. Consuming a kernel does not require adopting a runtime.
Reopen if.
- that benchmark shows its grouped GEMM ahead of ours on
sm_121