NVIDIA Nsight Python 1.1 attaches the Nsight Compute profiler directly to a running Python process, allowing performance analysis inside Jupyter notebooks and Colab without external command-line tools.
Setup on Colab
The tutorial runs on a free Colab T4 instance. The environment requires Nsight Compute version 2026.2.1 or newer, but the default image ships an older build. The script updates the package index and installs version 2026.3.1 via the NVIDIA CUDA apt repository. It then queries dpkg to locate the ncu executable and prepends its directory to the system PATH.
Installation of the Python library pins the commit bb96db4260b1823d86ecab9acfed0646d7af79ad to ensure the 1.1.0 pre-release features are present. The code installs the cupti-python package into a separate directory because the library requires a newer libcupti version than the one bundled with PyTorch.
Import order is critical. The script imports nsight before importing PyTorch. This loads the injection library into the current process, ensuring it can intercept CUDA work initiated after that point.
Profiling a single kernel
The first benchmark measures a 2048 x 2048 matrix multiplication in fp32. The code creates two random tensors and wraps only the multiplication in nsight.annotate. This isolates the kernel launch from the random number generation overhead.
A decorator named nsight.analyze.kernel runs with thermal_mode="off" and a custom metric aggregator. Calling the function triggers ncu in attach mode on the notebook process. The result is a Pandas DataFrame containing one row per annotation, configuration, and metric.
The output prints the kernel name, the average time duration in nanoseconds, the number of runs, and the GPU clock speeds. It calculates performance as TFLOP/s using the formula: 2 * 2048^3 / (time * 1e-9) / 1e12. The stability flag shows as undefined because only a single run occurred.
Runs and stability
The next section tests vector addition and matrix multiplication across two configurations: batch sizes of 16384 and 16777216. The benchmark runs 20 iterations for each setting.
The function show formats the resulting DataFrame for readability, scaling the time columns by milliseconds. The output displays the metric values alongside the configuration parameters.
What it means
Developers gain a direct way to inspect GPU kernel performance from within standard Python workflows. The approach removes the need to switch contexts between a running script and a separate profiler window. Metrics like clock speeds and memory bandwidth become accessible as structured data, allowing immediate calculation of throughput and identification of compute-bound versus memory-bound workloads.
