GGML Scope streams the activations of an inference. While the stream is open, every llama_decode walks the compute graph node by node through the backend scheduler eval callback; each node output tensor is read back and reduced to statistics. One frame is emitted per decode over Server Sent Events, frame i is the graph that produced token i.
Frame JSON { seq, n_nodes, n_leafs, t_capture_ms, nodes }. A node is { name, op, type, shape, buffer, src, stats } where src holds positions in nodes, topologically ordered. Tensors only ever seen as inputs are leaf entries tagged leaf (weight, kv, input, uncaptured); trivial data movement carries plumbing: true. Stats per node, over the finite values, accumulated in double: n, n_nan, n_inf, n_zero, min, max, mean, rms.
RMS = sqrt( Σ vᵢ² / n_finite ) over every element of the tensor, walked through the strides so permuted views read correctly, NaN and Inf excluded from both the sum and the count.
RMS Delta = rms(node, t) − rms(node, t−1), paired by node name across consecutive decodes. The bar is signed and centered on zero, scaled by the largest absolute delta of the frame: right means the activation grew on this token, left means it shrank. The RMS column keeps the absolute value beside it.
Range = the [min, max] interval of the tensor on a symmetric log axis centered on zero, with a tick at the mean. Outlier hunting: a max running out alone while min and mean stay near zero is a massive activation.
Attention internals (KQ, softmax) only appear with -fa off, flash attention fuses them into a single node.
Enter a prompt and press Start.