perf-arm-spe - Man Page

perf record -e arm_spe//

The SPE (Statistical Profiling Extension) feature provides accurate attribution of latencies and events down to individual instructions. Rather than being interrupt-driven, it picks an instruction to sample and then captures data for it during execution. Data includes execution time in cycles. For loads and stores it also includes data address, cache miss events, and data origin.

The sampling has 5 stages:

1. Choose an operation
2. Collect data about the operation
3. Optionally discard the record based on a filter
4. Write the record to memory
5. Interrupt when the buffer is full

Up until this point no decoding of the SPE data was done by either the kernel or Perf. Only when the recorded file is opened with perf report or perf script does the decoding happen. When decoding the data, Perf generates "synthetic samples" as if these were generated at the time of the recording. These samples are the same as if normal sampling was done by Perf without using SPE, although they may have more attributes associated with them. For example a normal sample may have just the instruction pointer, but an SPE sample can have data addresses and latency attributes.

• Sampling, rather than tracing, cuts down the profiling problem to something more manageable for hardware. Only one sampled operation is in flight at a time.
• Allows precise attribution data, including: Full PC of instruction, data virtual and physical addresses.
• Allows correlation between an instruction and events, such as TLB and cache miss. (Data source indicates which particular cache was hit, but the meaning is implementation defined because different implementations can have different cache configurations.)

However, SPE does not provide any call-graph information, and relies on statistical methods.

When an operation is sampled while a previous sampled operation has not finished, a collision occurs. The new sample is dropped. Collisions affect the integrity of the data, so the sample rate should be set to avoid collisions.

The sample_collision PMU event can be used to determine the number of lost samples. Although this count is based on collisions before filtering occurs. Therefore this can not be used as an exact number for samples dropped that would have made it through the filter, but can be a rough guide.

If an implementation samples micro-operations instead of instructions, the results of sampling must be weighted accordingly.

For example, if a given instruction A is always converted into two micro-operations, A0 and A1, it becomes twice as likely to appear in the sample population.

The coarse effect of conversions, and, if applicable, sampling of speculative operations, can be estimated from the sample_pop and inst_retired PMU events.

The ARM_SPE_PMU config must be set to build as either a module or statically.

Depending on CPU model, the kernel may need to be booted with page table isolation disabled (kpti=off). If KPTI needs to be disabled, this will fail with a console message "profiling buffer inaccessible. Try passing kpti=off on the kernel command line".

For the full criteria that determine whether KPTI needs to be forced off or not, see function unmap_kernel_at_el0() in the kernel sources. Common cases where it’s not required are on the CPUs in kpti_safe_list, or on Arm v8.5+ where FEAT_E0PD is mandatory.

The SPE interrupt must also be described by the firmware. If the module is loaded and KPTI is disabled (or isn’t required to be disabled) but the SPE PMU still doesn’t show in /sys/bus/event_source/devices/, then it’s possible that the SPE interrupt isn’t described by ACPI or DT. In this case no warning will be printed by the driver.

You can record a session with SPE samples:

perf record -e arm_spe// -- ./mybench

The sample period is set from the -c option, and because the minimum interval is used by default it’s recommended to set this to a higher value. The value is written to PMSIRR.INTERVAL.

branch_filter=1     - collect branches only (PMSFCR.B)
event_filter=<mask> - filter on specific events (PMSEVFR) - see bitfield description below
jitter=1            - use jitter to avoid resonance when sampling (PMSIRR.RND)
load_filter=1       - collect loads only (PMSFCR.LD)
min_latency=<n>     - collect only samples with this latency or higher* (PMSLATFR)
pa_enable=1         - collect physical address (as well as VA) of loads/stores (PMSCR.PA) - requires privilege
pct_enable=1        - collect physical timestamp instead of virtual timestamp (PMSCR.PCT) - requires privilege
store_filter=1      - collect stores only (PMSFCR.ST)
ts_enable=1         - enable timestamping with value of generic timer (PMSCR.TS)

bit 1     - instruction retired (i.e. omit speculative instructions)
bit 3     - L1D refill
bit 5     - TLB refill
bit 7     - mispredict
bit 11    - misaligned access

perf record -e arm_spe/event_filter=2/ -- ./mybench

perf record -e arm_spe/event_filter=0x80/ -- ./mybench

Available samples
0 arm_spe//
0 dummy:u
21 l1d-miss
897 l1d-access
5 llc-miss
7 llc-access
2 tlb-miss
1K tlb-access
36 branch-miss
0 remote-access
900 memory

perf report --itrace=i1i

perf report --mem-mode

perf-record(1), perf-script(1), perf-report(1), perf-inject(1)

perf(1), perf-c2c(1), perf-mem(1).