This page describes the software Proof of Concept (PoC) for the ACPI RQSC table which describes QoS controllers in the system that implement the Capacity and Bandwidth QoS Register Interface (CBQRI).

This PoC builds upon the original CBQRI PoC. Linux now supports CBQRI controller information passed through Device Tree or ACPI. The choice depends on how Qemu is invoked - see below for details.

Resources

ACPI RQSC table spec

https://github.com/riscv-non-isa/riscv-rqsc/tree/main/src

The Capacity and Bandwidth QoS controllers in a system are described by the ACPI RQSC table. The actual register interface is specified in the RISC-V Capacity and Bandwidth Controller QoS Register Interface specification cite:[CBQRI].
The RQSC table describes the properties of the QoS controllers in the system. The table also describes the topological arrangement of the QoS controllers and resources in the system. The topology is expressed in terms of the location of the resources within the system and the relation between teh QoS Controller and the resource it manages.

RVI PRS meeting

RQSC slides from RVI Platform Runtime Services (PRS) TG meeting on March 27, 2025.

Linux

Repo: https://github.com/tt-fustini/linux
Branch: dfustini/cbqri-mpam-snapshot-v6.14-rc1
- Based on the James Morse MPAM support snapshot for 6.14: mpam/snapshot/v6.14-rc1
  - Update: Some of the patches to decouple resctrl from arch/x86 got merged for 6.15
- Note: riscv defconfig works okay but a smaller subset of drivers to reduce build time are enabled in the downstream linux.config

Qemu

Repo: GitHub - tt-fustini/qemu
Branch: riscv-cbqri-rqsc-pptt
Build and run: ./build.sh && ./run-acpi.sh
- TODO: add note about dependencies to build Qemu and EDK2

run-acpi.sh

#!/bin/bash
export LX=$HOME/dev/linux
export BR=$HOME/dev/images
export EDK=$HOME/dev/edk2
build/qemu-system-riscv64 \
	-M virt,pflash0=pflash0,pflash1=pflash1,aia=aplic-imsic  \
	-nographic \
	-m 1G \
	-smp cpus=8,sockets=1,clusters=2,cores=4,threads=1 \
	-serial mon:stdio \
	-kernel ${LX}/arch/riscv/boot/Image \
	-append "root=/dev/vda ro loglevel=8 ro console=ttyS0 rootwait earlycon=uart8250,mmio,0x10000000 acpi.debug_layer=0xffffffff acpi.debug_level=0xffffffff" \
	-blockdev node-name=pflash0,driver=file,read-only=on,filename=$EDK/RISCV_VIRT_CODE.fd \
	-blockdev node-name=pflash1,driver=file,filename=$EDK/RISCV_VIRT_VARS.fd \
	-drive if=none,file=${BR}/rootfs.ext2,format=raw,id=hd0 \
	-device virtio-blk-device,drive=hd0 \
	-device qemu-xhci \
	-device usb-kbd \
	-device virtio-net-pci,netdev=net0 \
	-netdev user,id=net0 \
	-device riscv.cbqri.capacity,max_mcids=256,max_rcids=64,ncblks=12,alloc_op_flush_rcid=false,mon_op_config_event=false,mon_op_read_counter=false,mon_evt_id_none=false,mon_evt_id_occupancy=false,mmio_base=0x04820000 \
	-device riscv.cbqri.capacity,max_mcids=256,max_rcids=64,ncblks=12,alloc_op_flush_rcid=false,mon_op_config_event=false,mon_op_read_counter=false,mon_evt_id_none=false,mon_evt_id_occupancy=false,mmio_base=0x04821000 \
	-device riscv.cbqri.capacity,max_mcids=256,max_rcids=64,ncblks=16,mmio_base=0x0482B000 \
	-device riscv.cbqri.bandwidth,max_mcids=256,max_rcids=64,nbwblks=1024,mrbwb=819,mmio_base=0x04828000 \
	-device riscv.cbqri.bandwidth,max_mcids=256,max_rcids=64,nbwblks=1024,mrbwb=819,mmio_base=0x04829000 \
	-device riscv.cbqri.bandwidth,max_mcids=256,max_rcids=64,nbwblks=1024,mrbwb=819,mmio_base=0x0482a000 \
	#-numa node,cpus=0-3,nodeid=0,mem=512M -numa node,cpus=4-7,nodeid=1,mem=512M \

PPTT table in Qemu

Dump and disassembled PPTT table: https://gist.github.com/tt-fustini/70ac54ed6ef13c8baf4462d7ca8f6fd4
PoC based on last PPTT series from Sia Jee Heng (StarFive)
Proper upstream solution would be to collaborate on ARM patches

RQSC table in Qemu

Dump and disassembled RQSC table: https://gist.github.com/tt-fustini/cd4b89a52c72debb4308bee68b580cce

Cache ID

Qemu uses the CBQRI controller's mmio_base address as the Cache ID in both the PPTT and RQSC tables for cache controllers. A real platform would probably do it differently but this is a simple way to ensure the Cache ID is unique.

The Linux PPTT driver is now able to populate the cache information for each cpu.
Example of cpu0 (cluster 0) and cpu4 (cluster 1). They have different instances of the L2 cache controller (note: sysfs prints the Cache ID in decimal).

 for i in `find /sys/devices/system/cpu/cpu0/cache/index* -type f | grep -E '/(type|size|id|level)'`; do echo "$i: `cat $i`"; done
/sys/devices/system/cpu/cpu0/cache/index0/id: 29
/sys/devices/system/cpu/cpu0/cache/index0/type: Data
/sys/devices/system/cpu/cpu0/cache/index0/size: 64K
/sys/devices/system/cpu/cpu0/cache/index0/level: 1
/sys/devices/system/cpu/cpu0/cache/index1/id: 30
/sys/devices/system/cpu/cpu0/cache/index1/type: Instruction
/sys/devices/system/cpu/cpu0/cache/index1/size: 64K
/sys/devices/system/cpu/cpu0/cache/index1/level: 1
/sys/devices/system/cpu/cpu0/cache/index2/id: 75632640
/sys/devices/system/cpu/cpu0/cache/index2/type: Unified
/sys/devices/system/cpu/cpu0/cache/index2/size: 750K
/sys/devices/system/cpu/cpu0/cache/index2/level: 2
/sys/devices/system/cpu/cpu0/cache/index3/id: 75673600
/sys/devices/system/cpu/cpu0/cache/index3/type: Unified
/sys/devices/system/cpu/cpu0/cache/index3/size: 3072K
/sys/devices/system/cpu/cpu0/cache/index3/level: 3
# for i in `find /sys/devices/system/cpu/cpu4/cache/index* -type f | grep -E '/(type|size|id|level)'`; do echo "$i: `cat $i`"; done
/sys/devices/system/cpu/cpu4/cache/index0/id: 29
/sys/devices/system/cpu/cpu4/cache/index0/type: Data
/sys/devices/system/cpu/cpu4/cache/index0/size: 64K
/sys/devices/system/cpu/cpu4/cache/index0/level: 1
/sys/devices/system/cpu/cpu4/cache/index1/id: 30
/sys/devices/system/cpu/cpu4/cache/index1/type: Instruction
/sys/devices/system/cpu/cpu4/cache/index1/size: 64K
/sys/devices/system/cpu/cpu4/cache/index1/level: 1
/sys/devices/system/cpu/cpu4/cache/index2/id: 75628544
/sys/devices/system/cpu/cpu4/cache/index2/type: Unified
/sys/devices/system/cpu/cpu4/cache/index2/size: 750K
/sys/devices/system/cpu/cpu4/cache/index2/level: 2
/sys/devices/system/cpu/cpu4/cache/index3/id: 75673600
/sys/devices/system/cpu/cpu4/cache/index3/type: Unified
/sys/devices/system/cpu/cpu4/cache/index3/size: 3072K
/sys/devices/system/cpu/cpu4/cache/index3/level: 3

resctrl demo

The full resctrl user interface is documented in the kernel

https://docs.kernel.org/arch/x86/resctrl.html

Example commands to mount and modify the schemata:

mount -t resctrl resctrl /sys/fs/resctrl

ls -la /sys/fs/resctrl/

cd /sys/fs/resctrl/

cat /sys/fs/resctrl/schemata

echo 'L2:4=ff0' >  /sys/fs/resctrl/schemata

cat /sys/fs/resctrl/schemata

Running in Qemu:

# mount -t resctrl resctrl /sys/fs/resctrl
# cd /sys/fs/resctrl/
# cat /sys/fs/resctrl/schemata
MB:0=0;1=0;2=0
L2:4=0;5=0
L3:3=0
# echo 'L2:4=ff0' >  /sys/fs/resctrl/schemata
# echo 'L2:5=fff' >  /sys/fs/resctrl/schemata
# echo 'L2:5=0ff' >  /sys/fs/resctrl/schemata
# cat /sys/fs/resctrl/schemata
MB:0=0;1=0;2=0
L2:4=ff0;5=ff
L3:3=0
# echo 'L3:3=fff' >  /sys/fs/resctrl/schemata
# echo 'L3:3=ff0' >  /sys/fs/resctrl/schemata
# cat /sys/fs/resctrl/schemata
MB:0=0;1=0;2=0
L2:4=ff0;5=ff
L3:3=ff0
# mkdir p1
# cd p1
# echo $$ > tasks
# cat schemata 
MB:0=100;1=100;2=100
L2:4=fff;5=fff
L3:3=ffff
# cat ../schemata 
MB:0=0;1=0;2=0
L2:4=ff0;5=ff
L3:3=ff0

Boot log

pdp7@thelio:~/dev/qemu$ ./build.sh && ./run-acpi.sh
# mount -t resctrl resctrl /sys/fs/resctrl
# cd /sys/fs/resctrl/
# ls -la
total 0
dr-xr-xr-x    5 root     root             0 Jan  1 00:06 .
drwxr-xr-x    9 root     root             0 Jan  1 00:06 ..
-rw-r--r--    1 root     root             0 Jan  1 00:06 cpus
-rw-r--r--    1 root     root             0 Jan  1 00:06 cpus_list
dr-xr-xr-x    6 root     root             0 Jan  1 00:06 info
-rw-r--r--    1 root     root             0 Jan  1 00:06 mode
dr-xr-xr-x    2 root     root             0 Jan  1 00:06 mon_data
dr-xr-xr-x    2 root     root             0 Jan  1 00:06 mon_groups
-rw-r--r--    1 root     root             0 Jan  1 00:06 schemata
-r--r--r--    1 root     root             0 Jan  1 00:06 size
-rw-r--r--    1 root     root             0 Jan  1 00:06 tasks
# cat /sys/fs/resctrl/schemata
MB:0=0;1=0;2=0
L2:4=0;5=0
L3:3=0
# echo 'L2:4=ff0' >  /sys/fs/resctrl/schemata
# echo 'L2:5=fff' >  /sys/fs/resctrl/schemata
# echo 'L2:5=0ff' >  /sys/fs/resctrl/schemata
# cat /sys/fs/resctrl/schemata
MB:0=0;1=0;2=0
L2:4=ff0;5=ff
L3:3=0
# echo 'L3:3=fff' >  /sys/fs/resctrl/schemata
# echo 'L3:3=ff0' >  /sys/fs/resctrl/schemata
# cat /sys/fs/resctrl/schemata
MB:0=0;1=0;2=0
L2:4=ff0;5=ff
L3:3=ff0
# mkdir p1
# cd p1
# echo $$ > tasks
# ls -la
total 0
drwxr-xr-x    4 root     root             0 Jan  1 00:07 .
dr-xr-xr-x    6 root     root             0 Jan  1 00:06 ..
-rw-r--r--    1 root     root             0 Jan  1 00:07 cpus
-rw-r--r--    1 root     root             0 Jan  1 00:07 cpus_list
-rw-r--r--    1 root     root             0 Jan  1 00:07 mode
drwxr-xr-x    2 root     root             0 Jan  1 00:07 mon_data
drwxr-xr-x    2 root     root             0 Jan  1 00:07 mon_groups
-rw-r--r--    1 root     root             0 Jan  1 00:07 schemata
-r--r--r--    1 root     root             0 Jan  1 00:07 size
-rw-r--r--    1 root     root             0 Jan  1 00:07 tasks
# pwd
/sys/fs/resctrl/p1
# cat schemata 
MB:0=100;1=100;2=100
L2:4=fff;5=fff
L3:3=ffff
# cat ../schemata 
MB:0=0;1=0;2=0
L2:4=ff0;5=ff
L3:3=ff0
#