Modding cpu affinity improves cake SQM significantly. But...

[StarWhiz]

[EDIT] This problem was solved please see my second post below this one

I wrote this script to optimize CPU affinity so I that I could do cake SQM with the R6S on my 1400Mbps connection. I am open to suggestions on improving the script. Before the cpu affinity optimization I could only push up to 800 Mbps with Cake SQM turned on. After, I could do 1400Mbps easily which is close the the max my ISP provides.

This script below is saved in /root/performancetweak.sh

Code: Select all

#!/bin/bash

# Save the output of /proc/interrupts in a variable
interrupts=$(cat /proc/interrupts)

# Extract the numbers associated with eth1-0 and eth2-0 using grep and awk
eth1_0=$(echo "$interrupts" | grep "eth1-0" | awk '{print $1}' | tr -d ':')
eth2_0=$(echo "$interrupts" | grep "eth2-0" | awk '{print $1}' | tr -d ':')

# Display current CPU cores assigned to current IRQs and queues
echo "CPU Affinity for ETH1 2.5gbs LAN was $(cat /proc/irq/"$eth1_0"/smp_affinity)"
echo "CPU Affinity for ETH2 2.5gbs WAN was $(cat /proc/irq/"$eth2_0"/smp_affinity)"
echo "CPU cores assigned to ETH0 queue rx-0 was: $(cat /sys/class/net/eth0/queues/rx-0/rps_cpus)"
echo "CPU cores assigned to ETH1 queue rx-0 was: $(cat /sys/class/net/eth1/queues/rx-0/rps_cpus)"
echo "CPU cores assigned to ETH2 queue rx-0 was: $(cat /sys/class/net/eth2/queues/rx-0/rps_cpus)"

# Set the CPU affinity for IRQs using variables
echo -n ff > /sys/class/net/eth2/queues/rx-0/rps_cpus
echo -n ff > /sys/class/net/eth1/queues/rx-0/rps_cpus
echo -n ff > /sys/class/net/eth0/queues/rx-0/rps_cpus
echo -n 30 > /proc/irq/"$eth2_0"/smp_affinity
echo -n c0 > /proc/irq/"$eth1_0"/smp_affinity

# Display new CPU cores assigned to new IRQs and queues
echo "CPU Affinity for ETH1 2.5gbs LAN is now $(cat /proc/irq/"$eth1_0"/smp_affinity)"
echo "CPU Affinity for ETH2 2.5gbs WAN is now $(cat /proc/irq/"$eth2_0"/smp_affinity)"
echo "CPU cores assigned to ETH0 queue rx-0 is now: $(cat /sys/class/net/eth0/queues/rx-0/rps_cpus)"
echo "CPU cores assigned to ETH1 queue rx-0 is now: $(cat /sys/class/net/eth1/queues/rx-0/rps_cpus)"
echo "CPU cores assigned to ETH2 queue rx-0 is now: $(cat /sys/class/net/eth2/queues/rx-0/rps_cpus)"

The script works great! However everytime I make a change in sqm the cpu affinity gets reverted back to old values

Code: Select all

CPU Affinity for ETH1 2.5gbs LAN was 04
CPU Affinity for ETH2 2.5gbs WAN was 08

Instead of the new values I set

Code: Select all

CPU Affinity for ETH1 2.5gbs LAN is now c0
CPU Affinity for ETH2 2.5gbs WAN is now 30

I fixed this by modifying /etc/init.d/sqm by adding ` /root/performancetweak.sh` to each function of sqm init.d

Code: Select all

#!/bin/sh /etc/rc.common

START=50
USE_PROCD=1

service_triggers()
{
        procd_add_reload_trigger "sqm"
}

reload_service()
{
        stop "$@"
        start "$@"
        /root/performancetweak.sh
}

start_service()
{
        /usr/lib/sqm/run.sh start "$@"
        /root/performancetweak.sh
}

stop_service()
{
        /usr/lib/sqm/run.sh stop "$@"
        /root/performancetweak.sh
}

boot()
{
        export SQM_VERBOSITY_MIN=5 # Silence errors
        start "$@"
        /root/performancetweak.sh
}

This works great! Now everytime I modify sqm the changes are retained.

I have one issue. On reboot the cpu affinity is reverted.

I added to my /etc/rc.local

Code: Select all

/root/performancetweak.sh

exit 0

This is my logread on boot

Code: Select all

Sun Jan  7 20:10:15 2024 user.notice ucitrack: Setting up /etc/config/sqm reload dependency on /etc/config/firewall
Sun Jan  7 20:10:15 2024 user.notice ucitrack: Setting up /etc/config/odhcpd reload dependency on /etc/config/dhcp
Sun Jan  7 20:10:15 2024 user.notice ucitrack: Setting up non-init /etc/config/fstab reload handler: /sbin/block mount
Sun Jan  7 20:10:15 2024 user.notice ucitrack: Setting up /etc/config/system reload trigger for non-procd /etc/init.d/led
Sun Jan  7 20:10:15 2024 user.notice ucitrack: Setting up /etc/config/luci_statistics reload dependency on /etc/config/system
Sun Jan  7 20:10:15 2024 user.notice ucitrack: Setting up /etc/config/dhcp reload dependency on /etc/config/system
Sun Jan  7 20:10:15 2024 daemon.info avahi-daemon[3825]: Server startup complete. Host name is openwrt.local. Local service cookie is 4273136675.
Sun Jan  7 20:10:15 2024 daemon.notice procd: /etc/rc.d/S95done: CPU Affinity for ETH1 2.5gbs LAN was 04
Sun Jan  7 20:10:15 2024 daemon.notice procd: /etc/rc.d/S95done: CPU Affinity for ETH2 2.5gbs WAN was 08
Sun Jan  7 20:10:15 2024 daemon.notice procd: /etc/rc.d/S95done: CPU cores assigned to ETH0 queue rx-0 was: fe
Sun Jan  7 20:10:15 2024 daemon.notice procd: /etc/rc.d/S95done: CPU cores assigned to ETH1 queue rx-0 was: fe
Sun Jan  7 20:10:15 2024 daemon.notice procd: /etc/rc.d/S95done: CPU cores assigned to ETH2 queue rx-0 was: fe
Sun Jan  7 20:10:15 2024 daemon.notice procd: /etc/rc.d/S95done: CPU Affinity for ETH1 2.5gbs LAN is now c0
Sun Jan  7 20:10:15 2024 daemon.notice procd: /etc/rc.d/S95done: CPU Affinity for ETH2 2.5gbs WAN is now 30
Sun Jan  7 20:10:15 2024 daemon.notice procd: /etc/rc.d/S95done: CPU cores assigned to ETH0 queue rx-0 is now: ff
Sun Jan  7 20:10:15 2024 daemon.notice procd: /etc/rc.d/S95done: CPU cores assigned to ETH1 queue rx-0 is now: ff
Sun Jan  7 20:10:15 2024 daemon.notice procd: /etc/rc.d/S95done: CPU cores assigned to ETH2 queue rx-0 is now: ff
Sun Jan  7 20:10:15 2024 daemon.notice procd: /etc/rc.d/S96led: setting up led WAN
Sun Jan  7 20:10:15 2024 daemon.notice procd: /etc/rc.d/S96led: setting up led LAN1
Sun Jan  7 20:10:15 2024 daemon.notice procd: /etc/rc.d/S96led: setting up led LAN2
Sun Jan  7 20:10:15 2024 daemon.info aria2: Instance "main" disabled.

As you can see it was successful however... at some point the cpu affinity still gets reverted back to the old values. What is the root cause?

I have to revert back the cpu affinity by either rerunning the script or by changing sqm settings so that it runs based on my modified init.d. The modified sqm init.d is how I solve the problem where every time we make a change in sqm the cpu affinity gets reverted back.

Anyways I also posted this issue on GitHub: https://github.com/StarWhiz/NanoPi-R6S- ... M/issues/1

[StarWhiz]

I solved my issue.

The solution was to modify /etc/hotplug.d/net/40-net-smp-affinity

from

Code: Select all

friendlyelec,nanopi-r6s)
        set_interface_core 2 "eth0"
        echo fe > /sys/class/net/eth0/queues/rx-0/rps_cpus
        set_interface_core 4 "eth1-0"
        set_interface_core 4 "eth1-16"
        set_interface_core 4 "eth1-18"
        echo fe > /sys/class/net/eth1/queues/rx-0/rps_cpus
        set_interface_core 8 "eth2-0"
        set_interface_core 8 "eth2-16"
        set_interface_core 8 "eth2-18"
        echo fe > /sys/class/net/eth2/queues/rx-0/rps_cpus
        ;;

to

Code: Select all

friendlyelec,nanopi-r6s)
        set_interface_core f0 "eth0"
        echo ff > /sys/class/net/eth0/queues/rx-0/rps_cpus
        set_interface_core 30 "eth1-0"
        set_interface_core 30 "eth1-16"
        set_interface_core 30 "eth1-18"
        echo ff > /sys/class/net/eth1/queues/rx-0/rps_cpus
        set_interface_core c0 "eth2-0"
        set_interface_core c0 "eth2-16"
        set_interface_core c0 "eth2-18"
        echo ff > /sys/class/net/eth2/queues/rx-0/rps_cpus
        ;;

and reboot

All changes seem to persist after this.

I updated my GitHub at https://github.com/StarWhiz/NanoPi-R6S- ... gabit-SQM/ to reflect this change

Credits to: https://openwrt.org/docs/guide-user/adv ... ng_smp_irq

[nikoloz]

Off topic:

Hi!
How do I compose the New Post?
I have been seeking for the "New Topic" button, But I have not found it yet!
Could anyone kindly index me, where the "New Topic" button is supposed to be?