Hyperthreading or not?

[brucehoult]

I have a Ryzen 5600G (6 cores 12 threads) and I installed a utility that constantly and quietly shows CPU usage on my desktop. There are some applications that make brief usage of more then one thread but for me those are quite rare. It is nice if a "make -j12" finishes in a few minutes instead of half an hour, but that sort of usage is rare for me.

I believe you that is true for you, and it's probably true for most people.

However it's not true of developers. Yeah, you might spend an hour editing code using a fraction of one CPU core, but then the difference between a few seconds and a few minutes to build and run your code is the difference between losing your train of thought -- your "flow" -- and not. Also CI (continuous integration) can use a whole bunch of cores. And running thousands of independent unit tests.

And especially developers who are helping design an ISA. You think of a new instruction. You add it to the simulator and the compiler. You recompile the compiler to be able to generate code using the new instruction. Then you build all the libraries. Then the Linux kernel. Then all the basic packages for a distro (typically Buildroot or Yocto for speed if you're doing this). It's a LOT of stuff to build to test how your new instruction works, how much difference it makes to code size, how much difference it makes to speed (if your emulator is cycle-accurate, or if you build a hardware core for an FPGA)

Many of Apple's core customers are doing things that can use all the cores they can get. Photoshop. Video and audio editing and compressors.

Benchmarks show large speed increases for multi core processors, but (unfortunately) the single thread performance is still very much the driving factor.

Right, which is why we have CPUs that might drop down close to 2 GHz with all 24 or 32 or 64 cores running flat out, and then burst to 6 GHz when just one or two cores are being used.

Even with applications that do support multi threading, it's rare to see CPU usage above 30% (which suggests only 4 of the 12 threads are actually used). Hyperthreading started in 2002: https://en.wikipedia.org/wiki/Hyper-threading and now, 22 years later we still have a long way to go before "mainstream" software developers start taking it seriously. I have for example a FreeCAD drawing, that takes several minutes to load, and during that time, CPU usage is a steady 8 percent (i.e 100/12 = 1 thread). It seems that programs are designed as a single thread, and then some limited multithreading is being attempted only after it is discovered that the application gets slow and sluggish.

Depends on the OS.

Apple introduced "Grand Central Dispatch" into Mac OS and iOS and compilers in 2009 -- and incidentally it was added to FreeBSD the same year. Apple quite strongly encourages developers to use GCD.

GCD consists of breaking your program into individual processing steps, with inputs, outputs, and dependencies between them. The library then organises the processing steps and decides  which ones can be run in parallel based on their dependencies and the number of CPU cores you have.

It is not unlike "make", but within a program.

At the time GCD was introduced most people had just 2 CPU cores (Core 2 Duo), but of course now 4, 8, 10, 12, 16 are common.

This FreeBSD page has a simple explanation and example:

https://wiki.freebsd.org/GrandCentralDispatch

[brucehoult]

(...)a utility that constantly and quietly shows CPU usage on my desktop.(...)

What is the name of this utility? Seems very useful.

htop? nmon? System Monitor? (Task Manager in Windows, Activity Monitor on Mac) 

Or if you prefer something in the menu bar, System Load Indicator (apt install indicator-multiload) or gnome-system-monitor or TopHat

[garrettm]

On modern AMD hardware, disabling SMT lowers performance and does NOT lower power draw. So, for some architectures, there is no tangible benefit of disabling SMT/HT:

https://www.phoronix.com/review/amd-epyc-9755-smt

https://www.phoronix.com/review/amd-ryzen-zen5-smt

[BeBuLamar]

My computer has 6 cores each CPU and each CPU has 12 threads. As someone mentioned it rarely uses a lot of thread but when I tried DxO Photolab 8 it seemed to use all the available cores.

[5U4GB]

As others have sort of pointed out, it depends on the workload.  With an embarassingly parallel benchmark which several people have posted, hyperthreading helps.  OTOH if you've got an aggressively single-threaded application (MS Office immediately springs to mind) you want one core in turbo mode and the rest idle as much as possible to allocate the thermal budget to the turbo-mode core.

[NiHaoMike]

On modern kernels, you can use /sys/devices/system/cpu/smt/control to turn hyperthreading on or off without a reboot, so you can set it to whichever one is better for the particular task you're using it for at the moment.

[brucehoult]

On modern kernels, you can use /sys/devices/system/cpu/smt/control to turn hyperthreading on or off without a reboot, so you can set it to whichever one is better for the particular task you're using it for at the moment.

On Linux you can just directly tell each program which CPU cores it (and it's children) are allowed to use.

e.g. on my i9-13900HX laptop, taskset -c 0-15 uses only the P cores, with hyperthreading, 0-15:2 uses only one thread on each P core (or 1-15:2 uses the other one). 16-31 uses the E cores. 0-15:2,16-31 uses the E cores and one hyperthread on each P core.

No need to reboot, and you can partition cores between tasks however you wish. e.g. use taskset to run bash and then everything you run from that bash will use only those cores.

[NiHaoMike]

My understanding is that although you can use taskset to block a process from using both threads of a core at once (it's unaware of hyperthreading, mostly the case for old multithreaded software), it will not stop other processes from using the other thread of a core and degrading performance. Ideally, the solution would be some process manager that would allow certain processes to reserve cores so that they would not have other processes contending for resources on those cores.

[brucehoult]

My understanding is that although you can use taskset to block a process from using both threads of a core at once (it's unaware of hyperthreading, mostly the case for old multithreaded software), it will not stop other processes from using the other thread of a core and degrading performance. Ideally, the solution would be some process manager that would allow certain processes to reserve cores so that they would not have other processes contending for resources on those cores.

You can do all your other things from a complementary taskset, with neither once including the 2nd hyperthread "core"(s) that you want to go unused.

Sure, the OS might still occasionally schedule one of its brief housekeeping tasks onto that hyperthread core, but that's unlikely to make any measurable difference to throughput -- just a little jitter from time to time.  If you want real-time tasks or something then you want one or more cores that the OS doesn't manage at all, and that don't share caches or any other resources with OS-managed cores -- preferably with a dedicated ITIM and DTIM. I'm assuming we're doing this on a "close enough is good enough" probabilistic basis.

[paulca]

AMD consumer chips have a load sensing "single thread boost" mechanism.  When a heavy single thread load is detected, the CPU schedules it across the 2 fastest cores.  This allows the thread to produce more heat but spread it across a wider area and therefore facilitate higher boost clocks for single thread loads.

So it's kinda like negative hyperthreading.  Instead of running multiple threads per core they are using multiple cores to run one thread.

[brucehoult]

AMD consumer chips have a load sensing "single thread boost" mechanism.  When a heavy single thread load is detected, the CPU schedules it across the 2 fastest cores.  This allows the thread to produce more heat but spread it across a wider area and therefore facilitate higher boost clocks for single thread loads.

So it's kinda like negative hyperthreading.  Instead of running multiple threads per core they are using multiple cores to run one thread.

How does a CPU (hardware) schedule an OS construct (thread)?

Does it dynamically renumber the cores behind the OSes back?

[ejeffrey]

The cpu doesn't actually do that.  The scheduler does.

Rather, the CPU provides data tables to the scheduler on which cores are preferred. On a multi core system, each CPU can have a different max boost frequency, and in AMD CPUs especially the cost to migrate between different core complexes is high because they don't share L3 cache. 

[brucehoult]

The cpu doesn't actually do that.  The scheduler does.

That is what I would expect.

in AMD CPUs especially the cost to migrate between different core complexes is high because they don't share L3 cache.

On, say, the 9950X, each cluster of 8 cores on a CCD shares 32 MB of L3 cache. As there is an inter-CCD bandwidth of 64 GB/s, the time to transfer the entire L3 from one CCD to the other is 0.5 ms.

So if you move a single active thread from one CCD to the idle CCD once every 50 ms that's a 1% overhead.

That sounds better than down-clocking by even 100 MHz (1.75%) from 5.7 GHz to me.

[coppice]

The cpu doesn't actually do that.  The scheduler does.

Rather, the CPU provides data tables to the scheduler on which cores are preferred. On a multi core system, each CPU can have a different max boost frequency, and in AMD CPUs especially the cost to migrate between different core complexes is high because they don't share L3 cache.

This is always the case for any non-homogenous multi-core system. The CPU can't independently do what is needed. The OS can't independently do what is needed. There needs to be cooperation. The horrible early results with hyper-threading were mostly due to the OS being inadequately aware of the nature of the CPUs it was scheduling for, so it would often run 4 threads on 2 CPUs, leaving 2 CPUs idle in a quad CPU package. As the OSes added hyper-thread awareness performance improved. Whatever you do to make cores and memory less symmetric in their behaviour requires the hardware to tell the OS what it looks like, and the OS to understand that information and behave sensibly.

[paulca]

If it is a BIOS option and works in both linux and windows without specific loads....  it's not the OS.

The CPU hardware is well aware of what threads are....  an incrementing program counter.  Nothing more.

[coppice]

If it is a BIOS option and works in both linux and windows without specific loads....  it's not the OS.

That makes no sense at all. The BIOS turns off the hyperthreading capability in the hardware, so the OS doesn't even see a hyperthread capable environment. x86 CPUs which are hyperthread capable present the virtual CPUs to the OS, and also lets the OS know their nature. If the OS ignores their nature it just sees a lot of cores, and uses them, often very poorly. If the OS is smart enough to use the extra information it can achieve better performance.