Author Topic: Hyperthreading or not?  (Read 9369 times)

0 Members and 3 Guests are viewing this topic.

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4842
  • Country: nz
Re: Hyperthreading or not?
« Reply #25 on: December 21, 2024, 09:37:30 am »
I have a Ryzen 5600G (6 cores 12 threads) and I installed a utility that constantly and quietly shows CPU usage on my desktop. There are some applications that make brief usage of more then one thread but for me those are quite rare. It is nice if a "make -j12" finishes in a few minutes instead of half an hour, but that sort of usage is rare for me.

I believe you that is true for you, and it's probably true for most people.

However it's not true of developers. Yeah, you might spend an hour editing code using a fraction of one CPU core, but then the difference between a few seconds and a few minutes to build and run your code is the difference between losing your train of thought -- your "flow" -- and not. Also CI (continuous integration) can use a whole bunch of cores. And running thousands of independent unit tests.

And especially developers who are helping design an ISA. You think of a new instruction. You add it to the simulator and the compiler. You recompile the compiler to be able to generate code using the new instruction. Then you build all the libraries. Then the Linux kernel. Then all the basic packages for a distro (typically Buildroot or Yocto for speed if you're doing this). It's a LOT of stuff to build to test how your new instruction works, how much difference it makes to code size, how much difference it makes to speed (if your emulator is cycle-accurate, or if you build a hardware core for an FPGA)

Many of Apple's core customers are doing things that can use all the cores they can get. Photoshop. Video and audio editing and compressors.

Quote
Benchmarks show large speed increases for multi core processors, but (unfortunately) the single thread performance is still very much the driving factor.

Right, which is why we have CPUs that might drop down close to 2 GHz with all 24 or 32 or 64 cores running flat out, and then burst to 6 GHz when just one or two cores are being used.

Quote
Even with applications that do support multi threading, it's rare to see CPU usage above 30% (which suggests only 4 of the 12 threads are actually used). Hyperthreading started in 2002: https://en.wikipedia.org/wiki/Hyper-threading and now, 22 years later we still have a long way to go before "mainstream" software developers start taking it seriously. I have for example a FreeCAD drawing, that takes several minutes to load, and during that time, CPU usage is a steady 8 percent (i.e 100/12 = 1 thread). It seems that programs are designed as a single thread, and then some limited multithreading is being attempted only after it is discovered that the application gets slow and sluggish.

Depends on the OS.

Apple introduced "Grand Central Dispatch" into Mac OS and iOS and compilers in 2009 -- and incidentally it was added to FreeBSD the same year. Apple quite strongly encourages developers to use GCD.

GCD consists of breaking your program into individual processing steps, with inputs, outputs, and dependencies between them. The library then organises the processing steps and decides  which ones can be run in parallel based on their dependencies and the number of CPU cores you have.

It is not unlike "make", but within a program.

At the time GCD was introduced most people had just 2 CPU cores (Core 2 Duo), but of course now 4, 8, 10, 12, 16 are common.

This FreeBSD page has a simple explanation and example:

https://wiki.freebsd.org/GrandCentralDispatch
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4842
  • Country: nz
Re: Hyperthreading or not?
« Reply #26 on: December 21, 2024, 09:47:28 am »
(...)a utility that constantly and quietly shows CPU usage on my desktop.(...)
What is the name of this utility? Seems very useful.

htop? nmon? System Monitor? (Task Manager in Windows, Activity Monitor on Mac) 

Or if you prefer something in the menu bar, System Load Indicator (apt install indicator-multiload) or gnome-system-monitor or TopHat
 

Offline garrettm

  • Frequent Contributor
  • **
  • Posts: 342
  • Country: us
Re: Hyperthreading or not?
« Reply #27 on: December 22, 2024, 02:13:02 am »
On modern AMD hardware, disabling SMT lowers performance and does NOT lower power draw. So, for some architectures, there is no tangible benefit of disabling SMT/HT:

https://www.phoronix.com/review/amd-epyc-9755-smt

https://www.phoronix.com/review/amd-ryzen-zen5-smt
 

Offline BeBuLamarTopic starter

  • Super Contributor
  • ***
  • Posts: 1429
  • Country: us
Re: Hyperthreading or not?
« Reply #28 on: December 24, 2024, 10:32:37 pm »
My computer has 6 cores each CPU and each CPU has 12 threads. As someone mentioned it rarely uses a lot of thread but when I tried DxO Photolab 8 it seemed to use all the available cores.
 

Offline 5U4GB

  • Frequent Contributor
  • **
  • Posts: 647
  • Country: au
Re: Hyperthreading or not?
« Reply #29 on: December 25, 2024, 03:58:06 am »
As others have sort of pointed out, it depends on the workload.  With an embarassingly parallel benchmark which several people have posted, hyperthreading helps.  OTOH if you've got an aggressively single-threaded application (MS Office immediately springs to mind) you want one core in turbo mode and the rest idle as much as possible to allocate the thermal budget to the turbo-mode core.
 

Online NiHaoMike

  • Super Contributor
  • ***
  • Posts: 9330
  • Country: us
  • "Don't turn it on - Take it apart!"
    • Facebook Page
Re: Hyperthreading or not?
« Reply #30 on: December 25, 2024, 12:42:02 pm »
On modern kernels, you can use /sys/devices/system/cpu/smt/control to turn hyperthreading on or off without a reboot, so you can set it to whichever one is better for the particular task you're using it for at the moment.
Cryptocurrency has taught me to love math and at the same time be baffled by it.

Cryptocurrency lesson 0: Altcoins and Bitcoin are not the same thing.
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4842
  • Country: nz
Re: Hyperthreading or not?
« Reply #31 on: December 27, 2024, 01:43:55 am »
On modern kernels, you can use /sys/devices/system/cpu/smt/control to turn hyperthreading on or off without a reboot, so you can set it to whichever one is better for the particular task you're using it for at the moment.

On Linux you can just directly tell each program which CPU cores it (and it's children) are allowed to use.

e.g. on my i9-13900HX laptop, taskset -c 0-15 uses only the P cores, with hyperthreading, 0-15:2 uses only one thread on each P core (or 1-15:2 uses the other one). 16-31 uses the E cores. 0-15:2,16-31 uses the E cores and one hyperthread on each P core.

No need to reboot, and you can partition cores between tasks however you wish. e.g. use taskset to run bash and then everything you run from that bash will use only those cores.
 

Online NiHaoMike

  • Super Contributor
  • ***
  • Posts: 9330
  • Country: us
  • "Don't turn it on - Take it apart!"
    • Facebook Page
Re: Hyperthreading or not?
« Reply #32 on: December 27, 2024, 04:06:44 am »
My understanding is that although you can use taskset to block a process from using both threads of a core at once (it's unaware of hyperthreading, mostly the case for old multithreaded software), it will not stop other processes from using the other thread of a core and degrading performance. Ideally, the solution would be some process manager that would allow certain processes to reserve cores so that they would not have other processes contending for resources on those cores.
Cryptocurrency has taught me to love math and at the same time be baffled by it.

Cryptocurrency lesson 0: Altcoins and Bitcoin are not the same thing.
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4842
  • Country: nz
Re: Hyperthreading or not?
« Reply #33 on: December 27, 2024, 07:13:03 am »
My understanding is that although you can use taskset to block a process from using both threads of a core at once (it's unaware of hyperthreading, mostly the case for old multithreaded software), it will not stop other processes from using the other thread of a core and degrading performance. Ideally, the solution would be some process manager that would allow certain processes to reserve cores so that they would not have other processes contending for resources on those cores.

You can do all your other things from a complementary taskset, with neither once including the 2nd hyperthread "core"(s) that you want to go unused.

Sure, the OS might still occasionally schedule one of its brief housekeeping tasks onto that hyperthread core, but that's unlikely to make any measurable difference to throughput -- just a little jitter from time to time.  If you want real-time tasks or something then you want one or more cores that the OS doesn't manage at all, and that don't share caches or any other resources with OS-managed cores -- preferably with a dedicated ITIM and DTIM. I'm assuming we're doing this on a "close enough is good enough" probabilistic basis.
 

Offline paulca

  • Super Contributor
  • ***
  • Posts: 4428
  • Country: gb
Re: Hyperthreading or not?
« Reply #34 on: December 27, 2024, 10:02:24 am »
AMD consumer chips have a load sensing "single thread boost" mechanism.  When a heavy single thread load is detected, the CPU schedules it across the 2 fastest cores.  This allows the thread to produce more heat but spread it across a wider area and therefore facilitate higher boost clocks for single thread loads.

So it's kinda like negative hyperthreading.  Instead of running multiple threads per core they are using multiple cores to run one thread.
"What could possibly go wrong?"
Current Open Projects:  STM32F411RE+ESP32+TFT for home IoT (NoT) projects.  Child's advent xmas countdown toy.  Digital audio routing board.
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4842
  • Country: nz
Re: Hyperthreading or not?
« Reply #35 on: December 27, 2024, 10:55:18 am »
AMD consumer chips have a load sensing "single thread boost" mechanism.  When a heavy single thread load is detected, the CPU schedules it across the 2 fastest cores.  This allows the thread to produce more heat but spread it across a wider area and therefore facilitate higher boost clocks for single thread loads.

So it's kinda like negative hyperthreading.  Instead of running multiple threads per core they are using multiple cores to run one thread.

How does a CPU (hardware) schedule an OS construct (thread)?

Does it dynamically renumber the cores behind the OSes back?
 

Offline ejeffrey

  • Super Contributor
  • ***
  • Posts: 4062
  • Country: us
Re: Hyperthreading or not?
« Reply #36 on: December 28, 2024, 02:55:08 am »
The cpu doesn't actually do that.  The scheduler does.

Rather, the CPU provides data tables to the scheduler on which cores are preferred. On a multi core system, each CPU can have a different max boost frequency, and in AMD CPUs especially the cost to migrate between different core complexes is high because they don't share L3 cache. 
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4842
  • Country: nz
Re: Hyperthreading or not?
« Reply #37 on: December 28, 2024, 03:22:32 am »
The cpu doesn't actually do that.  The scheduler does.

That is what I would expect.

Quote
in AMD CPUs especially the cost to migrate between different core complexes is high because they don't share L3 cache.

On, say, the 9950X, each cluster of 8 cores on a CCD shares 32 MB of L3 cache. As there is an inter-CCD bandwidth of 64 GB/s, the time to transfer the entire L3 from one CCD to the other is 0.5 ms.

So if you move a single active thread from one CCD to the idle CCD once every 50 ms that's a 1% overhead.

That sounds better than down-clocking by even 100 MHz (1.75%) from 5.7 GHz to me.
 

Online coppice

  • Super Contributor
  • ***
  • Posts: 10173
  • Country: gb
Re: Hyperthreading or not?
« Reply #38 on: December 28, 2024, 03:59:54 pm »
The cpu doesn't actually do that.  The scheduler does.

Rather, the CPU provides data tables to the scheduler on which cores are preferred. On a multi core system, each CPU can have a different max boost frequency, and in AMD CPUs especially the cost to migrate between different core complexes is high because they don't share L3 cache.
This is always the case for any non-homogenous multi-core system. The CPU can't independently do what is needed. The OS can't independently do what is needed. There needs to be cooperation. The horrible early results with hyper-threading were mostly due to the OS being inadequately aware of the nature of the CPUs it was scheduling for, so it would often run 4 threads on 2 CPUs, leaving 2 CPUs idle in a quad CPU package. As the OSes added hyper-thread awareness performance improved. Whatever you do to make cores and memory less symmetric in their behaviour requires the hardware to tell the OS what it looks like, and the OS to understand that information and behave sensibly.
 

Offline paulca

  • Super Contributor
  • ***
  • Posts: 4428
  • Country: gb
Re: Hyperthreading or not?
« Reply #39 on: December 30, 2024, 01:07:53 am »
If it is a BIOS option and works in both linux and windows without specific loads....  it's not the OS.

The CPU hardware is well aware of what threads are....  an incrementing program counter.  Nothing more.
"What could possibly go wrong?"
Current Open Projects:  STM32F411RE+ESP32+TFT for home IoT (NoT) projects.  Child's advent xmas countdown toy.  Digital audio routing board.
 

Online coppice

  • Super Contributor
  • ***
  • Posts: 10173
  • Country: gb
Re: Hyperthreading or not?
« Reply #40 on: December 30, 2024, 02:12:43 pm »
If it is a BIOS option and works in both linux and windows without specific loads....  it's not the OS.
That makes no sense at all. The BIOS turns off the hyperthreading capability in the hardware, so the OS doesn't even see a hyperthread capable environment. x86 CPUs which are hyperthread capable present the virtual CPUs to the OS, and also lets the OS know their nature. If the OS ignores their nature it just sees a lot of cores, and uses them, often very poorly. If the OS is smart enough to use the extra information it can achieve better performance.
« Last Edit: December 30, 2024, 02:14:55 pm by coppice »
 

Offline paulca

  • Super Contributor
  • ***
  • Posts: 4428
  • Country: gb
Re: Hyperthreading or not?
« Reply #41 on: January 08, 2025, 01:29:49 pm »
If it is a BIOS option and works in both linux and windows without specific loads....  it's not the OS.
That makes no sense at all. The BIOS turns off the hyperthreading capability in the hardware, so the OS doesn't even see a hyperthread capable environment. x86 CPUs which are hyperthread capable present the virtual CPUs to the OS, and also lets the OS know their nature. If the OS ignores their nature it just sees a lot of cores, and uses them, often very poorly. If the OS is smart enough to use the extra information it can achieve better performance.

If you run a single thread of, say, Pi32 or some artificial single threaded CPU focused load and look at the AMD CPU core performance graphs you will find it running on 2 cores, not 1.  This is handled, as I understand it, at the CPU level, the OS is not involved. 

I don't know how it's done.  I assume the CPU is able to gaurantee the result will be exactly the same as it running on teh single core.  Completely transparent to the OS/Application.

It's similar to how a single threaded load which it can't do this with will automatically migrate around the cores to spread heat.  It will run one core at 100% until it gets to it's de-boost temp and migrate it to the other end of the silicon to another core.

My assumption is that the "cores" identified by the CPU-ID and hardware ident are not in fact the actual cores, but abstractions of them, allowing the AMD "fabric" to manage the CPU better than the OS.

... of course you can influence that "fabric" in many ways from runtime via driver APIs etc.  You can disable the features entirely.  I believe you can even disable all core migrations and dynamic thread boosts.  This would, of course, not be portable between CPUs.

https://www.tomshardware.com/news/amd-introduces-precision-boost-overdrive-2-boosts-single-thread-tremendously
« Last Edit: January 08, 2025, 01:38:12 pm by paulca »
"What could possibly go wrong?"
Current Open Projects:  STM32F411RE+ESP32+TFT for home IoT (NoT) projects.  Child's advent xmas countdown toy.  Digital audio routing board.
 

Offline ejeffrey

  • Super Contributor
  • ***
  • Posts: 4062
  • Country: us
Re: Hyperthreading or not?
« Reply #42 on: January 08, 2025, 04:22:50 pm »
That's not what PBO2 is.  PBO2 is a dynamic overclocking system that allows individual cores to boost to higher than normal frequency dependig on the voltage, current,  temperature, and activity across the chip.  It also comes with processor data tables on which cores have the highest potential boost frequency that allows the scheduler to prioritize the cores with the highest overclocking potential for single threaded workloads.
 

Online coppice

  • Super Contributor
  • ***
  • Posts: 10173
  • Country: gb
Re: Hyperthreading or not?
« Reply #43 on: January 08, 2025, 04:28:57 pm »
If you run a single thread of, say, Pi32 or some artificial single threaded CPU focused load and look at the AMD CPU core performance graphs you will find it running on 2 cores, not 1.  This is handled, as I understand it, at the CPU level, the OS is not involved. 
If you run any single thread task on a multi CPU machine you will usually see at least 2 cores working, one running the application, and at least one running the system activities.
 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4842
  • Country: nz
Re: Hyperthreading or not?
« Reply #44 on: January 08, 2025, 04:37:16 pm »
If you run a single thread of, say, Pi32 or some artificial single threaded CPU focused load and look at the AMD CPU core performance graphs you will find it running on 2 cores, not 1.  This is handled, as I understand it, at the CPU level, the OS is not involved. 
If you run any single thread task on a multi CPU machine you will usually see at least 2 cores working, one running the application, and at least one running the system activities.

System tasks shouldn't be using a whole core!

Here's my laptop right now. The biggest user is htop itself, at 0.7% of one core.

 

Offline brucehoult

  • Super Contributor
  • ***
  • Posts: 4842
  • Country: nz
Re: Hyperthreading or not?
« Reply #45 on: January 08, 2025, 04:40:07 pm »
Running a CPU-bound task (perl -e 'while(1){}').  Now TWO other cores are at 0.7%.

 

Online coppice

  • Super Contributor
  • ***
  • Posts: 10173
  • Country: gb
Re: Hyperthreading or not?
« Reply #46 on: January 08, 2025, 04:41:34 pm »
If you run a single thread of, say, Pi32 or some artificial single threaded CPU focused load and look at the AMD CPU core performance graphs you will find it running on 2 cores, not 1.  This is handled, as I understand it, at the CPU level, the OS is not involved. 
If you run any single thread task on a multi CPU machine you will usually see at least 2 cores working, one running the application, and at least one running the system activities.

System tasks shouldn't be using a whole core!

Here's my laptop right now. The biggest user is htop itself, at 0.7% of one core.
He didn't say it was using the whole of 2 cores. He said 2 cores were working. How much of a second core a single threaded task uses will depend on its nature. One that is number crunching may use very little. One doing I/O can often cause a lot of activity on a second core.
 

Offline paulca

  • Super Contributor
  • ***
  • Posts: 4428
  • Country: gb
Re: Hyperthreading or not?
« Reply #47 on: January 10, 2025, 11:50:48 am »
AMD chips (I don't know about their Intel cousins) are far more complicated these days than people seem to understand.

The thread will split 50/50 plus or minus the factor by which each core is rated, it's temperature and the power limits for it's VRMs.

So instead of 1 core running at 100% producing 100% of heat and ending up being clocked at, say 4Ghz when the die heats up.  It runs the thread on 2 cores at 50% each.  Those two cores at 50% can sustain maybe 4.6Ghz as they are cooler.

The end2end result is that your single threaded load ran at 4.6Ghz and not 4.0Ghz.   By using 2 cores to share the load.

The AMD CPU management systems, including PBO and others are extremely complicated.  They are basically automated overclocking systems which monitor and alter the cores and loads at 1000Hz.  Always aiming to move load to where it's cool and power is available.  Cool cores are boosted higher so they can heat up and the load be moved to another cooler core and repeat.

In some circles you will still fine people disabling all of this and running everything with a single fixed clock rate across the board.  They then soak test it with a 100% all core load for 24 hours and call it the fastest overclock possible.

They are correct, for an all core load for 24 hours.  However if you are using the PC as a normal person would and your loads a bursty desktop loads, then boosting the cores 1 or even 2 Ghz higher and then clocking them back just before they overheat actually increases your performance significantly.  While at the same time, if you DO run a 100% all core load, say rending a video or compiling a kernel, it will probably be a bit slower than the "All core overclock" was.

This a more complete guide


It has got more complicated in the 7 and 9 series though.
« Last Edit: January 10, 2025, 11:55:14 am by paulca »
"What could possibly go wrong?"
Current Open Projects:  STM32F411RE+ESP32+TFT for home IoT (NoT) projects.  Child's advent xmas countdown toy.  Digital audio routing board.
 
The following users thanked this post: tooki


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf