PC for software development - recommendations please!

[DimitriP]

I also stumbled on a benchmarking project for yocto ... It'll be interesting to see your 7200U results. https://github.com/sveinse/yocto-benchmark

I just tried to use that on Ubuntu 22.04 LTS. Fell into Python version incompatibility hell. CBA.

Last commit there Jan 2020.

If anyone who knows what they're doing and clones and fixes it, lmk.

I went passed the python issue by removing "python" from the installar and linking /usr/bin/python to python2
Then I discovered it also needs QEMU , (https://docs.yoctoproject.org/dev-manual/qemu.html )

So I jumped out of the rabbit hole while I could


 

[DimitriP]

I'm using an NVMe 2GB SSD (Samsung 970 EVO Plus), albeit via external  USB C.

Is there a multi-tasking disk benchmark?
My gut says that the multi-tasking performance of a USB drive might be particularly sucky compared to PCIe or SATA.  (But I don't have any actual data.)

Even connected via PCI or SATA, SSD performance can be dissapointing when it comes to "small" transfers.
Writing is yet another nightmare with SSDs sometimes getting beat by good old spinning drives.

[leo938]

For anyone still interested in this I attach a screenshot below during running compilation.

Also cat /proc/cpuinfo | grep "cpu MHz" returns

cpu MHz      : 2600.000
cpu MHz      : 2599.999
cpu MHz      : 2599.998
cpu MHz      : 2600.000

Not sure if this suggests thermal throttling is not occurring.

[Monkeh]

You'll know it's thermal throttling when the CPU temperature is at 95C. This is usually quite hard to do on a desktop except in an oven.

Or by putting a Dell workstation to sleep and waking it up again. What fan controller?

[retiredcaps]

Thanks for the suggestion, there's a HP Z620 going near me for €300, has 4 times the performance of my current setup!

I have a similar situation to yours.  For compiling LineageOS, on my i5-660 desktop (2 cores/4 threads) I get compile times of 9 hours 37 minutes from scratch (no ccache, make clean).

I recently got an used HP Z420 and with it's stock E5-1620 (4 cores/8 threads) it takes 4 hours 12 minutes (no ccahe, make clean).  Then I upgraded the CPU to a Xeon ES-2690 (8 cores/16 threads) from Aliexpress for $10 USD and now it takes 2 hours and 40 minutes (no ccache, make clean).

CPU temp on E5-1620 and E5-2690 at 80C with all cores running at 100%.

All done using a 500GB 7,200 RPM spinning disk (about 130MB/sec read speed using hdparm).

[DiTBho]

LOL 

my current builder is a cluster of two mac-mini
1) Intel(R) Core(TM)2 Duo CPU, P7550@2.26GHz, 8GB ram
2) PowerPC7447A, altivec supported@1.5HHz, 1GB ram

I need to invest money in an HP(1) Xeon machine, or, better still, in an Ampera Arm 128 cores 

(1) HP Z620 Workstation
2x Intel Xeon E5-2670 V2 2.50GHz/3.30GHz = 20 Cores Total / 40 Threads
64GB RAM
700 UKP

[james_s]

14 hours?! 

Isn't the Core2 Duo like 15 years old though? I would expect any modern machine to be vastly more powerful than that. I think this is the longest compile time I've ever heard of.

[DiTBho]

Isn't the Core2 Duo like 15 years old though?

it's this one, late 2009, old but enough for my average daily needs.

[brucehoult]

14 hours?! 

Isn't the Core2 Duo like 15 years old though? I would expect any modern machine to be vastly more powerful than that. I think this is the longest compile time I've ever heard of.

The Core2Duo itself is still fine for most people's daily computing needs. Any problems are more likely to be the very limited amount of RAM supported by the motherboards.

This includes programmers working on individual application programs.

But for large software builds of entire OSes and toolchains ... the more cores the better, if the build system has been updated to be able to use them.

And as much MHz boost as possible with 1 or 2 cores active. Too many build systems alternate between spending aaaaages using 2 cores to run autoconf ... and then a few seconds using 128 cores to compile all the C files ... and then aaaages using one core for a link or maybe the last big C++ compile.

As for compilation times ... when I bought an EOL sale Pentium Pro 200 with 32 MB RAM in late 1998 as my first dedicated Linux machine it took 24 hours to build the d2c Dylan compiler. After throwing another 128 MB RAM in there (the other 4 slots...) that time dropped to 30 minutes.

[brucehoult]

I've no experience building desktops, if I go with a DIY option is a big issue to upgrade the CPU in say 5 yrs? Will I likely need a new motherboard as well?

Impossible to say.

Historically, Intel liked to change CPU sockets every couple of generations, or for versions with different numbers of RAM channels etc

- LGA 1366 for Nehalem "Bloomfield"

- LGA 1156 for Nehalem "Lynnfield"

- LGA 1366 for Westmere

- LGA 1155 for Sandy Bridge, Ivy Bridge

- LGA 2011 for Sandy Bridge-E, Ivy Bridge-E

- LGA 1150 for Haswell, Broadwell

- LGA 2011-3 for Haswell-E, Broadwell-E

- LGA 1151 for Skylake, Kaby Lake, Coffee Lake (6700, 7***, 8***, 9***)

- LGA 2066 for Skylake-X, Kaby Lake-X

- LGA 1200 for Comet Lake, Cyprus Cove (10700, 11700)

- LGA 1700 for Colden Cove, Raptor Cove (12700, 13700


AMD on the other stuck with the AM4 socket for a LONG TIME -- 2017..2021 -- for non threadripper CPUs. Just went to AM5 Sep 2022.  AM5 good for the next five years? Maybe.....

[retiredcaps]

Then I upgraded the CPU to a Xeon ES-2690 (8 cores/16 threads) from Aliexpress for $10 USD and now it takes 2 hours and 40 minutes (no ccache, make clean).

E5-2697 v2 for $5 USD + free shipping on Aliexpress.  That's 12 cores/24 threads.

Sorry, suspect listing as I didn't see the small print on my phone.

However, my E5-2690 for $10 USD + free shipping that I got from Aliexpress is genuine and it works in my Z420.  That's why when I saw the posting for E5-2697 v2 for $5 USD + free shipping it didn't set off any alarm bells initially.

[brucehoult]

Then I upgraded the CPU to a Xeon ES-2690 (8 cores/16 threads) from Aliexpress for $10 USD and now it takes 2 hours and 40 minutes (no ccache, make clean).

E5-2697 v2 for $5 USD + free shipping on Aliexpress.  That's 12 cores/24 threads.

That falls in the "are you sure it's genuine?" range.

130W TDP. $2614 at introduction in September 2013. LGA 2011 socket same as "E" versions of Sandy Bridge, Ivy Bridge, Haswell, Broadwell i7s.

If you don't already have a compatible motherboard then you'd be better off buying an entire working machine, not just the CPU. Even at $5 for the CPU.

[rdl]

In the listing there is "Decorative Paintings" and a size of 20cm x 30cm ?
I would be careful.

[retiredcaps]

In the listing there is "Decorative Paintings" and a size of 20cm x 30cm ?
I would be careful.

When I first saw the posting, I didn't notice that on my phone, but now that I'm on a 24 inch monitor, I do see that now.  I'll modify my original post to say it's suspicious.  However, my E5-2690 for $10 USD + free shipping that I got from Aliexpress is genuine and it works in my Z420.  That's why when I saw the posting for E5-2697 v2 for $5 USD + free shipping it didn't set off any alarm bells initially.

[IanB]

Also estimated delivery of 6 Aug? Who has time to wait that long?

[nctnico]

I'm using an NVMe 2GB SSD (Samsung 970 EVO Plus), albeit via external  USB C.

Is there a multi-tasking disk benchmark?
My gut says that the multi-tasking performance of a USB drive might be particularly sucky compared to PCIe or SATA.  (But I don't have any actual data.)

Doesn't really matter for compiling large amounts of software. Just make sure to have lots of memory. The sources are typically compressed so the amount of data that needs to be read from the hard drive is minimal. The decompressed files are then stored in disk cache formed by unused memory. The compilation process itself will take longer than it takes to spoon feed all the files to the hard drive BUT the compilation process isn't hampered by it.

[leo938]

Just to finish this off if anyone is looking in the future..

I bought a seconds hand HP Z640 for around €650. 128GB RAM, E5-1680 v3 @ 3.20GHz × 8, 2TB SSD.

Pretty much like night & day!

[retiredcaps]

I bought a seconds hand HP Z640 for around €650. 128GB RAM, E5-1680 v3 @ 3.20GHz × 8, 2TB SSD.

So from the original 14 hours, what is your new compile time?

[leo938]

Approx 1.5hrs

[SiliconWizard]

For anyone still interested in this I attach a screenshot below during running compilation.

Also cat /proc/cpuinfo | grep "cpu MHz" returns

cpu MHz      : 2600.000
cpu MHz      : 2599.999
cpu MHz      : 2599.998
cpu MHz      : 2600.000

Not sure if this suggests thermal throttling is not occurring.

It's not hard to determine. Just look at the temperatures first. 'sensors' is a good start.
You have a Xeon CPU if I'm not mistaken? These have usually a higher max Tj than consumer CPUs - in the order of 100°C or slightly over. You'd have to have a very poor cooling solution to reach that temperature. It's almost impossible with those Xeon CPUs.

But what you are seeing is not completely different from thermal throttling. Intel CPUs have a max TDP, and while that can often be bypassed in some way in the BIOS for many consumer CPUs, it's usually not possible to bypass the TDP for Xeons. So, just look at the TDP of your Xeon and you'll have your answer.

These Xeon CPUs will never run at the max clock freq when all cores are at 100% utilization. They will throttle to the base frequency which is often 2.6GHz for Xeons of this generation.
There's nothing you can do about it.
Well, I've seen that for some specific Xeon in the E3 or E5 series (earlier than V4 or V3 I think), you can unlock them to get the boost frequency on all cores at 100% utilization, but that usually requires a modified BIOS. Not something I would try unless you feel adventurous.

[paulca]

Even in consumer high end gaming hardware where everything is unlocked AND you have an expensive water cooling system, it's extremely difficult to tune clock speeds for maximum 100% on all cores.

The times of setting a fixed clock on all cores is pretty much gone except for bespoke loads.  Everything is dynamic from the core clock speeds to which cores take which loads.  The latest AMDs, for example, when given a single threaded load, will parallel it on 2 cores and alternate the them 50/50 at max boost speed.  Doing balances the heat and allows a faster average clock speed than a single core at 100% could deliver.

There are still people who set fixed core clocks.  They usually score well on benchmarks, but when you get them to think it through for normal daily use, having the dynamics control the clock speed usually results in bursts of 50% more performance until the thermals pull it back.

The dynamics work not just on temperature but on voltage and current.  TDP TDC and so forth.

The difference between those CPUs and the likes of Xeon's is the later are built for industrial use.  They don't have the highest clocks, they don't have the highest performance, but they are built like a tank, built to be beaten on 24/7 in a rack server for 10+ years without failing.

Your average gaming CPU that gets overclocked and beaten on will fail quite often.  Mine throws DPC Watchdog time-outs every other cold start for example.  Not an issue for me, but in a data-centre that kind of stuff costs a fortune. 

[DiTBho]

The times of setting a fixed clock on all cores is pretty much gone except for bespoke loads.  Everything is dynamic from the core clock speeds to which cores take which loads.

I hacked both the firmware and the kernel of some modern ARM and MIPS computers to disable and remove every dynamic clock engine. It was too problematic on the Boorst main bus with its perpetual thermal thermal throttling, and on small SBC(1) the thermal management unit caused physical damage to the SoC.

So, from my point of view, this is the first thing I don't want; perhaps, am64 and x86-* are ready for this, but other architectures are not so much.


(1) the cheapest and simplest example I can give you is the NeoPi, where I had to force the clock to 600Mhz, instead of its dynamic range { 600 .. 1100 }Mhz to save the SoC from cooking like an omelet since the Thermal management doesn't work properly.

[Nominal Animal]

leo938: having 128GB of RAM, you should have the entire source tree in RAM cached for basically any build!  Nice!

Whenever the build tree is much larger than available RAM, using software-RAID1 and two identical drives to hold the tree, used to help a lot.
(Wait, isn't RAID1 mirroring?  Don't you mean RAID0?  No.  In Linux, sw-RAID1 reads access only one of the drives, and if the bottleneck is accessing many small files, you can increase the rate significantly –– not double, but somewhat close –– by using sw-RAID1.  You lose half the capacity, though.  In the past, I did this with spinny rust, in which case the difference compiling the Linux kernel and userspace was significant.)
Nowadays, RAM is so much cheaper, than it is makes much more sense to bulk up on RAM than do that.

Nevertheless, my next desktop/HPC machine will have dual SSD for storage; partially RAID-[01].  I have some very large continuous datasets (hundreds of megabytes) that I'd like to see how fast I can chew trough, mostly MD simulations and such; I'm really looking forward to 1GB/s+ transfer rate for those...
If there are any QChemists using Dalton here, they might wish to try the same.  (I'm writing my own simulators and visualizers, though.)


DiTBho, and others:

One reason I'm lusting after a small hobby milling machine (maybe Proxxon MF70), capable of milling 150×100×50mm aluminium blocks into custom heatsink-lids for my SBCs.  I often use Hammond die-cast aluminium boxes (and clones) for enclosures, but when dust/waterproof, you need solid connection to the chassis as a heatsink or the SBC will fry eventually.  So, outside should be ribbed for maximizing surface area, and inside needs "towers" to contact to the hot chips.

Currently I'm playing with vertical wall-hanging convection-cooled thin sheet steel enclosure for MikroTik RBM33G – the very same you got recently.  My first plan of using a knotty half-round Birch block (to make it decorative) was too heat-insulating – even if it produces only 3-4 Watts of heat in normal use in my case (LTE firewall) –, but maybe as a shaped veneer on the steel lid.  The steel might even help with EM noise stuff, who knows: the official CA433U enclosure definitely is stamped aluminium.  (Thin sheet steel is half the price of aluminium here, and much more easily available in automotive stores; otherwise I'd use aluminium sheet myself.)

[DiTBho]

One reason I'm lusting after a small hobby milling machine

The NanoPi is sold with a poor heatsink and you can replace it with *something else*, however, this only solves half of the problem: with a dynamic clock you also experiment issues with dynamic ram, which causes random instabilities and crashes, and this happens because both the CPU and RAM share the same PLL for the CPU clock and ram clock.

Bad design choice, I know, but that's how some SoCs are made, and you may think at this point "ok, we know Allwinner sells SoC with a bugged OR1 MPU, and we also know that OR1 MPU is there to manage the thermal unit", but even Mediatek sells this kind of things, so ... or they fix the thermal management and split the PLL once for all (and fix that buggy OpenRisc1 co-processor MPU(1)), or I will still patch/hack my kernels to disable it!


(1) I think they have to understand they have to spend more money to cure their products better! Otherwise, sorry, but your products are *de facto* not ready for thermal throttling, and I will disable it!

[Nominal Animal]

Exactly.  Many Odroids now use a extruded aluminium heatsink as their base, with the SoC on the bottom of the board; this is Very Nice, because the heatsink doesn't need milling.  With RBM33G, you need 14mm to reach the highest components (connectors in front), so say 17mm for comfort (and to being able to route any modem or WiFi antenna cables to the front like I want to, so all wires hang downwards on the wall).  An inch thick for top and a half inch for bottom, both 6.3"×4.8" or larger, will suffice, and have a bit of thickness left to form into shallow cooling fins.  For cooling WiFi/LTE cards, some kind of internal pillar with heatsinks is needed, too.  It's easier to do a convection-cooled "open" wall chassis for it, overall, and one can curve the top to make sure any accidental drops of water won't get inside.  (In my case, because of plants watered nearby and on the same wall.)

For desktop PCs, quiet but efficient cooling is one of my pet peeves.  Large piped heatsinks and large (140mm and larger) fans make it relatively easy for the 65W to 95W TDP class of processors, but it is easy to get a bit too laminar airflow, which does not go through motherboard components; one wants turbulent airflow on the motherboard surface to cover them.  Many motherboard manufacturers nowadays include heatsinks over all components that get hot, though; those are even easier to manage, I guess.