General > General Technical Chat
Linus Sebastion Turned Down $100M Offer
<< < (24/35) > >>
wraper:

--- Quote from: hans on August 18, 2023, 07:23:03 am ---The other components from GN I don't agree with at its core. Forcing an experiment outcome is IMO still a wrong approach to testing. Because that's the vibe I got from their video.

--- End quote ---
And I did not. It's like you don't agree with a straw man (vibe) you created in your head but blame GN. Double checking suspicious test results is not forcing experiment outcome at all. Even medical labs with much more standartized test procedures often redo analyses that had out of ordinary results.

--- Quote ---TLDR. 13x AMD Ryzen 7600 tested. CB23 MT runs in range 13.6k-14.4k. Clocks 4825MHz - 5075MHz. Load temperatures 79C - 88C. Load power 112W - 126W. Game FPS 60 to 86(!). FPS/W: 2.89 to 4.26
That latter has a 41% improvement on the same hardware bill of materials.
--- End quote ---
That is not FPS but power consumption in W with power limits disabled. Not to say FPS/W is not your usual benchmark metric. Actual performance difference between samples will be a few %. LMG let slip GPU test results which were like 2.5 times off.
hans:
Is there any reason or problem to make a discussion personal again? If there is, please let me know, otherwise it's not so cool.

Vibe is my poor and imprecise choice of wording. I don't have anything specifically towards any channel. But I do think that the stated demands is striving towards an utopia of predictability and tight clustering of data. If CPU performance can vary by e.g. 5% then it is questionable how representable any review data is without testing dozens of units. All these tests are samplesize N=1.

Outliers can be removed or retested, or maybe a retest a complete chart while you're at it (e.g. the 4090 might not be an outlier, but all other cards were). But picking bad data only on outliers is still a biased by expected results (its called survivorship bias). The underlying issues all hinge around failed qualifications of test/part conditions under test. A bad cooler mount, mixed data with different quality settings or not checking if a CPU is thermal throttling. But potentially also other mistakes, such as parallelising tests with identical (bill of materials) test benches.

So I think the critics of GN is somewhat simplified. Now granted, their general statement is still valid, and GN is perhaps the better of all channels out there as they do show some qualification of parts (e.g. clocks) before performance data. HUB does that for VRAM usage in certain game tests.
wraper:

--- Quote from: hans on August 18, 2023, 08:15:48 am ---Outliers can be removed or retested, or maybe a retest a complete chart while you're at it (e.g. the 4090 might not be an outlier, but all other cards were). But picking bad data only on outliers is still a biased by expected results. The underlying issues all hinge around failed qualifications of test/part conditions under test. A bad cooler mount, mixed data with different quality settings or not checking if a CPU is thermal throttling. But potentially also other mistakes, such as parallelising tests with identical (bill of materials) test benches.

--- End quote ---
The best is doing multiple rounds of tests by switching samples of say GPUs on a test stand on each round yet again and verifying settings each time. Not just setting it up once and running multiple times in the same possibly wrong configuration. However it would require much more time and effort, so double checking suspicious results is a bare minimum of due diligence.
EEVblog:
I can't help but think the TLDR from this is that Linus overreacted and dug his own hole here.
First mistake was taking the GN bait. He should have just Tweeted "Yeah, we'll do better" or some such and left it at that.
As for the former employee Tweet storm, it was clearly oportunistically timed to hammer home the pile-on, as it was 2+ years ago, and most of the stuff is just griping about a job she didn't like.
If he hadn't reacted to the GN thing it wouldn't have blown up, and it's likely the former employee tweet storm wouldn't have happened.
Next mistake was the apology tour video. Although it was a texbook fine corporate response video, and was genuine, again, it was not needed and won abolsutely no one back and it just drives the media frenzy more. As does halting video production.
Ice-Tea:
Kinda disagree. Some of the criticism is valid. They might a well have a look at it. And if they can afford to shut down for a week that would probably work better than asking your already overtasked workforce to cobble something together in between everything else.

Never waste a good crisis ;-)
Navigation
Message Index
Next page
Previous page
There was an error while thanking
Thanking...

Go to full version
Powered by SMFPacks Advanced Attachments Uploader Mod