EEVblog #726 - Dual Xeon Video Editing Machine Build

[george graves]

I didn't watch the video all the way through, but he's exporting to a file, and then using handbrake? Is that correct?

Yes

Eeek! Well there's your problem!  At least part of the problem.  Dave, no wonder why you are frustrated!

Why transcode twice? Plus handbrake is a consumer tool.  So is Sony Vegas "non-pro"  or what ever they call it(the pro version is ok - it does do non-standard video frames well- but that's not what we are talking about here). If I was video blogging for a living, I'd either live with some cheap software, or go towards a more professional set up.  After some 700 videos, you have to be getting sick of all these headaches. I honestly feel your pain.

I did a little test, and while running photoshop, a screen cap software, firefox with +40 tabs open(yea, I'm still on firefox - don't judge me), I imported a clip of your show, set the project as 1080p, 29.97.  And set it up for an export to encode.







I can export 1 min of video in 25 seconds.  And most of that is the overhead of the audio.  I can export an hour long show in about 6 mins.  (specs are 1080p, 29.97). It's not magic.  It's just a professional workflow.  QT references files, and such - it would be even faster if I had used Avid's AMA importer.

And before you say "oh, well that was just a simple clip"  here... a 3 camera talking head shoot,



That's a way more complicated timeline then you'll ever do on the EEVblog.

I can't help to notice that you keep complaining about not getting "pro" performance out of a "consumer" set up. After editing 700 videos, the learning curve to some new software should be minimal.  And with that, you can actually use a supported video card, and all that jazz.  And with that, you'll get a "pro" workflow.  Trust me...it's all about the work flow. And the right tools let you do that.

[dexters_lab]

Did you see it on the same two recent EEVBlog videos I reported, or on other non-EEVBlog YT videos?

no idea tbh, i dont pay that much attention to video glitches!

which ones did you report?

[Grapsus]

Dave, I might have an explanation for CPU2 slowness. This machine you built is a NUMA architecture. It's almost like two separate computers with a fast interconnect. Each CPU has it's own memory controller and its own RAM. When CPU2 tries to access CPU1 RAM it is significantly slower than when it accesses its own RAM. In a NUMA architecture the memory space is absolutely non-linear with respect to access time. That's why extra precaution must be taken in order to fully exploit such a machine. When a program or the OS is not aware of the non-linearities in memory space, performance can drop a lot.

I don't know how it works on windows, but on Linux you can set an affinity for certain CPUs and certain memory regions on NUMA architectures, it solves the problem when you have several processes sharing the resources. For a single process I don't think there is a way around it if the software is not aware of NUMA and doesn't schedule its threads and memory accesses in a certain way...

[Dany J.]

Hello everyone wanted to join the community since a long time, well now i did after this video i hope i can help everyone with my knowledge and learn from you guys, though i wish there is a proper introductions forum.

anyhow i just wanted to share with you my setup since now you got a True graphics card you can boost the rendering speed tremendously using OpenCl offcourse.

the main thing here is to use the MainConcept codec it may seem slow horrible but when an AMD card is around it's awesome. first of all make sure you install the latest Omega catalyst from AMD. Enter movie studio then go to "Options" "preferences" then select the "video" tab and make sure your graphic card can be selected in there, select it and restart movie studio.. now the real test is use the same video you used as a benchmark, choose the MainConcept AVC/AAC then internet video 1080p and customize it with these settings:

-res 1920x1080
-profile high
-framerate 50 (untick auto adjust)
-untick use deblocking filter
-choose constant bitrate (i recommend i MAX of 10Mbps, although you can choose whatever you like but youtube will compress it to max 5-6mbps)

go to system and click check gpu you should get OpenCL is available..( not GPU, not AMD, not Cuda)
and finally choose render using OpenCl if available.

Sorry for the details but honestly it's a pain to see my i5-3570k along with a radeon HD6950 perform wayyyyy better than a dual Xeon, R9-290 PC.

Dave you should see a huge boost for real, try it and let me know we should figure this out my machine isn't better than yours lol

[Fungus]

Dave: I'd seriously give Cyberlink PowerDirector a try instead of Sony...

I just rendered a 9 minute 22 second video video in 2 minutes 13 seconds on my Intel i5@3.5GHz, that's 4.2 times real-time speed.

Video was 1080p@25fps (to match my camera).

Yes, you render video at twice the frame rate but you also an have an i7 with twice as many cores as my i5, it should work out about the same. If you're currently rendering at 0.5 times real-time then you should end up 8 times faster than before.

------------------------------------------

Update: Just for a laugh I downloaded this entire 53 minute blog video and re-encoded it at 1080p@50fps. It took 15 minutes 52 seconds and my CPU usage was 50% the whole time - must only be using two cores (I had the 'Hardware encoder' option selected).

[firewalker]

Is necessary to export a re-encoded video from the editing software, since you are using a two step process?

Alexander.

[Fungus]

Is necessary to export a re-encoded video from the editing software, since you are using a two step process?

If he's adding overlays, then, yes...

[firewalker]

Can the program export to a "fast format" with minimum compression? Yes the file would be crazy big.

I never had to play around with so demanding formats, and lengthy videos.

Alexander.

[Towger]

Here we go. Here we go. Here we go...  They have come out of the woodwork with their comments on how mine is faster/better than yours.

Dave,

Time to run a competition. Upload the raw test clips and get them produce a video on how they edited them together and show how fast they rendered to them to the required formats/size.

[jazz]

Does the vfw Version of x264 not work with Movie Studio? http://sourceforge.net/projects/x264vfw/ (since that's basically what he's using to render it in the end anyway)
I'm guessing it either doesn't work, or there are other major downsides to using it? Sorry if it was mentioned elsewhere and I just missed it.

[dexters_lab]

Can the program export to a "fast format" with minimum compression? Yes the file would be crazy big.

I never had to play around with so demanding formats, and lengthy videos.

Alexander.

it can yes, but 'crazy big' would be a major understatement

just test rendered something on mine... 18Gb per minute of video

[Fungus]

Time to run a competition.

Maybe.

Dick-waving about hardware won't be very useful (nobody's going to buy a new machine) but software comparisons could be worthwhile. I chose PowerDirector after comparing it with Premiere Pro. Rendering speed was a big factor, plus it's a lot simpler to use for 'blog' type videos.

I'm guessing Dave really wants a super codec for Sony Movie but I'd say take a look at PowerDirector if you haven't tried it.

get them produce a video on how they edited them together

For PowerDirector:

1) Drop the clips in the 'bin' at top left
2) Drag them to the timeline, trim the head/tail of each one (if needed)
3) Right-click the audio and select "normalize track"
4) Go to the 'produce' screen, choose your compression preset, render it.

Optional steps (it only takes a few minutes so it's worth doing):
5) Load the output of step 4, extract the audio to a .wav file (right-click->"Extract audio")
6) Compress the dynamic range using Audacity.
7) Replace the audio in the video.
8 ) Render again - this step goes really fast because only the audio needs to be compressed (the software is smart enough to not recompress the video track if it matches the output format).

[mariush]

Dave, lots of suggestions and stuff here.

Did you consider to output the whole video as 59.94 fps instead of 50 fps when you're using two cameras?  It's probably easier to add a bunch of frames to make 60 frames out of 50, compared to dropping frames... you should check that out. Might decrease rendering time.

With this new machine, you should check again lossless codecs like Magicyuv : http://magicyuv.com/index.php/download/magicyuv for the initial rendering.  It's a heavily threaded codec, it would love those 12-24 cores and it would compress output fast but the downside is for a 1080p 60fps video, the bitrate of the output is probably going to be in the range of 80-100 mbps.  Don't matter, since you're going to recompress anyway, right? 

You just have to render to avi and select the codec for video and leave uncompressed audio and afterwards handbrake will recompress to h264 and aac or mp3 audio.  And you should disable the live preview window while rendering the video.
 

[v81]

Hi Forum, Hi Dave, DS1054z arrived today
My 1st scope
My 1st post

Saw a link to the forum whilst typing the below on youtube, thought i might be better here....


Seeing heaps of comments re "try plugging in the additional power connector".
It's not going to happen as the PSU involved does not have the connector, only comes in the 850W and higher models (check qty of EPS connectors in TechSpecs tab.
It might be a good idea to upgrade considering the workload involved should fully load both CPU's whilst in progress, hence why not make maximum power available?

I do understand your reasons for not using the GPU, thumbs up on that.

RE: Hyperthreading, it has no significant benefit in this scenario.
You will likely find it has no effect on the i7 if you turn it off there too.
It is fairly normal for the HT cores to not appear fully loaded in some cases.
Best practice when theorizing speed is to forget the HT exists and count using physical cores, IE your i7 has 4 x 3.xGHz, new rig has 12x2.6GHz

I think i can rustle up 8 sticks of near matched (2gig?)DDR3 memory, can confirm in the morning.
If you're interested in trying it let me know.

I think RAM + CPU power + SSD source&destination (i know you don't wanna hear it) along with some method tweaking will net you some excellent results.
Ultimately i think we should see all 12 cores fully utilized through every step where multi-threading could possibly be used.

If your hard disks are not the bottleneck at any point then i really believe you've yet to find the best method/codec to do this.

I think you are yet to see the best of this dual socket beast, don't give up on it yet.

You came across on the video as if to say "My way is the best way" and didn't sound too keen on hearing any opinions, but maybe consider that some of us actually are good at this kind of thing (but i promise i suck at engineering, 1st ever scope arrived today, i'll be watching your other vids to learn to use it!).

.....

Other things i'd try....

1)Try looking for bottle necks in Resource Monitor, i usually open it from the performance tab of task manager.
Check the various tabs, i would head straight to the Disk tab and see if any disks are near or at 100% Active time (blue line in graphs).

2)Rotate the CPU fans as you considered.
The thermal paste install method was A1 in my opinion if a little much in qty, but you mentioned that.

3)Fire up the intake fans to encourage fresh ambient air into the case.
If you'd like to cut back on fans then make it the exhaust fan, the warmer air will find its way out due to both convection + positive case pressure from intake fans (leave the top open).
The cpu fans will also encourage airflow in the right direction.
The intakes will also provide airflow to the PSU/GPU if/when needed, possibly stopping the GPU from ramping up its noisy fans (they are only quiet whilst they are slow!)

Please be kind, i'm new!

[fourier]

I've been interested in how much problems you've been having trying to get great performance out of your machines for what seems to be a relatively simple project.

Dave, is there any chance you can upload the 1 minute project test + rendered file and your encode settings somewhere? I really feel there's better ways to get faster performance out of your projects than just throwing massively overkill hardware at it, but I can only experiment if I have some test files to work from.

[andiz]

Hi everyone,
my first posting here. I was struggling for some time (no, not another forum, time consuming, etc.) but finally i decided to register and contribute to this nice forum. I think the benefits are much higher than being a passive reader.

@Dave:
Don't waste your money in getting faster RAM for your setup. It isn't worth at all, because in terms of video encoding (and 90% of all other workloads), the speed increase you can expect from faster memory than DDR3-1333 is 5%.

Have a look at:
http://www.anandtech.com/show/6372/memory-performance-16gb-ddr31333-to-ddr32400-on-ivy-bridge-igp-with-gskill/11

Tests from other reviewers lead to the same results so I think you can rely on that.


Greetings from Germany...

[v81]

.....Did you consider to output the whole video as 59.94 fps instead of 50 fps when you're using two cameras?  It's probably easier to add a bunch of frames to make 60 frames out of 50, compared to dropping frames... you should check that out. Might decrease rendering time....

Only theorizing here, but i'd have thought dropping frames would take less effort than creating frames?

[dexters_lab]

Does the vfw Version of x264 not work with Movie Studio? http://sourceforge.net/projects/x264vfw/ (since that's basically what he's using to render it in the end anyway)
I'm guessing it either doesn't work, or there are other major downsides to using it? Sorry if it was mentioned elsewhere and I just missed it.

it does work, quality of the output is good and more optimised like the output from handbrake, but vfw just makes things seem clunky... .avi files, separate audio codec needed

i tried it on mine but it was slower than the built-in codecs

[Corporate666]

Dick-waving about hardware won't be very useful (nobody's going to buy a new machine) but software comparisons could be worthwhile. I chose PowerDirector after comparing it with Premiere Pro. Rendering speed was a big factor, plus it's a lot simpler to use for 'blog' type videos.

I'm guessing Dave really wants a super codec for Sony Movie but I'd say take a look at PowerDirector if you haven't tried it.

Dave has already said he is not interested in switching software or dramatically changing his workflow.

Everyone has an idea of what he could do differently to speed things up, but many of the suggestions are just false (like "you need SSD's" or "you need more RAM!" or "It's your video card!").

Now it's even looking like the Xeon suggestion isn't working out either.

I do think the root cause of all this is Sony Vegas sucks... but if that's the constraint he's imposed on his workflow (which is completely reasonable, IMO), then any suggestions have to work within that.

[dexters_lab]

RE: Hyperthreading, it has no significant benefit in this scenario.
You will likely find it has no effect on the i7 if you turn it off there too.

really? on my i7 vegas is 30% slower rendering with HT turned off.

[dexters_lab]

I do think the root cause of all this is Sony Vegas sucks

you could be right there, i like the editor it's seem quick and powerful to me, it's just their output codecs seem  to let it down, if they could get the output quality and effeciency as good as the handbrake engine we wouldn't need to render it twice

[salviador]

Why not the new i7-5960x ??

[DanielS]

Another thing to keep in mind: LGA2011 CPUs have a quad-channel memory controller so unless you put four similar DIMMs on each CPU, you are only enabling half of the RAM bandwidth each CPU is capable of. This could be a massive bottleneck when all 24 threads are enabled.

[bktemp]

Maybe Dave could tweak the compression settings in Sony Vegas for more speed: In most H264 encoders it is possible to select between different levels of efforts the software takes to reduce the data rate. Using a faster setting gives a lower quality and a larger filesize, but since filesize does not matter because it gets recompressed afterwards using Handbrake the lower quality could be compensated by using a higher bitrate. I would also try using PCM instead of AAC as audio format.

[Corporate666]

I do think the root cause of all this is Sony Vegas sucks

you could be right there, i like the editor it's seem quick and powerful to me, it's just their output codecs seem  to let it down, if they could get the output quality and effeciency as good as the handbrake engine we wouldn't need to render it twice

When I was choosing software, I tried Vegas, Avid and PowerDirector.  I settled on PD as the one I liked best - I found Sony just to be awkward.  I actually find all these programs to be too geared towards the casual user and they try to dumb down their interfaces which makes for frustration for experienced PC users.

Having said that, didn't Dave say in one of his previous videos that someone from Sony had contacted him to say that some option or other shouldn't be grayed out and was going to look into it?  Anything ever come of that, Dave?