New AMD Ryzen Threadripper Editing Machine

[EEVblog]

I did see that for NVENC rendering, the Pascal Quadro's can do 9 streams of H.264 1080P high quality frames simultaneously or 21 streams of the Highest quantity of frames. For consumer GPU's, NVENC limits it to just 2 streams. Sounds like nVidia has crippled the non-Quadro Pascal based cards like the GTX 10X0 series.

I know someone who I think has hacked this...

[kalel]

If anyone knows, does AMD have similar limitations using their codec?

[Someone]

If you're not doing anything complex then the major NLEs all work the same, you drop the media into your pool, create clips on the timeline and trim them to size with blades and then compose with ripples/slips/rolls. The shortcuts are largely common for these operations and have very little difference in how they are used.

In theory, yes, in practice, no.
Take Premiere for example, everything raves about so I tried it for few videos. Because of the way it does modes or whatever, I had to do twice as many clicks to do that same basic task as in Vegas.
I confirmed with Premiere users that's "just the way it works". Well screw that, I'm not going to add hundreds of annoying clicks to my workflow.

Having personally used most of the NLE packages I don't see big differences, but you need to use keyboard shortcuts and not just the mouse UI.

I know. I was assured by experienced Premiere users that keyboard shortcuts didn't help here, and scripting wouldn't work. It was fundamental in the way Premiere did want I wanted it to do, it was simply more work than Vegas.

You keep referring out to some very specific action which you didn't like but wont expand on it, but its easy to find some aspect which works differently or less efficiently going between any given examples. We see this all the time in people here comparing some niggle they find in a specific brand of test equipment and they go on and on about it without looking at the bigger picture.

This is a thread where you are asking for assistance in speeding up your outputs, most video editors worry about having enough speed to do realtime interactive editing/colouring and leave it at that because rendering is "solved" by the majors (pro end software) as it can be farmed out to multiple compute nodes if speed is needed. Last I remember you were doing an intermediate render and then further compression with handbrake to reduce upload time, hasn't your internet connection changed a few times recently? Time to reassess the value of further compression? Or look at better ways to do compression, while keeping your editing flow in Vegas.

[EEVblog]

You keep referring out to some very specific action which you didn't like but wont expand on it

Because I don't want to waste my time going through explaining it all again, and I forget exact details, it's on the forum somewhere.

, but its easy to find some aspect which works differently or less efficiently going between any given examples.

But this was MY EXACT WORKFLOW editing an actual EEVblog video.

This is a thread where you are asking for assistance in speeding up your outputs

No, I asked for comment on the Ryzen processor.
Now please drop it.

[tszaboo]

So you recommend I do that for what end benefit?

IDK, but some people, particularly, coming from the software world have this habbit. You ask a question, they tell you that you should throw away all your tools, all your work process, and switch to something completely different. Like you have a problem with a SQL database query, and they answer you that you should change to a completely different database engine, with different servers and everything. Like that is going to fix your issue. Or you want to fix a .bat file, and you are supposed to switch to Linux. Or you are looking for a x IC for your project with a PIC18, and suddenly you are supposed to switch to PSOC.

Just throw away everything that you are doing now, and switch to their beloved system.
It is infuriating. It tells a story, that the person asking the question is a clueless moron, because he is not using their beloved system. While the story is, person giving advice has no idea how much a change costs. Time, money effort. Someone told me the other day, I should stop using Altium. I told him that I have a few thousand hours experience with it, so it would take me years to switch to another software.

[borjam]

Just throw away everything that you are doing now, and switch to their beloved system.
It is infuriating. It tells a story, that the person asking the question is a clueless moron, because he is not using their beloved system. While the story is, person giving advice has no idea how much a change costs. Time, money effort. Someone told me the other day, I should stop using Altium. I told him that I have a few thousand hours experience with it, so it would take me years to switch to another software.

I am surprised because editing video without really complex stuff should be a no brainer nowadays. It was ten years ago as well.

So, maybe there is one bad step in his workflow that is causing the headaches. The most obvious one is of course the format of the source video. It's much more efficient to batch transcode it to a format intended for edition like Apple's ProRes (pardon the Apple mention, I guess Adobe and otehrs have their own intermediate format) or, with enough disk bandwidth, to uncompressed video.

Anyway, just trying to be helpful and of course not intending to infuriate anyone. And I am sure others share the same position. I will drop it.

[EEVblog]

So, maybe there is one bad step in his workflow that is causing the headaches. The most obvious one is of course the format of the source video.

The format of the input files makes zero difference to the editing workflow in the NLE.
Unless you chose to transcode them first, then it's of course adding an extra step to the workflow.

[Someone]

If you're not doing anything complex then the major NLEs all work the same, you drop the media into your pool, create clips on the timeline and trim them to size with blades and then compose with ripples/slips/rolls. The shortcuts are largely common for these operations and have very little difference in how they are used.

In theory, yes, in practice, no.
Take Premiere for example, everything raves about so I tried it for few videos. Because of the way it does modes or whatever, I had to do twice as many clicks to do that same basic task as in Vegas.
I confirmed with Premiere users that's "just the way it works". Well screw that, I'm not going to add hundreds of annoying clicks to my workflow.

Having personally used most of the NLE packages I don't see big differences, but you need to use keyboard shortcuts and not just the mouse UI.

I know. I was assured by experienced Premiere users that keyboard shortcuts didn't help here, and scripting wouldn't work. It was fundamental in the way Premiere did want I wanted it to do, it was simply more work than Vegas.

, but its easy to find some aspect which works differently or less efficiently going between any given examples.

But this was MY EXACT WORKFLOW editing an actual EEVblog video.

I think you are confusing workflow with tools/methods. From what I understand from your previous explanations and videos your workflow is:

1 shoot in sequence in camera (no external anything)
2 copy from media to local drive
3 drop the lot onto a timeline and edit
4 export to compressed intermediate format
5 handbrake to further compressed format for upload
6 send to youtube
7 mark up and annotate after upload

2, 4, and 5 are all easy targets for changing the workflow that don't involve any change to your editing methods or tool. The proposed change of computer is again (same as last time https://www.eevblog.com/forum/chat/i-tried-a-mac-for-video-editing/?all) only about steps 4/5 as the others are already limited in other ways.

If you don't like some particular way another NLE edits thats fine, Vegas is what you're comfortable editing with and works best for you. But when trying to save time there are many other steps in the workflow that can be sped up (or even eliminated) by considering other NLEs to do those tasks.

Whatever show stopping issue you had with Premier is of no particular interest to discuss, but you keep bringing it up as the blockage to consider the tool when its sounding like some minor UI difference in how to do a very specific action. A user coming from the other direction could likely find similar scale problems in trying to do their work exactly the same method they are used to in tool X with Vegas instead. Some times the same result can be achieved in many different ways and trying to force one way of working onto a tool which has other better ways of doing the same thing is silly, we see it again here with the various test equipment threads that devolve into camps of "the one true way to do it". You put it out there that you've "tried them all" when you clearly haven't made serious efforts to learn how to best use other tools available as in the previous thread linked above. And tools change over the years, what might have been impossible to work around may no longer exist as a problem.

[hayatepilot]

The way I see it, the core problem seems to be that the motherboard (drivers...) are not properly supported on WIN10.
This causes the problems that dave is experiencing like the boot issues, crashes and inconsistent performance of the PC.

Despite very powerful hardware the software can't leverage it.
Switching to different, newer hardware that is supportet for WIN10 could improve the reliability and consitency of your workflow.

If this is a Threadripper or Intel i9 probably doesn't really matter all that much.

Maybe it's worth testing on a spare PC if VEGA is slow/buggy there too before investing in new powerful hardware.

[borjam]

So, maybe there is one bad step in his workflow that is causing the headaches. The most obvious one is of course the format of the source video.

The format of the input files makes zero difference to the editing workflow in the NLE.
Unless you chose to transcode them first, then it's of course adding an extra step to the workflow.

Modern editors can edit H.264, true. Final Cut, for example (sorry but it's the NLE I have used) works on the H.264
files while it transcodes then in background. Once transcoded to ProRes it switches to working with ProRes, which is
much more efficient. You only notice that, suddenly, everything is much more agile and your CPU usage can go down.

Remember that delivery codecs such as H.264 are not a stream of full frames, but a stream of some complete frames called key frames
and differences between them. While locating a certain frame in an edition capable codec is trivial, it's an expensive operation. But there
is yet another reason to transcode to an edition program before editing. You are not sure how many encoding/decoding tasks
are being performed on your clips if you make changes such as adjusting color, adding fades and whatnot. Remember that video compression
is lossy.

If you batch convert to an edition or uncompressed format you have a much better control on the quality of the final product. The video
editor program, whichever you use, will always achieve the top possible performance for your hardware. Moreover, the edited master video in
intermediate format will be the equivalent of a high quality photographic negative. What if next year you need to remaster your current H.264
files to a different codec like H.265? (Just using the names as examples)

I only edit video rarely but I have edited a lot of audio, done live recordings and even mixed/mastered some releases. In the audio world you
record uncompressed audio at the best possible quality and encoding to AAC or MP3 is the very last step. Editing video in a delivery codec,
despite being possible, is the equivalent to mixing and mastering audio in MP3. 

[Someone]

I only edit video rarely but I have edited a lot of audio, done live recordings and even mixed/mastered some releases. In the audio world you record uncompressed audio at the best possible quality and encoding to AAC or MP3 is the very last step. Editing video in a delivery codec, despite being possible, is the equivalent to mixing and mastering audio in MP3. 

There are people using a full RAW workflow (not what Dave calls raw but RAW bayer data from the image sensors) who do store lossless information all the way to compression for delivery but its usually too costly for small productions. Dave has found that he gets acceptable quality for his material with rather modest data rates in the camera so the source material is already heavily compressed and a compressed intermediate isn't such a problem other than the wasted time. But you'll get best quality if you go through as few lossy stages along the way as possible which is why rendering direct from the source material (no matter the codec or bitrate) is always best.

Modern editors can edit H.264, true. Final Cut, for example (sorry but it's the NLE I have used) works on the H.264 files while it transcodes then in background. Once transcoded to ProRes it switches to working with ProRes, which is
much more efficient. You only notice that, suddenly, everything is much more agile and your CPU usage can go down.

FCP (X) can actually use the MTS sources directly without transcoding:
https://discussions.apple.com/thread/3154831
As can the other majors. There are several choices for the internal project/cache/editing format in FC and they can greatly affect the CPU use but most modern video editing machines are fast enough that decoding several 1080p streams during editing is no longer a substantial strain on the system and you can disable caches if you know the load wont be high.

[borjam]

There are people using a full RAW workflow (not what Dave calls raw but RAW bayer data from the image sensors) who do store lossless information all the way to compression for delivery but its usually too costly for small productions. Dave has found that he gets acceptable quality for his material with rather modest data rates in the camera so the source material is already heavily compressed and a compressed intermediate isn't such a problem other than the wasted time. But you'll get best quality if you go through as few lossy stages along the way as possible which is why rendering direct from the source material (no matter the codec or bitrate) is always best.

RAW data from the sensors would be a huge overkill. But a format with independent frames is a must. Clearly something is wrong if Dave is having performance problems. There are compressed edition formats as well, it's just that the compression isn't as aggressive as the one
employed on delivery codecs. And, again, frames are independent

[Someone]

There are people using a full RAW workflow (not what Dave calls raw but RAW bayer data from the image sensors) who do store lossless information all the way to compression for delivery but its usually too costly for small productions. Dave has found that he gets acceptable quality for his material with rather modest data rates in the camera so the source material is already heavily compressed and a compressed intermediate isn't such a problem other than the wasted time. But you'll get best quality if you go through as few lossy stages along the way as possible which is why rendering direct from the source material (no matter the codec or bitrate) is always best.

RAW data from the sensors would be a huge overkill. But a format with independent frames is a must.

It used to be but isn't anymore, decode is fast enough that you can scrub and the computer fills in the frames quick enough even if it needs to decode and then discard the intermediate frames which are never shown.

[borjam]

RAW data from the sensors would be a huge overkill. But a format with independent frames is a must.

It used to be but isn't anymore, decode is fast enough that you can scrub and the computer fills in the frames quick enough even if it needs to decode and then discard the intermediate frames which are never shown.
[/quote]
Except when it doesn't. Maybe something is wrong with Dave's setup. I/O bandwidth?

Anyway, as in any other fields, compromise is sometimes a bad idea. A single compromise probably won't do much damage, but the effect of several different compromises or poor decissions don't usually add up, but multiply.

All of this is pure speculation on my part of course. Only one thing is certain. "Our" Dave has some really annoying video edition problem.

[Naguissa]

But were you able to solve the windows problems?

No. I cannot upgrade to the latest release, it just locks up at 33% install regardless of how long I leave it.
And every time I boot it locks up, I have to hard power off and start again and it always works a 2nd time.

Same happened (and maybe happens) to my child's Turion. Maybe Spectre/Meltdown patches, as my case?

Enviado desde mi Jolla mediante Tapatalk

[nctnico]

So you recommend I do that for what end benefit?

IDK, but some people, particularly, coming from the software world have this habbit. You ask a question, they tell you that you should throw away all your tools, all your work process, and switch to something completely different. Like you have a problem with a SQL database query, and they answer you that you should change to a completely different database engine, with different servers and everything.

This is a rather extreme way of putting it. Sometimes when a tool doesn't get you where you need to be it is better to switch to a tool which works than keep on making do and wasting your time. In many cases it won't be easy to switch so there have to be clear advantages.

[EEVblog]

But there is yet another reason to transcode to an edition program before editing. You are not sure how many encoding/decoding tasks
are being performed on your clips if you make changes such as adjusting color, adding fades and whatnot.

On every editor I've used, the answer is zero.
The NLE editor does not alter the source files, it simply stores what you want done to them. Only when you play it or render it does it actually apply the changes and create the resultant file.
For those interested, I shoot in AVCHD and edit directly on those files.

What if next year you need to remaster your current H.264
files to a different codec like H.265? (Just using the names as examples)

Easy, you just re-render in H.265 from the original source files you worked from.

[Mr. Scram]

On every editor I've used, the answer is zero.
The NLE editor does not alter the source files, it simply stores what you want done to them. Only when you play it or render it does it actually apply the changes and create the resultant file.
For those interested, I shoot in AVCHD and edit directly on those files.


Easy, you just re-render in H.265 from the original source files you worked from.

Do you keep everything?

[amspire]

I have found out some info on the nVidia video encoding. The NVENC encoders in the later nVidia cards are hardware encoding engines, and they are beasts. The GTX1060 uses the GP106 chip - it has 1 of these engines. The GTX1070, 1080 and 1080i uses the GP104 chip and it has 2 NVENC engines. If you get a Titan with the GP100 chip - you get 3! 

When I say they are beasts, on the Pascal cards, they can render H.264 or H.265 to 8K. How fast are they? I believe one NVENC engine can render 1080P H.264 at the high quality setting at about 350 frames/sec.

The problem is keeping these beasts fed. So with the Quadro chips, you can run 9 video streams to try and keep two of these engines busy.

Now nVidia seems to be really strict about the rendering license with NVENC and so for non-quadro cards, you can only run 2 streams. When I say they are really strict, NVENC drivers are only available using the closed source nVidia software and will never be open source. And get this - no matter how many GTX1080's you have, you only get 2 streams. If you have one Quadro that can manage 9 streams and 3 GTX1080 cards, you get 9+2=11 streams!

Ok - so how fast are the consumer cards anyway?

I downloaded the open source MediaCoder package as it has NVENC in it and rendered Dave's 2 minute video. On my slow Phenom II pc with the GTX1080 graphics, it was just under 1 minute for both H264 and H265 encoding with the CPU running at 100%. MediaCoder is far more efficient then the Magix codecs and here are the render times I believe you can get on a faster PC with a 1080 card:

H.264:  20 to 25 seconds for the 2 minute video
H.265:  30 seconds for the 2 minute video

By the way, I also tried CUDA rendering from MediaCoder and it is much more fragile and slower. I never actually finished a render. NVENC kills it for both speed and robustness.

Now on my PC, the Vegas 15 rendering is a sadder story. With the slow processor, it takes me 3 minutes to render Dave's video with the Magix NVENC codecs. Dave on his dual Xeons can get down towards 1 minute.

Definitely, the Magix encoders like to take their time. So are we stuck with that?

No, you can run Vegas with other encoders.

With Avisynth, you can render out of Vegas to a file that is actually a Avisynth stream. Nothing actually touches the hard drive at all. The output of the stream can go to the MeGUI transcoder or MediaCoder and you can directly call the NVENC encoders from there. The format you pick to render out of Vegas is usually the Video for Windows DV format or YUV format that generates about 10G per minute of video, but that 10G is just streamed through to the NVENC encoder.

I cannot tell you how fast Vegas is at outputting the YUV format as I haven't set up an Avisynth stream, and I have a 5 year old WD Blue as the drive. Just writing the 22G YUV AVI file to the drive takes about 6 minutes.

If this method did work well, you can probably use Vegas 14 instead of Vegas 15 if you like 14 better. Both have the Video for Windows codecs.

I did see that for NVENC rendering, the Pascal Quadro's can do 9 streams of H.264 1080P high quality frames simultaneously or 21 streams of the Highest quantity of frames. For consumer GPU's, NVENC limits it to just 2 streams. Sounds like nVidia has crippled the non-Quadro Pascal based cards like the GTX 10X0 series.

I know someone who I think has hacked this...

It could work, but nVidia are really keen to protect their licensing. I can't help feeling that the closed source NVENC software with do a thorough job determining the type of card and probably will not be fooled by a hacked card.

The 2 stream restriction is in the nVida NVENC software on the PC and not in the card. It does look like the non-Quadro card has everything it needs to run the same number of streams as the Quadro. If the hack works, then 

[borjam]

But there is yet another reason to transcode to an edition program before editing. You are not sure how many encoding/decoding tasks
are being performed on your clips if you make changes such as adjusting color, adding fades and whatnot.

On every editor I've used, the answer is zero.
The NLE editor does not alter the source files, it simply stores what you want done to them. Only when you play it or render it does it actually apply the changes and create the resultant file.

Of course it doesn't alter the source files. I'm talking about intermediate changes.

What if next year you need to remaster your current H.264
files to a different codec like H.265? (Just using the names as examples)

Easy, you just re-render in H.265 from the original source files you worked from.
[/quote]
Isn't it easier to have a "negative" master copy, already edited, and just encode it into the final delivery format without
even launching the editor? If one of my victims asks for a MP3 version of a track I've mixed I don't need to launch
the multi track editor. I just get the final mix and encode it.

[EEVblog]

Of course it doesn't alter the source files. I'm talking about intermediate changes.

There are zero intermediate changes if you render direct from the NLE.

Isn't it easier to have a "negative" master copy, already edited, and just encode it into the final delivery format without
even launching the editor?

Sure, if you want to store a massive lossless rendered copy for that purpose down the track then knock yourself out.

If one of my victims asks for a MP3 version of a track I've mixed I don't need to launch
the multi track editor. I just get the final mix and encode it.

Video is kinda bigger in file size than audio.
IIRC a 2TB hard drive will only hold a handful of lossless rendered original videos of the length I typically do. And I have over 1200 videos.
I keep all the original camera files, and that already takes up enough room.
I actually used to keep high bit rate original "master" renders back when I had to heavily compress with handbrake to get the file sizes down so they wouldn't take all day to upload.

[EEVblog]

If this method did work well, you can probably use Vegas 14 instead of Vegas 15 if you like 14 better. Both have the Video for Windows codecs.

I've experimented with that many times but have always had issues.
You can also do this with x264 (that Handbrake uses) for a constant quality output file just like handbrake, but I've had issues with this too.

[EEVblog]

Do you keep everything?

Yes, I have original camera files from the very first episodes.
A few have been lost, but I would have over 1200 videos worth of original files.

[dryjoints]

Dave, if you'd diverted all the energy which you've put into this thread, into buying a VERY POWERFUL iMac (Apple have a returns policy - no quibble), sitting down and TRYING the fully working trial version of Final Cut Pro X, you'd be making FAR more headway than here, where you're just demonstrating what an utterly pedantic perfectionist luddite you seem to be. All this waffle time could be spent TRYING to be open minded - I don't think you want anyone's "help", you just want to keep chasing your tail and then wondering why things don't work out.

When you take your blinkers off and TRY something which the majority of editors use (and I do not JUST mean FCPX), then you might actually move forward instead of jogging on the spot getting nowhere. How do you KNOW there is no better way when you're clearly unwilling to sit down, turn OFF the internet and the "opinions" of your precious band of followers, and TRY FOR YOURSELF and form your OWN opinions, after giving it a week or two of HARD investigation? You know as well as I do that you are a ranter and a very impatient person, quick to jump to conclusions and deride anything that doesn't work EXACTLY as you expected... well maybe it's time to have a good long talk to yourself, quit complaining that "It's not like Vegas does it, blah blah..." and TRY?

I hope it works out for you, and that we don't come back in 2 years time to see a re-hash of the same kind of thread (this is clearly an ongoing issue, you've been yammering away about this for years... sheesh... sort yourself out, you're A PROFESSIONAL!)

[Mr. Scram]

Dave, if you'd diverted all the energy which you've put into this thread, into buying a VERY POWERFUL iMac (Apple have a returns policy - no quibble), sitting down and TRYING the fully working trial version of Final Cut Pro X, you'd be making FAR more headway than here, where you're just demonstrating what an utterly pedantic perfectionist luddite you seem to be. All this waffle time could be spent TRYING to be open minded - I don't think you want anyone's "help", you just want to keep chasing your tail and then wondering why things don't work out.

When you take your blinkers off and TRY something which the majority of editors use (and I do not JUST mean FCPX), then you might actually move forward instead of jogging on the spot getting nowhere. How do you KNOW there is no better way when you're clearly unwilling to sit down, turn OFF the internet and the "opinions" of your precious band of followers, and TRY FOR YOURSELF and form your OWN opinions, after giving it a week or two of HARD investigation? You know as well as I do that you are a ranter and a very impatient person, quick to jump to conclusions and deride anything that doesn't work EXACTLY as you expected... well maybe it's time to have a good long talk to yourself, quit complaining that "It's not like Vegas does it, blah blah..." and TRY?

I hope it works out for you, and that we don't come back in 2 years time to see a re-hash of the same kind of thread (this is clearly an ongoing issue, you've been yammering away about this for years... sheesh... sort yourself out, you're A PROFESSIONAL!)

Considering it's been ages since Apple has updated the Mac Pro, it isn't really a recommended option for professionals. Many pros have abandoned Apple the past years, as its obvious Apple is shifting ever more towards the consumer and away from professional use. The 5K iMac you mention is basically a regular mainstream i7 system with a good screen. That's actually nice, but nothing really pro oriented. The thermal side of things is obviously not optimally suited for calculation intensive workloads. That honestly makes your push for open-mindedness, or "you should do exactly as I say" as some would call it, a bit puzzling. Are there any tangible benefits other than just "it's better" and "act like a pro" that give you reason to push the matter so relentlessly?

The link posted below is a pretty good reflection of the sentiments of quite a few people who earn their keep with video.

https://www.joelwsmith.com/blog/my-switch-from-mac-to-pc-for-video-editing