10GBASE-T - Too much speed for home use?

[SiliconWizard]

At home I switched to a 1Gb LAN around 2006 IIRC. Yeah.
Still not on 10Gb. But I plan on redoing my LAN with fiber, 10Gb or over. Still has a cost.

[ejeffrey]

The only 'domesticated' use case for such bandwidth is a $10,000 MacPro video studio, with 8K video stored on a SAN/Raid array loaded with M2 EVO ssd modules.

That's pretty much as big of an exaggeration as the person who needs > 1 Gb networking for their 4k tv.

A commodity file server can easily exceed 1 Gb even with ordinary mechanical hard drives much less SSDs.  If you have a local file server and  either multiple wired clients or a single client with a 10 G nic you can easily can easily use that speed.  And multi gigabit internet options are... Not super common but not exactly rare either. 

Do you "need" that?  Depends, but probably not.  But it's definitely possible to use that bandwidth with relatively common hardware.

But outside of the Hollywood hills, there is little justification for this level of investment.

What level of investment?  10GBase-T NICs are under $100 and switches < $50 / port.  It's not as cheap as 1 Gb networking equipment but it's not a huge premium to build a 10 G network.  I think that if you are building a network that will have a home file server you should strongly consider 10 G. 

btw My fastest outbound-upstream link is a mere 100Mbps on a 330Mbps FTTP link. When I started networking, 1Mbps over coax was regarded a top enterprise level speed - and was priced accordingly.

Good for you.

[Halcyon]

As much as I bag out the National Broadband Network (NBN) in Australia, largely based on previous years of butt-hurt, I'm fortunate enough to have fibre to the premises now.

It's not uncommon to see Gigabit (or higher) WAN links to homes these days depending on which part of the world you live in. Commencing this year, NBN are already planning to expand residential fibre services beyond the 1 Gigabit limit.

Consumer routers such as those made by Asus and Netgear are already offering 10 Gigabit ports and 802.11ax Wi-Fi is already capable of saturating a Gigabit link.

[py-bb]

Europe. Lol where'd you get Afghanistan from?

Well you're flying their flag.

sorry what?

[py-bb]

A modern 15krm SAS drive will give you 370mib/sec (320mib/sec on track) speeds give or take. So you'll see an average of 320mib/sec but it's really 370mib/sec with pauses for the track changes.

You can saturate network connections much more easily with SSDs though, SAS comes in a few speeds and is usually dual connector - so SAS 12gbps (bit) (gen III - we're on 24 which is really 22.5 in main deployment - but most drive stuff is still 12gbps max really) will give you 24gbps per SSD (both connectors) - now you can saturate stuff.

And the big proper enterprise SSDs are your best chance at them sustaining that performance too.

[NiHaoMike]

One use case I can think of would be wireless VR, in combination with 802.11ay or whatever short range, high bandwidth Wifi standard becomes popular by then. In fact, 10G might not be enough for high resolutions and 25G or 40G might be needed to make it work.

[mariush]

Europe. Lol where'd you get Afghanistan from?

Well you're flying their flag.

sorry what?

Change the country / flag from your profile ... it shows up on the left side

[metebalci]

A modern 15krm SAS drive will give you 370mib/sec (320mib/sec on track) speeds give or take. So you'll see an average of 320mib/sec but it's really 370mib/sec with pauses for the track changes.

You can saturate network connections much more easily with SSDs though, SAS comes in a few speeds and is usually dual connector - so SAS 12gbps (bit) (gen III - we're on 24 which is really 22.5 in main deployment - but most drive stuff is still 12gbps max really) will give you 24gbps per SSD (both connectors) - now you can saturate stuff.

And the big proper enterprise SSDs are your best chance at them sustaining that performance too.

SATA SSD yes more easy to saturate 10G and PCIe SSD can saturate even 40G easily, even the not performance oriented PM9A3 I am using has >6GB/s sequential read spec. I was talking about HDDs.

[metebalci]

At work, I have a new NAS with 8 Enterprise HDDs  as Raid6, and this easily can saturate the 10 Gbit/s link in linear operation. No problem here, depends a bit on the bang the CPU delivers for checksum calculation.

Just wondered if this is hardware raid, Linux mdraid or zfs ?

[py-bb]

A modern 15krm SAS drive will give you 370mib/sec (320mib/sec on track) speeds give or take. So you'll see an average of 320mib/sec but it's really 370mib/sec with pauses for the track changes.

You can saturate network connections much more easily with SSDs though, SAS comes in a few speeds and is usually dual connector - so SAS 12gbps (bit) (gen III - we're on 24 which is really 22.5 in main deployment - but most drive stuff is still 12gbps max really) will give you 24gbps per SSD (both connectors) - now you can saturate stuff.

And the big proper enterprise SSDs are your best chance at them sustaining that performance too.

SATA SSD yes more easy to saturate 10G and PCIe SSD can saturate even 40G easily, even the not performance oriented PM9A3 I am using has >6GB/s sequential read spec. I was talking about HDDs.

SATA SSD wont, they're limited to 6gbps which is ~500mib/sec give or take (you'll see 480mib/sec at least in practice) - nowhere near the 1.10gib/sec of 10G

[madires]

When you don't need 10Gbps, but a bit more than 1Gbps would be nice, then link aggregation could be an alternative. There are a lot of inexpensive semi-managable SOHO switches supporting LACP/LAGs. NASes with two ethernet ports usually support it too, and all modern OSes anyway.

[bitwelder]

One more caution for 10 Gbps copper cables: if the line is terminated at some SFP+ transceivers, do check that the transceivers you use are rated for the length of cable you plan to use.
We happen to have a 10Gbps copper line which as measured by the switch should be around 40m, and we used transceivers rated for 30m.
The 10 Gbps link works, but the switch detects some transmission error.

[sokoloff]

When you don't need 10Gbps, but a bit more than 1Gbps would be nice, then link aggregation could be an alternative. There are a lot of inexpensive semi-managable SOHO switches supporting LACP/LAGs. NASes with two ethernet ports usually support it too, and all modern OSes anyway.

Link aggregation increases aggregate throughtput for multiple clients, but many configurations do not increase speed for single client/single connection. (That depends on the hashing algorithm chosen; if the connection used is hashed by MAC address pairing, a single client-server pair will only use one of the links.)

[Halcyon]

When you don't need 10Gbps, but a bit more than 1Gbps would be nice, then link aggregation could be an alternative. There are a lot of inexpensive semi-managable SOHO switches supporting LACP/LAGs. NASes with two ethernet ports usually support it too, and all modern OSes anyway.

Link aggregation increases aggregate throughtput for multiple clients, but many configurations do not increase speed for single client/single connection. (That depends on the hashing algorithm chosen; if the connection used is hashed by MAC address pairing, a single client-server pair will only use one of the links.)

Correct and a common mistake people make. Almost always it's MAC or IP based, which means even if you have a LAG set up on a client PC, the maximum throughput you'll get is that of one single NIC. Where it really becomes useful is where you have multiple clients accessing a single resource (or you want redundancy in your links).

[metebalci]

A modern 15krm SAS drive will give you 370mib/sec (320mib/sec on track) speeds give or take. So you'll see an average of 320mib/sec but it's really 370mib/sec with pauses for the track changes.

You can saturate network connections much more easily with SSDs though, SAS comes in a few speeds and is usually dual connector - so SAS 12gbps (bit) (gen III - we're on 24 which is really 22.5 in main deployment - but most drive stuff is still 12gbps max really) will give you 24gbps per SSD (both connectors) - now you can saturate stuff.

And the big proper enterprise SSDs are your best chance at them sustaining that performance too.

SATA SSD yes more easy to saturate 10G and PCIe SSD can saturate even 40G easily, even the not performance oriented PM9A3 I am using has >6GB/s sequential read spec. I was talking about HDDs.

SATA SSD wont, they're limited to 6gbps which is ~500mib/sec give or take (you'll see 480mib/sec at least in practice) - nowhere near the 1.10gib/sec of 10G

Naturally I meant multiple SATA SSDs. I just meant it is easier to saturate 10G comparing to HDDs, which has almost the half-speed.

[metebalci]

When you don't need 10Gbps, but a bit more than 1Gbps would be nice, then link aggregation could be an alternative. There are a lot of inexpensive semi-managable SOHO switches supporting LACP/LAGs. NASes with two ethernet ports usually support it too, and all modern OSes anyway.

Link aggregation increases aggregate throughtput for multiple clients, but many configurations do not increase speed for single client/single connection. (That depends on the hashing algorithm chosen; if the connection used is hashed by MAC address pairing, a single client-server pair will only use one of the links.)

Correct and a common mistake people make. Almost always it's MAC or IP based, which means even if you have a LAG set up on a client PC, the maximum throughput you'll get is that of one single NIC. Where it really becomes useful is where you have multiple clients accessing a single resource (or you want redundancy in your links).

This opens up another topic. A 10G NIC probably have to use multiple queues/RSS to be able to reach this speed. Call this queues, flows, channels, threads, all are more or less about the same topic. If you use SMB for example, it is normally single threaded, hence it is hard to reach 10G depending on the NIC using SMB. SMB v3 brings multichannel support but it sometimes does not run properly out of the box. All of these high speed network tech is mainly for concurrent access, so increasing the capacity of a single pipeline (from a single NIC of PC to another single NIC on a NAS) becomes more and more difficult after a point.

[SiliconWizard]

When you don't need 10Gbps, but a bit more than 1Gbps would be nice, then link aggregation could be an alternative. There are a lot of inexpensive semi-managable SOHO switches supporting LACP/LAGs. NASes with two ethernet ports usually support it too, and all modern OSes anyway.

Note that 2.5Gb is becoming relatively common on new gear and less expensive than 10Gb - this may be a good alternative, 2.5x the throughput.

[AndyBeez]

Thinking about this, the ethernet port on domestic hardware may go extinct - Just like Firewire did. Remember how blisteringly fast those were back in 2005? Soon everything non-wireless will connect via USB-C. Or at least, the adapters will. I intended wiring my solid walled house with Cat6 cable, drill holes, run conduit and fit mounting boxes. Just like a real network guy! Instead, I purchased a couple of former enterprise 5Ghz repeaters and I have speed everywhere. And no relationships were harmed in the installation process either. 

[madires]

Link aggregation increases aggregate throughtput for multiple clients, but many configurations do not increase speed for single client/single connection. (That depends on the hashing algorithm chosen; if the connection used is hashed by MAC address pairing, a single client-server pair will only use one of the links.)

Correct and a common mistake people make. Almost always it's MAC or IP based, which means even if you have a LAG set up on a client PC, the maximum throughput you'll get is that of one single NIC.

... in the worst case. It also depends on the specific network traffic patterns and protocols used. So it can be anything between 1Gbps and the total throughput of the LAG.

[ve7xen]

Thinking about this, the ethernet port on domestic hardware may go extinct - Just like Firewire did. Remember how blisteringly fast those were back in 2005? Soon everything non-wireless will connect via USB-C. Or at least, the adapters will. I intended wiring my solid walled house with Cat6 cable, drill holes, run conduit and fit mounting boxes. Just like a real network guy! Instead, I purchased a couple of former enterprise 5Ghz repeaters and I have speed everywhere. And no relationships were harmed in the installation process either.

This has happened already, more or less. Most people don't have any wired Ethernet devices other than their modem/router itself these days, and if you want such a port on most modern laptops, the only option is a dongle.

But wireless still sucks relative to wired, and USB-C isn't a suitable replacement or competitor for wired Ethernet, so it will live on for a long time yet, just become less and less relevant in the home setting.

[james_s]

I still use wired gigabit for everything I can, wireless is reserved for portable devices. I can easily saturate the whole wireless network by copying large files from my laptop. The wired network has switches that route the traffic so transfering between two machines doesn't bog down the rest of the network. For the average consumer it's probably not noticeable but if you're copying around tens of gigabytes of video files or whatever it becomes very obvious.

[dobsonr741]

If you have thunderbolt ext storage and want to share it to another Mac certainly would need 10G, as Thunderbolt is already at 40gig.

[Halcyon]

This has happened already, more or less. Most people don't have any wired Ethernet devices other than their modem/router itself these days, and if you want such a port on most modern laptops, the only option is a dongle.

But wireless still sucks relative to wired, and USB-C isn't a suitable replacement or competitor for wired Ethernet, so it will live on for a long time yet, just become less and less relevant in the home setting.

It can suck, but it doesn't have to. Like with everything, if you invest a bit of money in proper gear (not those stupid consumer "mesh" products), you can actually get pretty amazing performance, even in RF noisy environments with a bit of tweaking. You'll notice most consumer devices, they like to default to channels 36 to 48 and sometimes around channel 155 down the other end of the 5 GHz ISM band. They often avoid the DFS channels altogether, even if they are better.

For my set up at home, I have two internal access points (Ubiquiti U6-LR) and one outside access point to provide 5 GHz coverage throughout my backyard. I use channels 106, 138 and 155 respectively, as there are no other networks on those channels. To avoid weather radar forcing your access points to jump to a non-DFS channel, jump on your local communications authority and search their database on any transmitters within 100 kilometers on the same channel(s) you plan to use. In Australia it's ACMA and you can do a frequency range search online: https://web.acma.gov.au/rrl/assignment_range.search

I actually switched to Wi-Fi on my main TV as the 100 Mbps NIC was getting overwhelmed with some of the 4K content coming off my network (which averaged at 110 Mbps). (Why Sony decided to only include a 10/100 NIC in one of their expensive TVs defies all logic).

[madires]

The wired network has switches that route the traffic so transfering between two machines doesn't bog down the rest of the network.

Sorry for being a bit pedantic, but switching and routing are two different concepts in the networking world. Switching happens at OSI layer 2, while routing happens at OSI layer 3. A classic ethernet switch (or bridge) switches ethernet frames between segments at the data-link layer. An IP router routes IP packets between networks at the network layer. The IP packet is stacked on top of the ethernet frame. Or in other words, the IP packet is the payload of the ethernet frame. A so called L3 switch (L3 for OSI layer 3) can do both.

[M0HZH]

At the office we're running ~100 clients with Hyper-V, SAN, CRM server and a bunch of other stuff, entirely on 1Gb.

I doubt you would really need 10Gb at home for anything other than video editing on a NAS or something like that. Even 2.5Gb is hard to justify (although it's relatively cheap now).