Author Topic: A question around responsibilities maintaining a large LAN (Ethernet).  (Read 854 times)

0 Members and 1 Guest are viewing this topic.

Offline peteb2Topic starter

  • Regular Contributor
  • *
  • Posts: 245
  • Country: nz
Love to have some light on this issue please. Situation: An IT department provides and acts as maintenance point-of-source for support on a reasonably large Local Area Network. Lots off servers, lots of clients ...switches, routers both cable & fibre etc.... a fairly big setup, typical of a Media Corporate.

Sharing bandwidth on this Network are a set of application specific servers and related client equipment which is entrusted to an engineering/electronics technician group whose duty it is to provide support to the folks that rely on this equipment to do their job. Any breakdown can result in large amounts of lost revenue.

Recently a pattern of breakdown began occurring. Lucky the backup system covered the sudden issue but soon it may not.  The usual fault finding included having log file records of events make by this specific system itself,  checked by the support dept of the company that originally supplied the application specific gear....

The diagnosis revealed what was possibly a sudden random loss of data at specific times so using the only available tools (WireShark) by the engineers, checks were made of just what was happening on the Network itself & pretty soon it could be seen a likely Packet loss/delay...

Now here's the quandary. Requesting assistance from the IT (Network system administrators) to provide proper technical tests of the LAN performance resulted in; "well xyx Pings ok so everything is ok".

My rant here as a maint-tech/engineer come benchie.... surely an IT Dept team endowed with the responsibility of managing a mission critical Network, would include a skill set of knowing exactly what is happening at the '1s & 0s' level of data transfer and for there to be some kind of actual TEST GEAR that is permanently or temporally connected 'listening' and monitors for packet loss, packet corruption, bit error rates or whatever needs to be done which would then indicate a good or randomly goofing Network?

I have no idea what IT folks do. Myself on the other hand, (a tech) surely they must be able to use some kind of test equipment that 'sniffs' any sort of LAN no matter how big right down to the raw signals of Ethernet to confirm that data is ALWAYS being passed with it being lost of corrupted by a switcher or bad RJ45 patching cable or at least that the Specifications (whatever they may be for Ethernet) are being adhered to?????

Clearly i need to go read up on 'Ethernet 101' but you can't tell me that "xyz Pings ok so the Network is 100% fine"  is a proper fault find when plainly something is wrong...

Looking forward to any input, (how do you do a proper technical analysis of Ethernet activity?) or am i thinking down the completely wrong path? TIA   :palm:

 

Online Someone

  • Super Contributor
  • ***
  • Posts: 5155
  • Country: au
    • send complaints here
Re: A question around responsibilities maintaining a large LAN (Ethernet).
« Reply #1 on: February 05, 2021, 10:36:39 pm »
IT is an enormous field, you're thinking of the narrower concept of a "network engineer" or similar. Trouble is the corporates/managers running that network might be either too small to have specialists, or have gotten by without them so don't see the need. Yes, there are a range of tools to test/diagnose these problems and some of them are embedded in those sorts of networks running full time to monitor and control issues.

Push it back up to your boss/management and ask for details of the network providers performance contract. (issue might be at your end where someone bought a link that isn't contracted to meet your needs for media streaming).
 
The following users thanked this post: peteb2

Offline nctnico

  • Super Contributor
  • ***
  • Posts: 28429
  • Country: nl
    • NCT Developments
Re: A question around responsibilities maintaining a large LAN (Ethernet).
« Reply #2 on: February 05, 2021, 11:00:53 pm »
In my network maintaining days I'd just replace the RJ45 patch cables and bin the old ones. In 99.9% of the cases that fixed the problem.
There are small lies, big lies and then there is what is on the screen of your oscilloscope.
 
The following users thanked this post: peteb2

Offline JohnnyMalaria

  • Super Contributor
  • ***
  • Posts: 1154
  • Country: us
    • Enlighten Scientific LLC
Re: A question around responsibilities maintaining a large LAN (Ethernet).
« Reply #3 on: February 05, 2021, 11:44:12 pm »
If the network is in a leased building, the IT department may not even have jurisdiction over the network infrastructure. Their responsibility may end at "well, it pings okay".

In my experience of working in a large corporation, "IT" means the computer on your desk, the software thereon and ensuring the security of the company's digital assets rather than the routers, cables etc.
 
The following users thanked this post: peteb2

Offline vad

  • Frequent Contributor
  • **
  • Posts: 518
  • Country: us
Re: A question around responsibilities maintaining a large LAN (Ethernet).
« Reply #4 on: February 05, 2021, 11:58:14 pm »
Use some packet loss monitoring software to measure packet loss, latency and latency jitter between the affected hosts over time. Issues at physical layer (poor cabling, failing switch port, failing NIC port, etc) will show up on the report.
« Last Edit: February 06, 2021, 12:14:37 am by vad »
 
The following users thanked this post: peteb2

Offline nightfire

  • Frequent Contributor
  • **
  • Posts: 587
  • Country: de
Re: A question around responsibilities maintaining a large LAN (Ethernet).
« Reply #5 on: February 06, 2021, 12:23:26 am »
As a Sysadmin and having some stations in my career including a datacenter some years ago, I can provide some insight whats typically happening inside such operations.

To make it short: Today most companies and Admin positions are quite shortstaffed in contrast to the workload they should theoretically provide. And systems tend to be more complex every year, which also goes for stuff like security.
And frankly: Most of todays IT staff only have very basic knowledge of the stuff deep down of the network stuff thats going on, and not enough diagnostic tools at hand (and usually not enough time) to do their job properly.

So maybe you are still stuck with some 1st level engineer that follows some simple procedures and cannot see what really is happening.
Correct procedure here would be to look at some switches and their statistics, what also could be another adventure giving being able to access those.
Debugging packet loss occurred some time in my career, and i also have encountered some situations even when really highly qualified network engineers came to the conclusion that some expensive Rack-Switches in a datacenter maybe were lying in their counters or themselves not being capable to cope with some stuff on physical level.
 
The following users thanked this post: peteb2

Offline xmo

  • Regular Contributor
  • *
  • Posts: 193
  • Country: us
Re: A question around responsibilities maintaining a large LAN (Ethernet).
« Reply #6 on: February 06, 2021, 04:28:31 pm »
 JohnnyMalaria wrote:  "In my experience of working in a large corporation, "IT" means the computer on your desk,..."

And those guys have an extensive set of capabilities including "ping", "reboot that", "check for updated drivers", and "reinstall Windows"

An actual network administrator for anything more complex than a simple 'flat' system should have all sorts of ways to look for problems.

In a "mission critical" network, every network device is managed.  Routers, switches, security appliances, etc - every one has a unique configuration file.  Network management software like Castlerock or Nokia NSP ties it all together.  All the devices use SNMP to report status and issues to the management software.

The network manager uses the software to monitor the health of the system, configure new devices, update existing devises, and diagnose problems.  The network manager should also have a 'toolkit' of instruments from companies like Fluke and Netscout for testing everything from bit error rate on fiber links to certifying network cabling infrastructure, patch panels, etc.

Should have anyway....

 
The following users thanked this post: peteb2


Share me

Digg  Facebook  SlashDot  Delicious  Technorati  Twitter  Google  Yahoo
Smf