Fast web PC

[ricko_uk]

Hi,
I need to buy a PC for heavy web browsing using several chrome plugins for websites pricing data comparison. The idea is to run hundreds of Chrome tabs each with its own plugin (at least 20 different types of plugins).

I tried with a couple of PCs (with 16Gb of RAM) but the CPS and memory get filled up quickly and the entire PC slows down to the point of taking 10-15 seconds to open a new tab and a minute to launch something simple like MS Word or even a new browser window.

All I need is just the PC, no monitor/mouse/keyboard. Preferably not more than 700-800 GBP (IF possible by prioritising the components that make the most difference)

A couple of questions:

1) what specs would I need to have a ultra fast solution for the above requirements? Which processor? Would more cores make a difference? Which is best AMD or Intel? What about type of RAM and type of SSD? Which of them makes the most difference for my application?

2) where is the best place to buy PCs nowadays in the UK? Amazon? Ebuyer? Others?

Also, I used to assemble my own PCs years ago but not sure that by the time I spend half a day assembling one the price difference is worth it? Any feedback?

Thank you

[Marco]

A second hand HP workstation with as much memory as you can fit in your budget?

[RenThraysk]

Would look into using headless chrome/chromium, and puppeteer, if needed to something more complicated than simple web scraping.
Puppeteer should be able control multiple chromes running on multiple PCs rather than trying to run 100s on just one.

[ricko_uk]

Thank you RenThraysk and Marco!

I looked up puppeteer but could not understand how I would use it for my application. Could you please give me a brief explanation?

It looks like I would need to write some code. Is that correct? I have zero experience in writing code for web...

Many thanks

[NiHaoMike]

A used server will work nicely. But before going there, have you tried upgrading the RAM and optimizing?

Some easy optimizations I can think of:
* Lightweight OS, for example Xubuntu.
* Adblocker, for example Ublock Origin.
* Disable media playback in browser.
* If your Internet access is fast and unlimited, lower the browser cache setting.
* Use Noscript to only enable the Javascript required for the sites to work.

[RenThraysk]

Thank you RenThraysk and Marco!

I looked up puppeteer but could not understand how I would use it for my application. Could you please give me a brief explanation?

It looks like I would need to write some code. Is that correct? I have zero experience in writing code for web...

Many thanks

Ah. Perhaps I don't understand what you are trying to accomplish. With puppeteer can completely automate chrome. So could go to each site and extract the pricing information, and then go onto the next. Rather than have 100s open simultaneously.
But yes, need to code, so a no go solution.

[ricko_uk]

Thank you all

What I need to do is open the plugins on very many Chrome tabs. Main issues are:
- when launching each plugin, it takes a while to load and download all data (sometimes up to a minute) so instead of waiting there I immediately open other tabs and start opening other plugins
- after a while the computer slows down immensly
- I have to look at the data and manually download it and often compare it so have to keep very many plugin windows/popups open
- zero experience programming for web/browsers etc

Any further suggestions? With those further clarifications, do I still need just tonnes of memory to make it all run smoother?

Thank you again

[MK14]

You are making it sound like the real problem is the huge/overloading plugins, you are clogging it up with.

I get the impression, that for secrecy reasons, privacy or worries about embarrassment, you don't want to say what you are really trying to achieve with your web browser.

Analogy: Someone is putting a trailer with 20 tons of stuff in it, behind/attached-to their normal car. Then complaining their car is way too slow. The real solutions are either to get a lorry, only take what can fit into the boot of the car, or hire a haulier to take the 20 tons of stuff to its destination for you.

If your existing computers are reasonably powerful (which they sound like they are), then a faster computer and/or more memory, may not really solve your problem.

Plugins can be severely security limited by the Google Chrome browser (I get the impression, they don't want plugins really), they can (presumably, I'm not 100% sure), be using a system, called scripts/interpreted languages. These can be extremely slow and take a very long time to run. In a way, which a faster computer can't really help with much. (i.e. fundamental limits on single thread performance).

One or more of your plugins may be faulty and/or very poorly written and/or being used in a way, which severely overloads your computer, hence the slow downs.

I'd also be worried, your computer(s) are getting infected with stuff (e.g. spy-ware), which may be making the problem considerably worse.

But, more memory might help, it is not really clear what the real/exact problem is. Details about what you are really trying to do, are too vague.

Most things can be achieved without using plugins, but without knowing what you are trying to do, it is difficult to help you, here. Even if you do need a better/faster computer, it is best to know what the real problem is.

[MK14]

This seems to explain some of what I was trying to say:

https://www.kaspersky.com/blog/browser-extensions-security/20886/

EDIT:
This one is probably too complex to easily understand, but it is another example:
https://www.i-programmer.info/news/86-browsers/8491-chrome-42-outlaws-java-plugin.html

What I also should have said. Is that it could well be the various plugins, insistence on sending lots of internet data via their own servers, that could be causing much of the delay. If so, a faster computer wouldn't really help, make the internet any faster, as it would be the external servers that are really slowing you down.
Even a super fast connection, wouldn't necessarily help, if the real problem is because of the plugins, as regards stuff external to your computer e.g. other peoples servers, for servicing the plugins.

[bd139]

I feel this is an "XY problem"

http://xyproblem.info/

I've automated similar processes using python. I've got a arbitrage scraper box that runs 24/7 throwing about 10,000 requests an hour out on a fanless celeron with 4gb of RAM and it sits about 2-5% utilisation and doesn't take any of my attention unless it finds something interesting

[janoc]

This definitely is an XY problem - or rather "if the only tool is a hammer, treat everything as a nail".

Normal web browser is going to be totally unusable and crashy with hundreds of tabs running plugins, regardless of what monstrous machine you try to run this on. Build a web scraper, that can easily run even on a RaspberryPi.

[ricko_uk]

Thank you MK14, bd139 and janoc,
 
MK14,
maybe that's what's happening.

bd139 and janoc,
how do you build a web scraper? I assume you do it programmatically but as per my previous post I don't have experience in any web-based programming. Are there other ways of doing it maybe simple high level block-based systems similar to some of NI's tools? Or perhaps extremely simple scripting?

[bd139]

Scripting. I use python + requests + BeautifulSoup: https://www.crummy.com/software/BeautifulSoup/bs4/doc/

Routine is:

1. get the page with requests
2. parse the content with BeautifulSoup into python objects
3. do whatever magic you need to do there.

[olkipukki]

Or perhaps extremely simple scripting?

https://en.wikipedia.org/wiki/Python_(programming_language)

Python is an interpreted, high-level, general-purpose programming language.

Just learn it once. 

[ricko_uk]

Thank you bd139 and olkipukki

[eugenenine]

Hi,
I need to buy a PC for heavy web browsing using several chrome plugins for websites pricing data comparison. The idea is to run hundreds of Chrome tabs each with its own plugin (at least 20 different types of plugins).

I tried with a couple of PCs (with 16Gb of RAM) but the CPS and memory get filled up quickly and the entire PC slows down to the point of taking 10-15 seconds to open a new tab and a minute to launch something simple like MS Word or even a new browser window.

MS Word = running MS Windows.  You don't need a new pc you just need a new OS.

[bd139]

But that wont run word.

Hell I use word (and excel - fight me) on windows 10 and I'm a through and trough Unix/Linux dude

[eugenenine]

so run windows/excel/word in a virtual machine for those few times you have to use them.

[ebastler]

You don't need a new pc you just need a new OS.

Are you implying that running a browser with a couple hundred tabs and different plugins, but under Linux, is the solution?
Or did you just feel the need to start some OS dispute? 

[bd139]

One trick pony has one trick!

so run windows/excel/word in a virtual machine for those few times you have to use them.

Run linux in a virtual machine for those few times you have to use them.

(I have a whole ubuntu based kubernetes cluster running on my desktop at the moment inside virtualbox, on windows)

[eugenenine]

You don't need a new pc you just need a new OS.

Are you implying that running a browser with a couple hundred tabs and different plugins, but under Linux, is the solution?
Or did you just feel the need to start some OS dispute? 

I have nearly the same laptop for work and home, Dell Latitude E7250 and a Dell latitude E7550 (E72xx is 12" screen, E75xx is 15" screen).  Work laptop has windows and 16G of RAM, Home laptop has Linux and 8G of RAM.  I can run 10-12 tabs in Firefox on the work laptop and Chrome barely runs on it.  Home laptop I have hundreds of tabs in Firefox Chrome at the same time.  Home laptop has the slower emmc drive interface but work has faster SATA.  Home laptop is many times faster than work laptop despite having less ram and a slower disk.
There are very different ways the OS's manage memory.  After Windows 2000 Microsoft removed the registry key that allowed you to change the 'swappiness' of the OS and Windows likes to leave some ram free to "speed up loading programs" as they say.  Linux will let you use all available ram before it swaps if needed.  The linux OS is that much more efficient.
Way back when I had windows 2000 running on a work desktop I could run 4-5 vmware clients running windows 2000 server.  I had a small DC, web server, app server, sql server and sometimes a win2000 or xp client to duplicate the configuration the company was using.  Upgraded to XP on that box and it could barely run two of the same vmware guests.
I really think MS made the change intentionally to make XP and later less efficient so they could sell their hypervisor platform. 
But whatever the reason Windows hasn't been the same since.
I setup a dual boot on my son's gaming pc and he was always amazed at how many more 'mods' he could run in Minecraft under Linux than he could under Windows.
You really should try it out, it makes a big difference in usability.

[brucehoult]

You're pretty clearly running out of RAM. Web browsers are huge pigs with memory and 16 GB is really a very small machine now. Get 64 GB or 128 GB or more. The CPU is unlikely to matter much but anyway you can get a hugely fast 12 core AMD 3900X for under $500 now.

But max out the RAM first.

[ebastler]

I have nearly the same laptop for work and home, Dell Latitude E7250 and a Dell latitude E7550 (E72xx is 12" screen, E75xx is 15" screen).  Work laptop has windows and 16G of RAM, Home laptop has Linux and 8G of RAM.  I can run 10-12 tabs in Firefox on the work laptop and Chrome barely runs on it. 

Thanks for taking the time to elaborate. But I cannot relate to your experience: I use a modest laptop at home (i5 CPU, 8 GByte RAM, 4 or 5 years old). While I don't keep 100s of tabs open, I do use a couple dozen or so, and I have never even begun to notice any performance issues. Using Firefox mostly, but Chrome runs just as fine. Does your work laptop get dragged down by some corporate security/antivirus package, maybe?

But more importantly, I think this thread had already moved on to discussing better, script-based solutions to the OP's problem -- better not just from a performance perspective, but also regarding robustness, automation and maintainability. That was the main point of my comment: Even if Linux should bring performance improvements, sticking with the browser-based scraping solution appears like a kludge.

[Jeroen3]

To answer the initial Y question:

For the purpose of running many chrome tabs, just get the most core count Ryzen available in your budget.
Most of the budget will go to memory anyway.
Also get a fast writing nvme ssd, a consistently fast one! Many also collapse in speed after writing a few GB.
Maybe even consider nvme raid for those valuable disk iops.

[brucehoult]

To answer the initial Y question:

For the purpose of running many chrome tabs, just get the most core count Ryzen available in your budget.
Most of the budget will go to memory anyway.

Normally I suggest 4 to 8 GB of RAM per core.

1 MB of RAM per MIPS of CPU has been a good rule of thumb for general purpose computing since the 1970s. Modern Out-of-Order CPUs average around 2 MIPS/MHz for each core.

If it's all web browser tabs (which are memory heavy and CPU-light) I'd double that.