Node JS programming for masochists

[nigelwright7557]

Recently got into Node JS website programming.
A very steep learning curve but managed to get a local host working on my PC.
You add functions from a library as you go along like pop up alerts etc.
I found a lot of what I needed to know online and youtube etc
Once I got it working  I tried to get the website online.
Not a chance !
Amazon Web services too k10 minutes to upload it successfully only for it not to run.
So tried Azure which was much faster.
That didnt run either.
I started splitting down my software to find what didnt run.
I found  popup alerts dont work online.
A getmac stopped it working too.

I then found a few bugs in my code as things didnt seem to be happening in the right order.
I found out that a lot of Node JS is asynchronous like database accesses.
So had to get into "promises" that do functions when processor is idle.
It them started getting complicated with promises inside promises.
Then  I had to work out what should be done inside or outside the promises.

The biggest pain is the Javascript debugging.
Visual Studio only does a syntax check without an error list.
You have to look through your code a green squigly line under the error which could be anywhere.
So it pays to make small changes so yo ucan quickly find any syntax errors.

The other major problem is debugging screen comes up then disappears quickly if you have a bug.
Not very helpful as you send debugging messages to the console !

So if you really hate yourself get into Node JS !

[SiliconWizard]

I for one would never touch that with a stick.

Oh, and I don't feel bad. The world is not missing out. There are some many people out there willing to do Node JS. I can do something else.

[szszoke]

Look into async/await to avoid dealing with promises inside promises a.k.a. callback hell.

[nigelwright7557]

Look into async/await to avoid dealing with promises inside promises a.k.a. callback hell.

I tried async/await and the software just crashes on them.
Node JS does its best to stop any blocking.

[Syntax Error]

@nigelwright7557 A 'promise' is just another fancy term for XHR callbacks, which is just another fancy term for http request replies. As with any async callback pattern, care should be taken with the expected response and any error handling. Plus keeping every code block inside the JSON-like program structure. [ Suddenly a missing comma means something! ]

Personally, I have many reasons to hate Javascript as I've been fighting it for over 20 years! In my opinion, any language that is not typesafe is not a language. For example, is a variable a string, a float number, a signed integer number or, all of them throughout runtime? You cannot be 100% certain. This lazyness is starting to creep into other languages < why let the programmer decide if a number is a uint32 if they never learned what an unsigned integer was at university?

The inconvenient truth about Javascript is this, JS was conceived as a means to manipulate the Document Object Model (DOM) of an HTML web page; read node attributes by ID, alter CSS styling, enhance the user experience, that sort of shtuff. It was never intended as a language to replace server-side PHP. Sadly, with the advent of 'frameworks' and other code bloated Google code solutions, Javascript is now the on goto language for the IT industry.

Tip: if you want a stupid salary at a time when there is a shortage of skilled programmers, carry on stessing with your Node.js

[nigelwright7557]

I havent done a lot of javascript before so I was taking on javascript and Node JS at the same time.
One thing I did like was JSON. Cookies are already in JSON format so that makes working with them easier.
Another thing that caught me out was html being defined before it was used in my code.
The punctuation is a nightmare.
I have admit I did on occasions just put in random punctuation if I got a squiggly line error.
Sometimes it worked and others I had to go back through code and ensure correct indentation.

I used Visual Studio community and this was often crashing on power up after the code appeared but before it got coloured in.
I had to downlaod the standard VS instead which was fine.
I put in a ticket to Microsoft complaining about the crashing and the lack of list of lines with errors in the lower window.
At least VS worked well with Azure. The database added in very easily.
I did try AWS but got warned pretty quickly I was going to get a bill for hosting despite not spending long on it.
They said I had used 750 hours which is nonsense so I just unsubscribed.

[szszoke]

Look into async/await to avoid dealing with promises inside promises a.k.a. callback hell.

I tried async/await and the software just crashes on them.
Node JS does its best to stop any blocking.

They are not supposed to do that. If you can post a snippet that crashes, and the exception that is thrown, I should be able to point out the cause.

[nigelwright7557]

Look into async/await to avoid dealing with promises inside promises a.k.a. callback hell.

I tried async/await and the software just crashes on them.
Node JS does its best to stop any blocking.

They are not supposed to do that. If you can post a snippet that crashes, and the exception that is thrown, I should be able to point out the cause.

The async await works fine offline but once uploaded to Azure crashes.
It looks like they reject alert confirm or anything that does a pop up.

[bd139]

I’ve got a fairly large nodejs deployment to keep alive. It’s a complete fucking shit show pile of cancer of a platform. Stay away.

I’d you want slightly less pain it’s probably better to deploy it manually behind nginx on an actual real box. Linode/digitalocean will rent you one.

Appropriate image attached

[szszoke]

I’ve got a fairly large nodejs deployment to keep alive. It’s a complete fucking shit show pile of cancer of a platform. Stay away.

I’d you want slightly less pain it’s probably better to deploy it manually behind nginx on an actual real box. Linode/digitalocean will rent you one.

Appropriate image attached

I'd say that is less of an issue with he platform and more of an issue with whoever developed your NodeJS deployment.

[bd139]

Definitely not. We’re not some small outfit. We’re pushing 10k requests a second off it with over 1M lines of it Real problems:

1. Deadlocking galore
2. Package management is a total shit show. Especially when it comes to finding which combination of 400 packages simultaneously works and doesn’t have more holes than a Swiss cheese (CVEs)
3. Really half the libraries are buggy and written by PHP developers who jumped ship who didn’t have a clue about software engineering to start with.
4. Incompatibilities between versions of components is constant DLL hell again.
5. v8 GC isn’t that good which means it spikes ram under load. This can cause OOM conditions under load heads.
6. All assurance is at runtime. So you have to run full test packs to test even minor changes for API breaks. Full integration test runs take 40 minutes so fucks up your turnaround time on delivery pipeline.
7. npm is full of malicious shit.
8. No precision types without slow hack job libraries.
9. Don’t even start me on the build tool chain which was written by fucking morons, then several different build chains spawned off written by different morons
10. The whole callback/continuation programming model is just shit. So difficult to unpick intent from everything with this plus quite frankly it doesn’t scale as well as per thread worker pools at large scale.
11. All the IDEs and tooling are shit because it’s almost impossible to effectively do type inference.

And yes this turd is being sliced up and ported away. Total sunk cost.

Edit: at least MS tried to kill off some of the excrement with TypeScript.

[Syntax Error]

@BD139 I've just been reading how to track down OOM issues with Node.JS...

I was expecting some clever Visual Studio/IDE breakpoint thing. No, instead it's insert a memory dump function in the suspect Javascript module and call it with a Timeout. Or use the Linux shell PS command to track greedy node processes. Seriously, this is built for a production/live runtime environment? I've read the hype about node being the only web service in the universe that's "none blocking", but really, I would not expect to find in my syslogs messages like javascript heap out of memory.

I've built a few thread safe HttpRequestHandlerFactory classes in my time, all of which performed asynchronously. That's async file I/O, async dataset creation, async DB table record locking, async DB table rollback, async security and crypto, async response write, async making the tea...

I'm guessing the advantage of node to the enterprise is it's basically free. All the enterprise needs is hire some post grad node nerd who's super excited that there's a package that stops render blocking javascript. And then all that's needed is to intergrate node with React or Ember frameworks. Yes Ember, that's a whole new sh1t show!

[bd139]

Actually that's not too bad. Look for node-inspector. That gives half decent memory profiling from v8.

The issue for us is we're on Kubernetes so the first sign of trouble we get is the kubernetes pod being evicted due to OOM which is when node bangs against the hard limit. In fact a lot of time is spent debugging the difference between pod resource limits and process limit tuning. node will quite happily puke when it reaches an arbitrary limit (I forget what it is - think 1.4Gb or something) so you then have to give it a certain amount of headroom just in case it can't count, which is a regular issue. That works out ~1Gi per pod. When you have 100 of them, that's 100Gi of RAM being wasted in your cluster which has a $$$ impact.

This sort of stuff is why I much much much prefer Go as a programming language. We have glorious runtime profiling and sampling https://golang.org/pkg/runtime/pprof/ and the allocator respects ulimits nicely. Also it has decent concurrent GC which means we don't end up with a few seconds here and there when a pod stops responding to traffic. Also no fecking async. It's all coroutines and channels which are far easier not to deadlock (and it has deadlock detection if you do screw that up).

And yes don't even get me started on Ember. I know a guy who was working with that who just quit and is starting a jacket potato van because he's "had enough of working with that sort of shit" 

[Syntax Error]

  Heaping hell! I now understand memory leaks are normal with node.js  It's quick on a domestic media server, but it just doesn't scale for enterprise deployments; where it's being deployed. A turd to be avoided. Thankyou EEVBlog you have been most educational.

For anyone interested on why node is such a memory hog:

Memory Leaks in NodeJS a quick overview
https://medium.com/tech-tajawal/memory-leaks-in-nodejs-quick-overview-988c23b24dba

Node.js memory limitations. Or how to blow up your app in 100 easy steps
https://medium.com/@ashleydavis75/node-js-memory-limitations-30d3fe2664c0

Node.js memory management in container environments
https://medium.com/the-node-js-collection/node-js-memory-management-in-container-environments-7eb8409a74e8

ps Good luck with his "love spuds" wagon.

[Stuart Coyle]

The OP gets it right, Javascript is hell. Why anyone thought putting javascript on the server was a good idea is beyond me.

[magic]

If all you know is a hammer...

[AtomicRob]

Wow, a lot of node js hate... It's a tool, like any tool it's good for some things and not for others. When you run into trouble like this the question to ask is, did we make the wrong architecture/platform choice, or is it the right platform choice but poorly implemented? As a hobbyist or student this is a good learning experience. In a professional setting this is sort of a lose-lose situation. You don't want to go back to the product manager with those two options

[nigelwright7557]

Wow, a lot of node js hate... It's a tool, like any tool it's good for some things and not for others. When you run into trouble like this the question to ask is, did we make the wrong architecture/platform choice, or is it the right platform choice but poorly implemented? As a hobbyist or student this is a good learning experience. In a professional setting this is sort of a lose-lose situation. You don't want to go back to the product manager with those two options

Node JS in my case was a necessity as I needed a none blocking responsive website.
I am not having a dig just relaying my experiences with it.
Its been a few weeks now since starting with it and i am slowly getting the hang of it.
Getting to grips with promises and async/await ok now.
My website is now doing pretty much what I wanted.
I didnt want to use mysql as my cheap web hosting doesnt support it to so moved over to MSSQL.
Took a while to get into it and got stuck a bit on interpreting data from MSSQL.
For some reason they embedded the JSON inside an extra level called recordset which I didnt know about until I stringified the JSON structure to the console.
Still all working well now.
I got in touch with Microsoft over poor JS debugging and they improved it in for me.
At least I now get a list of errors to work from.

[filssavi]

While JavaScript and node do have rough spots and downsides, I feel the hate here is totally undeserved

There are examples of fairly large companies that you might know (Netflix, PayPal, Uber…) that make extensive use of nodeJS, so evidently, if the application is architected correctly nodeJS is capable of handling plenty of traffic (now hard numbers are fairly hard to track down, but I am willing to bet that it is more than 10k requests/s). Of course when it comes to true scaling the language the server app is written in does not matter, as long as the infrastructure architecture (where scaling really happens) not have any bottlenecks.

The dependencies problem is absolutely nothing specific to JS, when dealing with security critical code, all the dependencies should be known and trusted. I can just as well copy paste crappy C code full of memory leaks, buffer overflows, and so on, but I don’t blame the C language for it. Moreover all language package management repositories (pypi, npm, etc) are more or less equally vulnerable to supply chain attacks, as they are typically run on shoestring budgets (with respect to their appeal to malicious actors) And cronically understaffed (running the package repo is not that glamour of a job)

JS, is a tool, there are things it is very good at doing like asynchronous event driven programming for strictly IO bound tasks (where having dozens of  threads is only a waste of memory and OS resources, as they will all be blocked while waiting for a DB query to get back) things it is more or less ok at like Purely CPU bound tasks ( where ince the JIT kicks in is not too far Back with respect to C) and things it is absolutely atrocious at, like anything memory intensive, or god forbids memory bound. It is up to the software engineer/architect to choose the right tool for the job

After all theoretically I could build a 20kW inverter with 2n2222 BJT, or an operational amplifier out of discrete high voltage IGBTs. Il would be very difficult and they would have terrible performance, however blaming BJTs or IGBTs for that is childish.

Now if youe boss forces you to do it or you inherit the maintenance of such a thing then yes I would be salty as well, however it still is not the language fault.

P.S. IMHO the OP should slow down and do thing at a time, first learn JS/node and the various modern web standards you need to use, then learn how to deploy your stack in a local machine (either bare metal or VM) and how to automate the deployment. And then and only then you can even start to think moving everything to the cloud, otherwise you are debugging 2/3 completely separate and buggy software components, interacting in unpredictable ways on somebody else’s computer (aka the cloud)