Multi Threads In Node.js
The Async Challange in Nodejs
Nodejs purpose was to allow fast input/output (io) and not to be used as a computation layer for processing. Despite this important fact, in one of the projects I was in involved, we were required to check a possiblity for image processing (crop & resize) in a nodejs layer which is also a rest api layer.
The solution had to use a nodejs async solution which will not block the event loop while processing the images.
I’ve looked into async lib and web worker lib.
I asked few team members at Tikal to share their experience with async operations in nodejs.
I was interested to know whether async lib actually opens new sub processes (like web worker does) and whether it actually doesn’t block the event loop, so it can do parallel work.
Few solutions were raised in order to deal with image processing in nodejs.
graphicMagick is nodejs module for image manipulation. It exposes an impressive api for manipulating and even comparing images through nodejs.
However, this solution requires ImageMagick to be installed on the local machine along side nodejs - so we had to drop this solution. Also, it was recommended that the machine should have more than one core so the task can be run in parallel to the nodejs api service.
synchronize.js is a nodejs module that allows writing asynchronous code as if it’s synchronous. It allows to do parallel work.
Cloud Based Solution - cloudinary.com
cloudinary.com is an online image cloud service.
Async & Parallel Processes in Node.js
The 2nd challenge that came out of this task was how to handle parallel processes in nodejs.
An important note that was raised: async lib does not spawn new processes and is used for flow management. The only way to have CPU intensive tasks performance in a non blocking way is to spawn a new process. Any other way will just delay the blocking to the next event loop tick.
Eventually, and wisely enough as I understand, we decided to not use any actual multi thread image processing with nodejs but rather use a micro service in rabbitmq which from there, will handle the task.
However, the good thing that came out from this process is the exposure to the web worker lib that i’ve found for nodejs (it works in chrome browser).
Feel free to share you experience with multi processing in nodejs.