Filed under: nodejs

Vert.x vs node.js

I kind of gave up on posting benchmarks on the web but I just found a post on the Vert.x blog with a benchmark that blew node.js out of the water so I just  did a quick run of those benchmarks and bottom line is I got ~ 60K/s using 4 vert.x instances and ~ 8K/s using 4 node.js processes.

I also ran vert.x directly using java (got the command line by looking at ps aux | grep java) in order to pass some JVM flags. Since I'm using a 64 bit OS I believe -server is on by default so I just experimented with -Xmx128m to reduce memory usage.

Here is my run of the benchmark:

Processor: Intel i7 Q820 - 4 cores + HT

OS: Ubuntu 11.04 x64

Java  1.7.0_01-b08

node v0.6.17

vert.x 1.0.final

 

Benchmark: ab -k -n 100000 -c 100 http://localhost:8080/

Mem analysis:

Using pmap: with pmap $PID | tail -1 >> mem.txt; SLEEP 0.001

Using system monitor: non scientific analysis by looking at the processes :P

 

vert.x (1 instance)

Mem (pmap): 2. 4G

Mem (System Monitor): 120M

Requests per second:   ~ 30K/s

(Added JVM flags -Xmx128m)

Mem (pmap): 446M

Mem (System Monitor): 75M

Requests per second:   ~ 30K/s

 

vert.x (4 instances)

Mem (pmap): 2. 4G

Mem (System Monitor): 120M

Requests per second:   ~ 60K/s

(Added JVM flags -Xmx128m)

Mem (pmap): 513M

Mem (System Monitor): 75M

Requests per second:   ~ 60K/s

 

vert.x (2 instances)

Requests per second:   ~ 50K/s

 

Node Server (1 process)

Mem (pmap):  ~ 640M

Mem (System Monitor): ~ 65M

Requests per second: ~ 2.6K/s

 

Node Cluster (4 processes)

Mem (pmap): ~ 640M per process ( +1 for the main process) => 3.2G

Mem (System Monitor): 4.5M + 4 * 65M => 264.5M

Requests per second: ~ 8K/s

 

Disclaimer

This is mainly for personal reference so please refrain from starting a flame war. 

I am using node.js for some current projects and really like it so this is not, by any means some node.js trashing post... just use what you like!

If you feel some of these benchmarks are unfair please perform your own and link to them in the comments... I'll gladly take a look.

Go vs Scala vs NodeJS

These test are run with:

ab -c 100 -n 50000 http://localhost:8000/

The memory usage is analyzed with : pmap $PID | tail -1 >> mem.txt; SLEEP 0.001

  Req/s Mem  
Go 9428.90 ~18.5M  
NodeJS 5775.84 ~646M  
Java(Netty) 7086.34 ~2 434M (No flags)
" 6931.62 ~512M (-server -Xmx128m -XX:+UseConcMarkSweepGC)
" 9682.42 ~577M (after a couple of runs for JIT warmup )

Conclusion ?

The mem test takes into account all the libraries loaded by the process as this seemed to be the fairest comparison.

The Java version might be doing a bit more than the the others since it was an experience I had here... Without explicit JVM flags it gets very memory hungry and I bet that a -server flag without memory restrictions would be a huge mem hog. Fortunately the last tests show that with a memory usage similar to NodeJS's it does a whole lot better (I'm happy cause I can keep using Scala instead of Javascript :P)

NodeJS has a lot of hype around it and it definetly deserves it but without the web workers support it is not capable of taking full advantage of several cores and you'd have to go with a local pool and a load balancer in front of it. It's memory usage is not the best.

Finally Google's Go is surely the big winner here. With very little memory usage and good performance it is the undisputed champion. Also, I'm using the 6g compiler but have read somewhere that gccgo could bring a 50% improvement. The language's not great but it sure beats C, C++ and even Java in my book ;)