Cluster API integration to allow TCP ports to be shared

[rcrichton]

Closes #405

This PR allows the OpenHIM to use the cluster API. This will share all servers with the workers, both TCP and HTTP, so, this allows us to cluster even with a single config file.

There is a bug in node preventing me from clustering the UDP sockets (See nodejs/node-v0.x-archive#9261) so those don't get clustered yet. The first worker to get to the server up hosts it.

Use as follows: node --harmony lib/server/js --cluster=auto - or replace auto with a number of workers.

@devcritter could you please review this?

[hnnesv]

Not sure if it's related to the udp bug - but I get the following error occasionally when stopping the server:

Error: Not running
    at Socket._healthCheck (dgram.js:452:11)
    at Socket.close (dgram.js:353:8)
    at /openhim-core-js/src/server.coffee:486:22
    at stopTasksProcessor (/openhim-core-js/src/server.coffee:461:7)
    at process.exports.stop.stop (/openhim-core-js/src/server.coffee:463:35)
    at process.emit (events.js:107:17)
    at process.exit (node.js:600:17)
    at process.<anonymous> (cluster.js:532:17)
    at process.g (events.js:199:16)
    at process.emit (events.js:104:17)

Maybe you should catch that error? (I couldn't see a isClosed method in the api)

[hnnesv]

Do the mongo connections still need run if the node is master? I believe this will start up a connection pool, so it might save some resources by excluding this

[rcrichton]

Good point. I'l look at removing it.

[rcrichton]

Hey @devcritter, it only possible to remove these if we move all /model/* requires out of the master as these exported connection are used there. I'm not too keen to make that change as its rather large, I don't know if it will affect anything else and I don't think we'll save much by doing so. What do you think?

[hnnesv]

Aah okay, yeah if it's a mission than definitely skip it. I'm sure it's not too big a deal ito resource usage.

[hnnesv]

Found an issue when creating new tcp channels from the console - it only starts up the tcp server on one of the workers. When restarting core though, all the workers pick up the tcp channel correctly. Presumably this happens because it's starting a tcp socket on whichever worker gets the API call.

A quick solution might be for the API call to just trigger a general restart?

[hnnesv]

(the same issue with stopping a TCP channel, e.g. deleting it)

[hnnesv]

@rcrichton Just some comments above, but otherwise it works really well! Very cool :)

[rcrichton]

Hey, @devcritter I also get that dgram server error some times. I don't think is an issue due to clustering. However, it may be because it isn't possible to wait for the dgram server to close as the function doesn't have a callback (the node API seems a bit different to the norm of the tcp and http API here). I will try to catch this but I'm not quite sure where it's getting thrown, I'm guess at server.close() time.

Also, good call on the TCP channels. I'l have to do some more work there.

[rcrichton]

Hey @devcritter, I've addressed the comments. Could you have a look at this again?

[hnnesv]

👍 Looks good to me. Excited to see this merged in! :)

Just note that the failing tests before merging

[rcrichton]

Cool. The test are failing due to a bad mongoose version that I fixed in master this morning. I've merged in master here so they should pass now. I'l wait until they pass before merging this.