NodeJS cURL Tutorial

NodeJS is a powerful evented Javascript platform running on top of Google’s V8 javascript engine. Over the last year the popularity of NodeJS has increased greatly, as shown by the number of discussions on programming forums such as Hacker News, Reddit and Stackoverflow. Despite this NodeJS is still young and it falls short to Python and Ruby when it comes to the number and range of libraries and modules available.

The Problem

I wanted to convert one of my web scraping applications I made in PHP to NodeJS. The application interacts with SOCKS4 and SOCKS5 proxies using PHP’s cURL bindings. Unfortunately there are no cURL bindings (or SOCKS protocol wrappers) written for NodeJS. NodeJS does come with an HTTP client module that can issue web requests, however it doesn’t support SOCKS proxies.

The Solution

Although there are no specific NodeJS bindings for cURL, we can still issue cURL requests via the command line interface. NodeJS comes with the child_process module which easily allows us to start processes and read their output. Doing so is fairly straight forward. We just need to import the exec method from the child_process module and call it. The first parameter is the command we want to execute and the second is a callback function that accepts error, stdout, stderr.

The Code

var util = require('util');
var exec = require('child_process').exec;

var command = 'curl -sL -w "%{http_code} %{time_total}\\n" "http://query7.com" -o /dev/null'

child = exec(command, function(error, stdout, stderr){

	console.log('stdout: ' + stdout);
	console.log('stderr: ' + stderr);

	if(error !== null)
	{
		console.log('exec error: ' + error);
	}

});

This returns the HTTP code and time it took for the request to be issued:

stdout: 200 0.710

Conclusion

Although NodeJS is still young and bindings haven’t been made for alot of C libraries, the code above shows how you can still interact with the functionality they provide. That being said, the number of NodeJS modules is growing weekly. Be sure to keep an eye on the module wiki and npm registry for a list of modules.