The Sheep-Pen of the Shaun

News

Shaun, the author of this blog is a semi-geek, clumsy developer, passionate speaker and incapable architect with about 10 years’ experience in .NET and JavaScript. He hopes to prove that software development is art rather than manufacturing. He's into cloud computing platform and technologies (Windows Azure, Amazon and Aliyun) and right now, Shaun is being attracted by JavaScript (Angular.js and Node.js) and he likes it.

Shaun is working at Worktile Inc. as the chief architect for overall design and develop worktile, a web-based collaboration and task management tool, and lesschat, a real-time communication aggregation tool.

.NET

Working behind a proxy server is a very common scenario in a cooperation environment. If we are using Windows and using some windows based application, such as Office, Visual Studio, etc. we almost no need to care about the proxy setting since they will use the Internet Connection setting to communicate with the services out of our company. But if we are using some cross platform or *-nix application, such as Node.js and some of tis 3rd party modules, they might not able to load the proxy setting automatically. In this case, we will encounter some Internet connection problem.

For example, let’s say I have a module named ‘curl.js’. Basically we can use this module to fire an HTTP request and get the response content shown on the console.

1:var http = require('http');

2: module.exports.google = function (callback) {

3:var req = http.request({

4: host: 'www.google.com',

5: port: 80,

6: path: '/',

7: method: 'GET'

8: }, function (res) {

9: console.log('STATUS: ' + res.statusCode);

10: console.log('HEADERS: ' + JSON.stringify(res.headers));

11: res.setEncoding('utf8');

12: res.on('data', function (chunk) {

13: console.log('BODY: ' + chunk);

14: });

15: res.on('end', function () {

16: callback(null);

17: });

18: });

19: req.on('error', function(e) {

20: console.log('problem with request: ');

21: console.log(JSON.stringify(e));

22: callback(e);

23: });

24: req.end();

25: };

So that in our code we can use this module like this.

1:var curl = require('./curl.js');

2: curl.google(function (error) {

3:if (error) {

4: console.log('error');

5: }

6:else {

7: console.log('ok');

8: }

9: });

This code works perfect if we have a direct Internet connection. But if we are working behind a proxy server we might get the error message as below.

This is because the ‘curl.js’ module doesn’t consider the proxy environment.

Although we can modify the ‘curl.js’ content in this example, what should we do if it’s a 3rd party module which we’d better not change its content. We have to figure out a solution by just changing our own code instead of change the module itself.

Different from NPM and Git, currently Node.js doesn’t have a global setting for proxy. So my solution is to override the HTTP request function being used in the module and let it support proxy. Based this thread in StackOverflow let’s create a new Node.js file named ‘http-proxy.js’ and paste the code below.

1:var http = require('http');

2:var __request = http.request;

3:

4:var proxy = {

5: host: '',

6: port: 0

7: };

8:

9:var _debug = false;

10:

11: http.request = function (options, callback) {

12:var __options = options;

13: __options.path = 'http://' + options.host + options.path;

14: __options.host = proxy.host;

15: __options.port = proxy.port;

16:if (_debug) {

17: console.log('=== http-proxy.js begin debug ===');

18: console.log(JSON.stringify(__options, null, 2));

19: console.log('=== http-proxy.js end debug ===');

20: }

21:var req = __request(__options, function (res) {

22: callback(res);

23: });

24:return req;

25: };

26:

27: module.exports = function (host, port, debug) {

28: proxy.host = host;

29: proxy.port = port;

30: _debug = debug || false;

31: };

In the code we firstly “copied” the original ‘http.request’ function for future usage. Then we rewrite this function, pass the proxy host and port into the request options, and then combine the target host with the path so that Node.js will firstly reach the proxy server then and tell it to send the request to the destination. Finally we exported the function so that user can specify the proxy information.

Now what we need to do is to add a line of code in our application to specify our proxy as below.

1: require('./http-proxy.js')('127.0.0.1', 3128);

2:

3:var curl = require('./curl.js');

4: curl.google(function (error) {

5:if (error) {

6: console.log('error');

7: }

8:else {

9: console.log('ok');

10: }

11: });

Then we will see it loaded the page content successfully through the proxy server.

Since we rewrite the http.request function to inject the proxy setting, we MUST put this line of code in every Node.js source file and it MUST be placed before all our codes including the ‘require’ statements.

Comments

#re: (Semi-)Global Proxy Setting for Node.jsPosted by Everton
on 5/13/2015 4:04 AM
how can i put a user and password?

#re: (Semi-)Global Proxy Setting for Node.jsPosted by Shaun
on 5/13/2015 8:41 AM
@Everton It doesn't support username and password. In my case, I'm using CNTLM to specify username and password, then use the proxy address of CNTLM in this module.

#re: (Semi-)Global Proxy Setting for Node.jsPosted by indolering
on 6/8/2015 6:26 AM
I get the following error:

```/usr/local/bin/node index.js/Users/indolering/easyapi-spec/proxy.js:40 callback(res); ^TypeError: undefined is not a function```

Which traces to:

var req = __request(__options, function(res){

callback(res);

});

#re: (Semi-)Global Proxy Setting for Node.jsPosted by Sam
on 6/29/2015 9:07 PM
Hey @Shaun, this is a great tutorial, but I am having issues connecting to sites that use 'https://' protocol. Can you help? Redirecting to a suitable forum would also be helpful.