Google Chrome

Google Chrome extension 'reload' bug

I have been experimenting with Google Chrome extension development and stumbled over a rather annoying issue:

In some cases an unpacked extension will not reload when you click on the 'Reload' link on the extension page.
The only way to get the extension to reload properly is to close & restart the browser itself.

This is extremely confusing and caused me quite a bit of headache while debugging my extension project. I was debugging the message passing between a background and content script when I noticed that a console.log() entry was still triggered every time I reloaded the extension, even though it was already commented out in the source code.

Luckily this is a known issue and you can 'star' it here: https://code.google.com/p/chromium/issues/detail?id=32699#c7

One commenter suggests adding a

console.log("edit 1");

line to the extension and then increment it every time you edit the file, which seems like a good solution until this issue has been addressed.

Nevertheless, google chrome extension development is a lot of fun and you can expect some more posts about it in the near future to showcase my extension.

Node by example

Node by example: 13. Links & Where to go from here

13. Links & Where to go from here

The complete source code can be downloaded here: http://github.com/Hendrik/node-by-example

Please note: Node.JS is still in active development, so that some articles may contain code that is no longer working with newer versions of Node.JS
During the time I wrote these articles node jumped from 0.1.3x to 0.1.9x resulting in several chapters requiring code changes to reflect the new or removed features.
In most cases you should be able to update the code by comparing the problematic parts with the Node.JS documentation and its unit tests.
If all fails try searching for the error in the official google groups.

Official Node.JS links:
Node.JS project homepage: http://nodejs.org/
Node.JS documentation: http://nodejs.org/api.html
Node.JS official google group: http://groups.google.com/group/nodejs
Node.JS official git repo: http://github.com/ry/node
Node.JS tests: http://github.com/ry/node/tree/master/test/
Node.JS JSConf 2009 presentation by Ryan Dahl: http://blip.tv/file/2899135
Node.JS Modules: http://wiki.github.com/ry/node/modules

Node.JS blogs & aggregators:
How To Node: http://howtonode.org/
NodeBlogs: http://nodeblogs.com/

Interesting projects using Node.JS:
Twitter Node.js WebSocket Example: http://github.com/andregoncalves/twitter-nodejs-websocket
Real time online activity monitor example with node.js and WebSocket: http://blog.new-bamboo.co.uk/2009/12/7/real-time-online-activity-monitor-example-with-node-js-and-websocket

Node by example

Node by example: 12. Sample Project

12. Sample Project

The complete source code can be downloaded here: http://github.com/Hendrik/node-by-example

The following is a sample project which displays your website visitors in real time on google maps using node with the geoip and websocket server modules.

Basically what the script does is to watch the domains access log file for any changes, parse the changed line(s) to retrieve the accessing IP and then send it to the client via websocket.

In order for this project to work you need the following:
- google maps API key, available here: http://code.google.com/apis/maps/signup.html
- browser with websocket support, e.g.: google chrome: http://www.google.com/chrome

You can download the full source here: http://github.com/Hendrik/node-by-example

Let's start with the server code: See 12_example_project/server.js

First we need to include the websocket server & geoip modules, as well as setting up variables for the geoip database location (make sure to change it to have it match your path) and the ip_cache. Only 1 marker should appear on the map per IP, so we need to make sure to cache the already parsed IPs.

var sys = require('sys'),
    ws = require('./lib/ws');
    geoip = require('./lib/geoip'),
    dbpath = '/usr/local/share/GeoIP/GeoLiteCity.dat',
    ip_cache = [],
    filename = process.ARGV[2];

 

The "filename" is the full location of your selected website log file, e.g.: /usr/local/apache/domlogs/my-site.com

if (!filename)
    return sys.puts('Usage: node " + __filename + " filename');

 

Next we create a function that will parse the geoip data into json format and then send it to the client via the websocket:

function result_output(result, websocket) {
  var json_result = {};
  for (var key in result) json_result[key] = result[key];
  websocket.write(JSON.stringify(json_result));
}

 

We need another function to query the geoip database, which will provide the geoip data based on the provided IP:

function db_connect(ip, websocket) {
  var con = new geoip.Connection(dbpath, function(con) {
    con.query(ip, function(result) {
      result_output(result, websocket);
      con.close();
    });
  });
}

 

Next start the child process to "tail -f" the provided log file. The "-f" flag makes tail monitor the end of the file continously until the process stops:

var tail = require('child_process').spawn('tail', ['-f', filename]);
sys.debug('start tailing logfile');

 

Then create the websocket server and make it listen on port 8000:

ws.createServer(function(websocket) {
  ...
}).listen(8000);

 

Now the tricky part:
once a websocket client connects we want to start processing the "tail" child process output.

websocket.addListener('connect', function(response) {
  sys.debug('connect: ' + response);

  tail.stdout.addListener('data', function(data) {
    ip = data.toString().split(" ")[0];
    if (!ip_cache[ip]) {
      sys.debug(ip);
      ip_cache[ip] = true;
      db_connect(ip, websocket);
    }
  });

}).addListener('data', function(data) {
  sys.debug(data);
}).addListener('close', function() {
  sys.debug('close');
});

 

So, every time the 'data' event emits for the "tail" child process, it will parse the provided string and extract the IP.
If the IP doesn't exist yet in the ip_cache, then the IP will be sent to the db_connect() function to get its geoip data, which in turn sends it to the result_output() function to send the geoip data in json format to the websocket client.

The full source code for the client side file is located in the provided index.html file. In order for it to work you need to replace the following:
GOOGLE_MAPS_API_KEY needs to be replaced with your google maps API key
YOUR_SERVER_IP_HERE needs to be replaced with your server IP

The juicy bit is the following, which creates the websocket in the browser and every time it receives data it will extract the latitude and longitude data to place a new marker on the google map at the location of the visitor, based on his/her IP:

ws = new WebSocket("ws://YOUR_SERVER_IP_HERE:8000/");

// when data is received from the websocket server, extract lat & long to place a new marker
// at the correct location

ws.onmessage = function(evt) {
  var data = eval("(" + evt.data + ")");

  map.addOverlay(new GMarker(new GLatLng(data.latitude, data.longitude)));

};

 

To test this example project start the node websocket server, e.g.:

node server.js /usr/local/apache/domlogs/my-website.com

where "/usr/local/apache/domlogs/my-website.com" needs to be replaced with your actual website log file.

 

Open the index.html file in a websocket compatible browser, e.g.: google chrome and then visit your site to trigger another entry in your log file.
This in turn should result in your IP being displayed in the console and the browser window will show a new location marker on the map based on your geoip location.

Node by example

Node by example: 11. REPL

11. REPL

The complete source code can be downloaded here: http://github.com/Hendrik/node-by-example

A Read-Eval-Print-Loop is available both as a standalone program and easily includable in other programs. REPL provides a way to interactively run JavaScript and see the results. It can be used for debugging, testing, or just trying things out.

Start a REPL via:

repl.start(prompt, stream)

where "prompt" is optional and defaults to "node> "
"stream" is options as well and defaults to process.openStdin().
Inside the REPL, Control+D will exit.

 

The following example will start a tcp server listening to port 8000 and we will use the REPL to check & modify the available variables:
See 12_repl/repl.js:

var sys = require("sys"),
    net = require("net"),
   repl = require("repl");

no_connections = 0;
some_var = "Hello";
net.createServer(function(socket) {
  sys.puts("Connection!");
  no_connections += 1;
  socket.end();
}).listen(8000);

repl.start("repl.js prompt> ");

 

First make sure to include the repl module:

var repl = require("repl");

 

Next we set 2 variables, which we will manipulate through the REPL:

no_connections = 0;
some_var = "Hello";

 

Any client connecting to the tcp server will increase the "no_connections" variable:

net.createServer(function(socket) {
  sys.puts("Connection!");
  no_connections += 1;
  socket.end();
}).listen(8000);

 

Start the example via:

node repl.js

 

You will notice that instead of a running process you are presented with a prompt:

# node repl.js
repl.js prompt>

 

Enter the following to get the value of the no_connections variable:

repl.js prompt> no_connections

 

Increase it one or more times and check again:

repl.js prompt> ++no_connections

 

As you can see the REPL is very useful for debugging and testing your node scripts.

Node by example

Node by example: 10. addon modules

10. addon modules

The complete source code can be downloaded here: http://github.com/Hendrik/node-by-example

The examples in this chapter are meant to showcase extending node with some of the available third party modules.
Websockets (server & client), MySQL access and GeoIP are covered in this chapter.

i) GeoIP:
The following example is based on the GeoIP module available at: http://github.com/strange/node-geoip

Please follow the installation instructions provided on above mentioned page.
Short version: you need to install & setup libGeoIP and build the module for the example to work.
See 10_addons/geoip.js:

/*  This example requires the GeoIP addon available at
 *  http://github.com/strange/node-geoip
 */

var sys = require("sys"),
  geoip = require("./lib/geoip");

var dbpath = "/usr/local/share/GeoIP/GeoLiteCity.dat";
var ip = "216.236.135.152";

sys.puts("Looking up IP: " + ip);
var con = new geoip.Connection(dbpath, function(con) {
  con.query(ip, function(result) {
    for (var attr in result) {
      sys.puts(attr + " : " + result[attr]);
    }
  });
});

 

First we need to include the geoip module:

var geoip = require("./lib/geoip");

 

Next is the location of the GeoLiteCity.dat file on your system:

var dbpath = "/usr/local/share/GeoIP/GeoLiteCity.dat";

 

In this example the GeoIP lookup will be made against the following IP:

var ip = "216.236.135.152";

 

Use geoip.Connection() to create a new GeoIP connection object:

var con = new geoip.Connection(dbpath, function(con) {
  ...
});

 

Then use con.query(ip, callback) to query the provided IP against the GeoIP database:

con.query(ip, function(result) {
  for (var attr in result) {
    sys.puts(attr + " : " + result[attr]);
  }
});

 

The output should be:

Looking up IP: 216.236.135.152
longitude : -73.97250366210938
latitude : 40.84830093383789
country_code : US
continent_code : NA
metro_code : 501
country : United States
city : Fort Lee
area_code : 201

 

One interesting use for this module is to parse your websites access log file in real time and have a google map updated in your browser with the location of your visitors using a websocket.

 

ii) MySQL connection:
In order to connect to MySQL we will use DBSlayer, a lightweight database abstraction layer.
You will need to install & configure DBSlayer and download the node.dbslayer.js module available at: http://github.com/shoeman22/node.dbslayer.js
Please follow the installation instructions at: http://github.com/shoeman22/node.dbslayer.js

Once configured you can run dbslayer via:
(assuming you setup the configuration at /etc/dbslayer.conf)

dbslayer -c /etc/dbslayer.conf -s lokidan -h 127.0.0.1

 

Please note: DBSlayer comes with limited security, so please make sure to add the "-h 127.0.0.1" option to allow requests from localhost only.

dbslayer.js itself can be executed via:

node dbslayer.js "SQL QUERY"

where "SQL QUERY" needs to be replaced with a SQL query based on your DBSlayer configuration.

 

For example:

# node dbslayer.js "SELECT id FROM notifications"
-------------------------
Row 0: 4
Row 1: 5
-------------------------
STAT Uptime: 3963181  Threads: 1  Questions: 319223  Slow queries: 0  Opens: 91107  Flush tables: 1  Open tables: 64  Queries per second avg: 0.081
-------------------------
CLIENT_INFO 5.0.90
-------------------------
HOST_INFO Localhost via UNIX socket
-------------------------
SERVER_VERSION 50090
-------------------------
CLIENT_VERSION 50090

 

 

iii) Websockets (Server & Client)
The Websocket server & client examples are each based on their own third party module.
For the Websocket server we use http://github.com/ncr/node.ws.js and for the Websocket client http://code.google.com/p/revhttp/source/browse/trunk/nodejs/node-ws-client.js
Both are already provided in the full source code and don't require any extra installation.

Let's start with the Websocket server. You will see that it is very similar to the http server setup:
See 10_addons/websocket_server.js:

var sys = require("sys"),
     ws = require("./lib/ws");

ws.createServer(function (websocket) {
  websocket.addListener("connect", function (resource) {
    sys.debug("connect: " + resource);
    websocket.write("test");
    setTimeout(websocket.end, 10 * 1000);
  });
  websocket.addListener("data", function (data) {
    sys.debug('DATA: ' + data);
    websocket.write("Thanks!");
  });
  websocket.addListener("close", function () {
    sys.debug("close");
  });
}).listen(8000);

 

First make sure to include the ws module:

var ws = require("./lib/ws");

 

Create the websocket server object, which provides the following event emitters:
"connect": when a connection is established to the websocket server
"data": when data is received by the websocket server
"close": when the connection gets closed

ws.createServer(function (websocket) {
...
}).listen(8000);

 

First the "connect" event, where we send out a "test" message to the connected client and start a timer to close the websocket after 10 seconds:

websocket.addListener("connect", function (resource) {
  sys.debug("connect: " + resource);
  websocket.write("test");
  setTimeout(websocket.end, 10 * 1000);
});

 

Output any received data to the console:

websocket.addListener("data", function (data) {
  sys.debug('DATA: ' + data);
  websocket.write("Thanks!");
});

 

Output a simple close message when the connection gets closed:

websocket.addListener("close", function () {
  sys.debug("close");
});

 

This is already it. You can connect to this websocket server using any browser that supports websockets or use the provided websocket client script to connect from node:

This websocket client example is based on the module available at: http://code.google.com/p/revhttp/source/browse/trunk/nodejs/node-ws-client.js

See 10_addons/websocket_client.js:

var sys = require("sys"),
     ws = require("./lib/node-ws-client");

var wsClient = ws.createClient('ws://127.0.0.1:8000/');

wsClient.addListener('connect', function() {
  sys.puts("Connected!");
  wsClient.write("Testing");
});

wsClient.addListener('data', function(chunk) {
  sys.puts("Received data: " + chunk);
});

wsClient.addListener('close', function() {
  sys.puts("Disconnected!");
});

setTimeout(function() {
  for(var i = 0; i<10; i++) {
    setTimeout(function() {
      wsClient.write("Hello WS world! " + Math.random().toString());
      sys.debug('data sent');
    }, i*200);
  }
}, 2000);

setTimeout(function() {
  wsClient.close();
}, 10000);

 

Same old, same old: first include the third party module:

var ws = require("./lib/node-ws-client");

 

Then create the Websocket client:

var wsClient = ws.createClient('ws://127.0.0.1:8000/');

 

The Websocket client provides the same emitters as the server object for "connect", "data" and "close" events.
In this example we send out a "Testing" message upon successful connection to the websocket server:

wsClient.addListener('connect', function() {
  sys.puts("Connected!");
  wsClient.write("Testing");
});

 

Any received data will be sent to the console:

wsClient.addListener('data', function(chunk) {
  sys.puts("Received data: " + chunk);
});

 

Add a simple "Disconnected!" output when the connection has been closed:

wsClient.addListener('close', function() {
  sys.puts("Disconnected!");
});

 

Next add a timer to send 10 random messages within a short interval to the Websocket server:

setTimeout(function() {
  for(var i = 0; i<10; i++) {
    setTimeout(function() {
      wsClient.write("Hello WS world! " + Math.random().toString());
      sys.debug('data sent');
    }, i*200);
  }
}, 2000);

 

And at last add another timer to automatically close the connection after 10 seconds:

setTimeout(function() {
  wsClient.close();
}, 10000);

 

If you have a browser that supports websockets, then you can use the websocket server & client modules for some nice streaming applications.
As mentioned above, one example would be to stream the locations of your website visitors to a google map in real time using a Websocket.

Next Posts