My last article was talking about the hexdump, I’ve continued to work on it and about some performances issues.

I was concerned by the time required to load blocks of bytes from r2 through synchronous calls. My first idea was to make them asynchronous since I had conceived the hexdump to keep enough lines visible inside the UI to be able to wait for new one to be added to the DOM. My first idea come to the concept of Promise. But, it comes that it wasn’t the proper tool for this goal. Promise doesn’t delegate job in the background and execute a callback then. If your task is synchronous, it will be kept synchronous. What I needed was a second thread of execution. Here come Web Workers.

Based on a message passing interface for communication with the main thread, web workers are also not connected to the DOM but they allow us to run tasks in the background. My task is not really long but it’s impact is noticiable because of the decreasing framerate.

The algorithm to fetch data isn’t complicated and the architecture behind this is very simple. It’s include a layer of cache to keep a small DOM. So the hexdump panel is drawn by the hexdump component which is plugged to the hexpair navigator. The navigator is here to coordinate the render with the UI and keep track of previous fetched data. The purpose here is to load the minimum amount of data from r2 and don’t re-ask for data we previously asked.

The usage is similar to this example:

// Instanciating an hexpair navigator
var nav = new HexPairNavigator(howManyLines, startOffset);

// Fetching a specific block (before, current and after)
nav.get(nav.Dir.BEFORE);
nav.get(nav.Dir.CURRENT);
nav.get(nav.Dir.AFTER);

// Moving
nav.go(nav.Dir.AFTER);

Before my modifications, the navigator was asked if the data wasn’t already here and then, we retrieved the data from r2 if required. Now, a web worker is spawn at the beginning and it waits for parameters to fetch chunks (bytes and associated flags). No specific work is done currently except small preparation inside a proper data structure but this could be improved later if performance in the browser is a concern. So, at the end of this process, the web worker send a message to the UI who process to save it and place it in the UI.

// We specify the file used to run the thread
var worker = new Worker('hexchunkProvider.js');

worker.onmessage = function(e) {
    // Receiving message from worker
}

// Sending a message to the worker
worker.postMessage(/* data */);

The code of the worker is pretty simple.

// Loading specific libs into our web worker
importScripts('r2.js');

self.onmessage = function(e) {
    // Processing in a completely independent thread

    // Then sending a response
    self.postMessage(/* data */);
}

With this implementation, we have broken the sequential order with HexPairNavigator::get then operate, so I’ve added the capacity to provide a callback to be able to interact with the incoming chunk:

nav.get(nav.Dir.CURRENT, function(chunk) {
    // Incorporate the chunk into the DOM
});

The algorithm using this is pretty simple :

  • UI ask for more lines
  • HexNavigator look for cached ones
  • R2 is asked to retrieve a block (through the web worker)
  • We insert it into the DOM with the UI component

The difficulty here is to synchronize different events like the scroll with the incoming data and modify the code to be adapted to the event-driven pattern required by this new situation. At the end, we can consider the retrieval of data from R2 command line asynchronous. But, more than this, it authorizes us to do more processing to limit the impact on the fluidity.