In my previous post, I wrote about the quest towards unifying web application architecture around a single shared codebase. Airbnb’s Rendr library provides one path forward on that quest. In this post, I’d like to provide a high-level explanation of my own solution for unifying application code.

There’s a few segments of an application’s code that will almost certainly be repeated on both the client and server: views, view state, and data models (excluding data access). To a certain extent, routing could be repeated as well; if you’re not using hashbangs, you’ll need to implement the same named routes on the client and server (although the actions you perform for those routes will be dramatically different).

The solution I’ve put together is still a work in progress, and is more a set of practices and conventions than a library. This flexibility means that while I’ve been implementing the approach using Handlebars, Backbone, and Express, you could theoretically use a completely different set of libraries. In this post, I’ll go over the basic ideas behind my approach; I’ll be putting together a sample implementation on Github that I’ll be able to write about in more detail in the future.

Unifying View Templates

Sharing template code is perhaps the easiest step towards achieving web application unity. With a full-JS stack, you’re compiling the same templates for rendering both server-side and client-side. So why not use the same templates for each?

In my experiments, I’ve been using Handlebars as my templating system. It’s based on the Mustache syntax, but has a number of powerful features, including partials and custom helpers. Handlebars is also the templating system behind Ember.js.

The key to sharing templates is the realization that what constitutes a “partial” can be relative to where the template is used — that is to say, the same template can be used as a full template on the client, and a partial template on the server. My templates are divided into three groups:

• client - client only templates
• shared - used on both the client and server
• server - server only templates

For the client, the templates in the client and shared folders are pre-compiled in a Grunt step, so for all intents and purposes there’s a single flat collection of templates. The server, however, references all of the templates in the shared folder as partials. What’s more, view helpers (in this case, special Handlebars tags) are stored in a shared/helpers directory that is referenced by both the client (in a Grunt step) and the server.

If you’re confused now, you won’t be when you see the actual implementation in my next post. What sounds complicated here is quite straightforward in practice.

Populating View Templates

If it’s easy to share templates on the client and server, getting the right data into those templates is a bit of a harder proposition. Handlebars renders data by running the template against a “context”, which is just a plain old hash. But how can we make sure that the contents of that hash are synchronized when rendering the same view on both the client and server?

The problem is exacerbated with how Backbone muddles view rendering logic. In Backbone, there’s not a fine line between the View logic (in Backbone.View) and Model logic (in Backbone.Model and Backbone.Collection). Backbone Views end up performing view state management by manipulating model instances, in order to avoid bloating Models with View specific code. But adding stateful logic into a Backbone View causes two problems:

1) it makes for larger, more complex views, as concerns are split between UX event management and controlling model state;

2) it makes it harder to share code with the server, as rendering state gets locked in a Backbone view.

We can get around these problems by reconsidering the role of a Backbone View — separating its concerns into that of Mediator and View Model. The Mediator role is one of event manager and template marshal; the view model is, most basically, a state controller.

By splitting the state controller into an object of its own concern, you can extract the logic into a module that can be shared by both the client (via Browserify) and the server (via require). The Backbone Mediator simply invokes the View Model, declaratively calling functions on it as users interact with the app — transitioning Backbone away from its traditional role of MVC framework into more of an MVVM implementation.

A View Model just returns a hash with the current state of data, which is supplied to the template. On the server, a View Model is instantiated with data accessed from the backend; the results of the View Model instance’s toJSON method are passed to the template for rendering. On the client, the same pattern is repeated, but the data is accessed via AJAX instead.

Sharing Models on Client and Server

Synchronizing model code on both the client and server is perhaps the hardest step towards achieving web application unity. The challenge primarily derives from a characteristic of Model code that we’ve inherited from the Rails world: specifically, a conflation of Model concerns between “data model object” and “data access object”. A data model object should identify a model object’s properties, its getters and setters, its relationships, and some convenient accessors; a data access object should deal with querying data sources for populating the data model objects.

Unfortunately, Backbone follows in the tradition of Rails’ ActiveRecord, flattening data accessing and data modeling into a single object. This flattening causes problems even beyond unifying a web application stack. How many Backbone applications have you seen that globally override the sync method, and use properties on model classes to determine how sync invocation works? (Not to pick on Backbone; it’s just the web application framework with which I’m most familiar.)

For now, I’ve abandoned trying to get Backbone Models and Collections to work on both the client and server. But there’s alternatives that can fill the gap nicely. The first of which I’m familiar is Dan Dean’s Tubbs library. Tubbs provides a DSL for defining data model objects, but also provides a pluggable dataStore. A single model module can be defined in a shared folder, and then required on both the client and server, with both supplying their own dataStore.

In order to make the separation between data modeling and data accessing even clearer, and the sharing of model code on the server and client easier, I created by own voltron-model library. While Voltron isn’t ready for primetime (yet), it should make the unified definition of data models significantly easier.

Whichever modeling library is chosen, the basic approach should be the same: data models are defined in a shared folder. These models are then imported by code on the server (or packaged with Browserify for the client). The server and client should decorate the models appropriately (either by adding to the model’s prototype or adding an accessor component). Thus, a model can be made to access Mongo or Postgres on the backend, or a REST service on the client.

The benefit of using Backbone on the client is that it doesn’t care if you only use limited parts of the library. Arrays of event-emitting models can provide much of the same benefit of Backbone Collections and Models, and Backbone Views and Routers see no difference.

Wrapping Up

All this discussion may be a bit confusing without a concrete implementation to reference; hopefully, I’ll be able to put together a sample application in the next few weeks. But the basic takeaway of the post is that unification of JS web applications can be made significantly easier with two factors:

• structuring your code so that it can be required by the server or packaged for the client in the same ways, whether by ingesting templates as partials or packaging code with Browserify;

• following the Node.js approach of writing small, single-purpose modules means that there’s less boilerplate to prevent you from using code in multiple environments

It’s been a long time since I’ve written anything for this site. Neither neglect nor procrastination prompted my absence, but rather, lots and lots of coding. All that code has left me with a surfeit of topics to cover here, so instead of talking about what I’ve been working on, I’m going to dive straight into the good stuff.

I want to write about the current “Holy Grail” of web application development: a unified application stack. Any dev who’s built a thick-client web app (regardless of library - Ember, Backbone, this problem is independent of tool choice) has dealt with the frustration of recreating logic in multiple parts of the application stack. It’s incredibly annoying to be writing the same code to handle model accessors, view state, and rendering logic in two different places; it’s even more irksome to be fixing bugs in one or both areas as your application matures. But let’s back up a bit. Why do we need to be writing the same logic in multiple places at all?

Generally speaking, modern web app development requires a choice from one of three architectural thrusts:

• thick server, thin client: view logic is written on the server, and rendering happens there as well. The client code deals with interactivity and AJAX requests, but is mostly just updating areas of the page with rendered view content returned by the server. This approach is most famously advocated for by 37 Signals. A thick server simplifies view code, but does so at the expense of larger AJAX payloads; why pull fully rendered code when you can just pull JSON data and render it with a client-side template?

• thick client, thin server: This architecture inverts the first approach; the server provides an API, which is basically just a wrapper around data access and caching. All of the app logic resides in the client. This approach can make for quick interactivity and rendering on the client, but presents a massive problem with “first load” delay, as the client needs to make two connections to the server: the first to load the client application code, and the second to pull the appropriate data to render. You may remember this problem from Old New Twitter.

• thick client, thick server: Unsurprisingly, this approach combines both of the previous options. Views are rendered on the server, which means the initial load is fast, but the client handles all the rendering from that point forward. The good news is that you have two fairly independent, (hopefully) highly performant application structures; the bad news is that you’re now repeating logic in view templates, view states, model helpers, even route handlers.

One exciting prospect of Node.js is that it makes Javascript an “isomorphic language”, which is to say that “any given line of code (with notable exceptions) can execute both on the client and the server” (h/t Nodejitsu). But that said, there haven’t been very compelling demonstrations that solve the code-duplication problem.

In the last few months, I’ve come up with a solution that allows for code reuse on client and server, leading to a unified codebase for handling view rendering, view states, and model interaction. It’s using components and libraries that you already know: Browserify, Backbone, Express, and Handlebars. But I’m going to hold off on describing that solution for now, and instead write about Rendr.

Rendr is the result of Airbnb’s first Node app, and it’s also the library that finally prompted me to close Vim and start writing here again. It’s the fulfillment of an idea that I first heard about from Tim Branyen: what if we used Express to render the same Backbone Views that we render on the client? That question is what prompted me on my own path towards application stack unity, so I’ve been very eager to learn more about Rendr.

Rendr took Tim Branyen’s initial idea and made it a reality. It’s important to note that Airbnb’s experiments here weren’t an excuse for architecture aeronautics; rather, they were expressly concerned with solving the “time to content” problem, which is to say, the rendering delay that Twitter famously faced. And by unifying client and server code, Rendr not only makes developers’ lives easier, it solves that time to content problem.

Rendr has a number of features and design approaches that I very much like:

• it’s described as “a library, not a framework”

• it doesn’t create a server side DOM

• it doesn’t try to collapse backend data access into the client tier, as some other frameworks have done

• it utilizes existing, familiar code (Express and Backbone)

• it keeps the rendered DOM on first load and attaches client objects to it, rather than instantiating client views and replacing the DOM with the results of those client views.

This last item is particularly important: rendering a page on the server, sending it to the client, and then recreating the DOM on the client can lead to odd rendering issues, interactivity problems, and problems with event listeners. But dynamically instantiating client objects and attaching them to an already-rendered DOM is not a trivial problem, and I commend Spike Brehm and the rest of the Airbnb team for taking the time to get this right.

You should definitely take the time to explore Rendr’s codebase. If it existed when I first set out to solve this same set of problems last year, then I probably would’ve abandoned my own quest and just used this library. But the key point of differentiation with the approach I took from Rendr’s design is modularity. Is it possible to achieve application stack unity, while using objects and code that can be implemented with any number of frameworks in any number of ways?

Stay tuned.

The entirety of the Javascript and Node.js universe is based around a continuation-passing coding style – that is to say, callbacks. This style of async coding greatly aggrevates Node critics, who constantly argue that continuation-passing invariably results in “callback hell” spaghetti code for any non-trivial problem. Node.js proponents respond that continuation-passing presents an easier mental model for async coding than alternative approaches, and that properly abstracted code avoids any gross “chains of doom.”

This post has nothing to do with that argument.

If you’re coding in Javascript, you’re going to be using callbacks, and if you’re building a library, you’re going to be supporting callbacks. It’s the reality of the ecosystem. But there are certain problems for which some syntactic sugar is helpful. Promises provide a wrapper for callbacks that allow for a chainable API around a sequence of continuations. They’re not a native part of the language, but there’s a strawman for the inclusion of Promises in a future version of JS. In the meantime, you can use the excellent Q Library from Kris Kowal. jQuery includes its own Promises implementation ($.ajax calls return Promises), but they’re slightly divergent from the Q spec. You can read this excellent guide explaining the differences between jQuery and Q promises, if you’re coming from one direction or the other.

So much for introductions. I’ve been using promises for awhile now in some of my Node code, and there’s several cases where their inclusion can clean up an API. (One area: Mongo integration, where any simple action can require several callbacks.) One benefit of Promises is that they can provide exception isolation, just like continuation-passing. The Q wiki provides the following example:

1
2
3
4
5
6
7

var info, processed;
try {
  info = JSON.parse(json);
  processed = process(info);
} catch (exception) {
  console.log("Information JSON malformed.");
}

In the above snippet, errors generated from the parser and the process function arne’t isolated. Promises, meanwhile, offer this exception isolation out of the box:

1
2
3
4
5
6
7
8

var processed = Q.call(function (json) {
  return JSON.parse(json);
})
.then(function (info) {
  return process(info);
}, function (exception) {
  console.log("Information JSON malformed.");
})

In the second example, the process function is only executed if JSON.parse doesn’t throw an error. Conversely, this same goal could be achieved with continuation-passing:

1
2
3
4
5
6
7
8
9

var processFn = function(cb) {
  asyncJSONParser(json, function(err, info) {
    if (err) {
      console.log("Information JSON malformed.");
      return cb(err);
    }
    cb(null, process(info));
  });
};

One caveat is that when you’re chaining your promises together, the behavior is slightly different than you might expect:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

var existingUser;
var promise = User.findOne({username: username})
  .then(function (user) {
    if (!user) {
      throw new Error("Username is incorrect");
    }
    existingUser = user;
    return User._hashPassword(password, user.get('salt'));
    }, function(error) {
      console.error('error in findOne');
      throw error;
  })
  .then(function (hashedPassword) {
    if (existingUser.get('hashedPassword') === hashedPassword) {
      return existingUser;
    } else {
      throw new Error("Password is incorrect");
    }
  }, function(error) {
    console.error('error in _hashPassword');
    throw error;
  });

In the above example, if there is an error in User.findOne, the error is propagated throughout the chain, which means that your console will have both error in findOne and error in _hashPassword printed. There’s no way of “breaking” from the promise chain early. But here’s the thing: you don’t have to. Unless you’re interested in transforming error objects, there’s not much point in catching promise errors in intermediary steps in the chain. So the above code would be abbreviated:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23

var promise = User.findOne({username: username})
  .then(function (user) {
    if (!user) {
      throw new Error("Username is incorrect");
    }
    existingUser = user;
    return User._hashPassword(password, user.get('salt'));
  })
  .then(function (hashedPassword) {
    if (existingUser.get('hashedPassword') === hashedPassword) {
      return existingUser;
    } else {
      throw new Error("Password is incorrect");
    }
  });

//later in chain or in a diffent function
promise.then(function (eventualResult) {
    //do something with result
  }, function (error) {
    //handle error
  });
}

There’s simply no point to having then failure handlers that just throw an error, since the library throws and propagates errors for you automatically. In promise chains, then, you have the benefit of isolating exceptions (only proceeding from step to step if everything is ok) without the extra semantics of if (err) return cb(err) from continuation passing.

Client-side unit testing is crazy.

Before you skip below to yell at me in the comments section, I don’t mean that the idea of client-side unit testing is crazy. On the contrary: we need client-side tests more than ever! With ever-increasing complexity in our frontend code, not to mention the reliance on frameworks, building a web application without client-side tests is completely inadvisible.

But the process. Ohhhh, the process of client-side testing is bonkers.

Let’s take the example of a modern Node.js web app setup. I’ve got the app’s configuration wired with grunt.js. (If you’re not using grunt, you should be!) During development, server files are watched for changes and automatically kickoff mocha tests, which display their results in the grunt console. Great!

Client-side changes kickoff their own unit tests, which of course are also using mocha for consitency in testing vocabulary. But because of the nature of client-side testing, we can’t just call require on the unit under test. Instead, we need something like the following setup:

• requirejs for managing dependencies

• PhantomJS for running a headless browser

• grunt-mocha, which manages the PhantomJS process and reports test results to the grunt console

• an HTML fixture for loading the requirejs config, the browser environment, and test files

This setup worked. It also felt downright medieval compared to the simple server-side mocha testing configuration.

I’m not the only person who’s frustrated with this process. Chris Dickinson at Urban Airship recently released Drive.js, a unit testing framework and test driver that eases some of the above configuration. But in any of the setups I’ve seen, your client-side “unit” tests feel much more like integration tests. If you need to load your entire client-side framework, load a headless browser, or even fire up an instance of your server to test a single function of your client-side code… why not just ignore client-side unit testing and skip straight to integration tests? The lack of true isolation of the system under test could lead to some frustrating false results anyway.

There needs to be a better way. I’m not looking for a lot in my testing setup:

• I want to run my client-side tests with the same mocha vocabulary and configuration as my server tests

• I want to isolate my tests to requiring a single file and its immediate dependencies, and testing it

That’s really it. And I think I’ve found a solution.

First, I’ve needed to refactor my client-side code to follow the CommonJS requires model. I’ve also switched to browserify from requirejs, although this switch is no longer a necessity — requirejs has introduced a new configuration option, cjsTranslate, which will wrap CommonJS defined modules. Using the CommonJS approach means that my mocha tests can require my client-side files. Great.

But the much larger problem with client-side testing is those pesky browser globals: the windowobject, the DOM itself, not to mention global code like jQuery, Backbone, etc. How can we test code that relies on the presence of those globals, without resorting to running PhantomJS?

The answer comes in two parts: jsdom, and the Node.js vm module.

jsdom provides a server-side implementation of the DOM, while the vm module provides the “global” object that we can use to provide a fake browser environment to our client-side files. While I’m still fleshing out the implementation, the results so far have drastically improved my ease with client-side testing.

Here’s how it works. In a mocha test file, instead of a strict require call, I’m using a helper (passing this, the mocha test context):

1

setupClientScript.call(this, "admin/postEdit", done);

setupClientScript loads a jsdom window, injects global javascripts (like jQuery), and wires the client-side file being tested. Here it looks in a slightly simplified form:

1
2
3
4
5
6
7
8
9
10
11
12
13

var setupClientScript = function(path, done) {
  var relPath = context.clientRoot + path;
  var self = this;
  jsdom.env({
    html: '<html><head></head><body></body></html>',
    src: context.globalScripts,
    done: function(errors, window) {
    self.clientRequires = requireClientScript(relPath, window, {});
    self.clientRequires[path] = self.clientRequires[relPath];
      done();
    }
  });
};

Remember, the client-side require can’t just inject the DOM into the file under test, it also needs to make sure the file’s dependencies have access to the DOM:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

var requireClientScript = function(path, window, r) {
  var script = fs.readFileSync(require.resolve(path)).toString();
  var module = { exports: {} };
  var context = vm.createContext(global);

  context.window = window;
  context.exports = module.exports;
  context.module = module;
  context.require = function(path) {
    r[path] = requireClientScript(path, window);
    return r[path];
  };
  vm.runInNewContext(script, context);

  if (r) {
    r[path] = module.exports;
    r.window = window;
    return r;
  } else {
    return module.exports;
  }
};

And that’s it. Now, any test function can call this.clientRequires to have access to the client-side file under test. No test server being spun up. No complex configuration or requirejs rewiring. No full-on headless browsers like PhantomJS. Just a single before setup call, and you can test with mocha as you normally would.

As I wrote above, I’m still fleshing out the implementation, working out the interface and fixing deficiencies. I’ll be publishing the code when things are more mature, but in the meantime I’d love to hear some feedback.

NodeConf 2012 was my first real “developer” conference. I had attended conferences in the past, but because of the heavy sponsorship of industry, most had felt like trade shows or job fairs. NodeConf was advertised as a conference for developers, by developers; Mikeal Rogers, the event’s main organizer, boasted that NodeConf 2012 would feature more core committers as presenters than any other Node.js conference.

As the event grew closer, my curiosity about what NodeConf would entail heightened. Mikeal didn’t divulge a clue as to the event’s schedule, and even perusing the twitter feeds of probable speakers and NodeConf’s site on lanyard didn’t reveal any helpful intelligence. At the opening party on Sunday, a general sense of cautious curiosity permeated the crowd. No one had ever attended a conference where the schedule would be revealed in realtime (appropriately for Node); my own conference inexperience left me more curious than most.

Mikeal had revealed two details: the major themes of each day of the conference. Day 1 was structured around the world of Node; Day 2 around the power of Node. NodeConf began with Node creator Ryan Dahl reconstructing the origins of the framework from livejournal entries and blurry photos of late night coding session in Germany. But as the day continued, I couldn’t see how the talks related to each other, or what I should be taking away from them. Each of the presentations, while excellent and often inspiring, didn’t leave me with a consistent narrative. I had expected “the world of Node” to consist mostly of talks like Isaac Schlueter’s “State of Node” presentation: perhaps a roundup of the big open source projects and frameworks that dominate the Node ecosystem, and a discussion of where they were headed. Instead, talks covered everything from realtime solutions with socket.io, to the creation of a whole new language. I had started the first day of NodeConf curious, but ended it confused.

Day Two changed all that.

As one talk on the “power of Node” led into the next, the topics and structure of the first day of NodeConf started to make sense to me. Each speaker demonstrated amazing new contributions to the Node ecosystem, from the Node-inspired Luvit framework built with Lua, to bleeding edge debugging tooling with dtrace, to ridiculous hardware demonstrations featuring human-detecting arduinos, three-dimensional saws, and radar laser robots controlled by Javascript. These talks didn’t just demonstrate powerful and frequently unexpected uses of Node: they helped me make sense of the previous day’s selection of talks.

Node is not just the Node framework: it’s a philosophy, an approach to solving problems. The first inkling of “Node as philosophy” came in substack’s post on Node’s aesthetic, discussing the “limited surface area” approach to Node’s modular problem-solving. But NodeConf demonstrated that the node philosophy goes much further than that:

1. async all of the things: Ryan’s initial work with libuv focused on the challenges of creating an async IO layer that worked across platforms, but that work has been extended with new languages like Fedor Indutny’s Candor and Tim Caswell’s Luvit framework. But it’s not just file IO - Max Ogden showed that the importance of a beautiful async API could be extended to browsers (utilizing ideas put forth by substack in day one), while Rick Waldron brilliantly demonstrated how JavaScript’s async nature could be used to solve problems which had always been approached synchronously. (Like robots. With lasers.)

2. embrace the hard problems: JavaScript may frequently be maligned for its odd “bad parts”, and JS developers have been looked down on in the past because of the history of DOM based spaghetti code, but NodeConf 2012 demonstrated a community that was unabashed in its tackling of serious computing issues. From Felix Geisendorfer’s’s writing of a binary MySQL parser that surpassed a c library in performance, to the community’s embracing of the concept of IO streams, to the use of dtrace in debugging: Node developers on the bleeding edge are not shying away from the toughest technical problems, but instead are taking them on at full speed and with high expectations.

3. a virtuous cycle of framework and community: Node’s “limited surface area” design of the core framework, along with the ease of use of NPM for publishing open source code and small useful modules, have led to an explosion of code. At NodeConf 2011, around 1800 packages had been published to NPM; now there are over 12000! Modules range from steaming cups of earl grey, to karaoke bar APIs, to websocket adapters and arduino libraries. The design of the framework drove its community adoption, but the community adoption has further developed the framework.

In the last talk of day two, Elijah Insua discussed the hardware hacking he’d done with Node. At one point, he mentioned that he came up with an idea that needed a midi driver, and he said something along the lines of: he never expected needing a midi driver, but as soon as he needed one, he came across one on NPM. The word that came to mind was serendipity: the contributions to Node, and the explorations of its philosophy, are feeding on themselves and driving completely unexpected innovation.

As I thought of the serendipitous nature of the Node ecosystem, Mikeal’s construction of NodeConf finally clicked for me. The two days of talks had been divided into groups of four presentations around a single theme; neither the themes nor the talks were made public, but each talk and theme bled into the next, with later speakers surprisingly stating how happy they were that a previous speaker had touched on a similar topic. The talks seemed to develop naturally, if unexpectedly; the overall direction was unclear to everyone except Mikeal, and gave the impression of the community simply exploring new ideas and pushing boundaries organically. The serendipity of Node’s development, then, found itself reflected in Mikeal’s construction of the conference. Does this sound ridiculously meta? Yes, but it completely worked.

As Node grows up and attracts adoption, it will be increasingly difficult to maintain the philosophically-driven, community-minded hacker ethos of the Node ecosystem, but I hope that Mikeal and the other stewards of the Node community can continue to foster talks and events that focus on the behind-the-scenes developers who are laying the building blocks for the future of this fascinating set of technologies.

When I originally started thinking about writing this post, I was going to phrase it in terms of spring cleaning. When I moved to Portland, I took the opportunity to reasses my development environment and to cleanup the libraries and source repositories I had scattered across my system. Since then, I’ve become the proud owner of a new retina MBP, so I’ve ended up building a new environment from scratch. Provisioning a new laptop is incredibly fast now; if you have your projects stored in version control, and you keep your dotfiles controlled as well, setting up a new laptop is as easy as:

• installing Homebrew
• installing iTerm2
• brew install git, macvim, tmux, tmuxinator, whatever langs and dbs you need
• pulling down your .dotfiles
• installing and syncing your Dropbox

I was developing on my new machine the day it arrived.

That experience of provisioning a machine so easily, and getting productive with it in no time at all, left me reconsidiering my original post on my toolset, which primarily boils down to zsh, tmux, and Vim. There’s plenty of material online to learn about these tools; a few of note are:

• The Text Triumvirate
• Unix as IDE
• Coming Home to Vim
• A Tmux Crash Course
• Dotfile Discovery
• Beautiful Tools

So why should you adopt tools with such a steep learning curve? Most discussions of IDEs and development environments quickly devolve into pointless arguments over the superiority of one tool over another, when in most cases the tool isn’t the most important factor; differences in productivity will more frequently boil down to experience and mastery of a toolset, rather than the toolset itself. I’ve seen masters of Vim, Eclipse, IntelliJ, and Textmate; in every case, the interface breaks down and gets out of the way of the user, who always “does” without any thought as to “how to do”.

So if mastery is more important than toolset, does your choice really matter?

Yes.

There’s two huge benefits of the Vim/tmux/shell toolset: ubiquity, and permanence.

Mastery of a tool takes many, many, many hours:

“Malcolm Gladwell’s observation that it takes 10,000 hours to become truly expert at something.” A little more than 3 years typically

— Jeff Atwood (@codinghorror) December 19, 2008

So whatever tool you choose, you want to be able to use it as much of the time as possible, in as many environments as possible.

A mastery of Eclipse, for example, doesn’t help you when you need to ssh into a unix environment (but vi/m would). Knowing Visual Studio inside and out can’t help you if you need to work on a Max or Linux box (but gvim and cygwin will still work on Windows). There’s not a lot that can compete with Vim in the ubiquity category; Emacs is the only other tool that comes to mind.

Permanence is a whole other matter. Technology changes constantly; some tools are improved, others are abandoned, still others fall out of use entirely. Are you a Flex developer who spent spent huge amounts of time mastering FlexBuilder and Eclipse? Out of luck now. Textmate? The tool that influenced countless developers to leave the Windows ecosystem eventually became pseudo-abandonware, leading to the current growth of tools liks Sublime Text 2.

Vim has been around since 1991. Its progenitor vi was written in 1976. (Again, Emacs is the only competitor here, having also been written in 1976.)

All of this is to say: don’t go and abandon your current toolset for mine! Just because what I use works for me, doesn’t mean it’s the best choice for you. If you’ve already mastered your toolset and are happy with it, stay there: you’d be wasting many productive hours on a new tool for the potential of hating it. And as wonderful as Vim/tmux/shell are for most, it’s not the right toolset for everyone or everything. If you’re doing mostly front-end web UX work, for example, Coda 2 would probably be perfect for your needs; iOS means you’re going to be stuck with Xcode, for better or worse.

But if you’re feeling constrained by your current toolset, or are making the jump from one technology to another, and haven’t given “the text triumvirate” a try, I’d very strongly recommend doing so. It’s worth the effort.

If you only know me through this blog, and have spent any amount of time playing Oregon Trail in your youth, you may have grown a bit concerned. It’s been four months since my last post, in which I wrote about my upcoming move to Portland, OR; since then, there’s been total silence here. I’m happy to report that, over the course of my cross-country move, I didn’t contract dysentery, drown in a reckless attempt at fording a river, or suffer a terrible firearms accident while hunting squirrels. In fact, I had better luck with the move than I possibly could have imagined. Our drive to Minneapolis avoided any nasty weather; our flight from Minneapolis to Portland proceeded without incident; our new house was solidly intact and without any animal squatters; and our movers arrived with all of our belongings at the exactly scheduled time (to the minute!) with nothing broken or lost.

Portland has lived up to, and perhaps exceeded, our expectations. Being able to bike around a city without a constant fear of death and dismemberment has been quite a refreshing change. Living in a house has allowed us to actually grow a legitimate garden instead of stuffing a few pots in a fire escape. We couldn’t be happier with the food scene here; the amount of craft beer in Portland is downright intimidating. And it’s been extraordinarily exciting to meet so many other new Portlanders who’ve moved here for similar reasons. Needless to say, if you’re a developer who’s thinking about Portland as a place to call home, feel free to drop me a line, or introduce yourself if you’ll be coming to node conf in July.

I’ve finally had to declare blog bankruptcy here, in order to just wipe the list of well-intentioned-blog-topics clear and get started writing again. Two topics will survive the culling. First, the move to Portland presented an opportunity for a spring cleaning of my development environment. I’ll be writing about how I organized my dotfiles, and explaining why tmux is not just a cargo culting fad, but a ridiculously awesome tool for maximizing your terminal productivity.

The other surviving topic surrounds Backbone.js. Since moving to Portland, I’ve continued working with Bloomberg Sports in New York on their application for MLB team analytics. (Think a modern web app version of Moneyball spreadsheets.) Working on such a large app has presented me with some difficult lessons on how to best use Backbone, especially for interacting with a polyglot backend, which I think would prove useful to any other web devs working on larger Backbone applications.

So now, back to work so I can actually get these posts written in the next week!

Two and a half years ago, Meaghin and I moved from Washington, DC to Brooklyn. Our move came as a bit of a shock to our family and friends - partly because of its suddenness, and also because of its proximity to our marriage. The move to Brooklyn was not motivated by a job opportunity or anything similar; instead, we had started feeling constrained by DC after five years of living there, and couldn’t imagine a more challenging and exciting place to live than New York (and Brooklyn in particular). A week after getting married, we made the decision to move, and three weeks after that, we found ourselves surrounded by boxes in an apartment overlooking the East River and Manhattan.

The first few months of living in New York definitely delivered on the “challenging” part of our expectations; not a day went by that didn’t evoke memories of Seinfeld plot-lines. Crazy co-op boards? Difficult supers? Ridiculous encounters on the subway? More than one New York stereotype proved its roots in authenticity. But as we grew used to the city and its neighborhoods, learned how to quickly cut from one area to another, discovered which subway lines consistently failed, and realized that one should avoid 5th Ave and Times Square at all costs, the challenges started to dissipate and left only excitement. New York, in its intensity and genuine melting pot of cultures and backgrounds, presents an amazing opportunity for people to define themselves. What matters most to you, and why does it matter? In a city where practically any hobby or interest can be pursued, you’ve got to seriously prioritize your time, and that prioritization goes a long way towards identifying exactly who you are.

But for all the excitement and growth, I started to notice another development in myself: pernicious cynicism. While I had once dreamed of tackling big problems and making the world a better place, years of living in DC and New York made me view such ideas as hopelessly naive. The seeming intractability of so many daily problems, the realization that apparently important movements were actually driven by petty motivations; older, wiser me didn’t look at idealists with disdain, but rather with a wistful admiration of their blissful ignorance.

Then I spent time in Portland, OR.

It’s hard to describe the feeling of Portland. And I don’t mean the city portrayed on Portlandia - although that show is certainly accurate in its skewering. I have never been to a place where so many people devote themselves to mastering a craft, not just because of the rewards of the mastery, but because not mastering that craft is simply not an option. There’s an entrepreneurial spirit that pervades Portland, but it’s one that’s devoted to developing new ideas and contributing back to the community. Even this “return to community” is not the traditional “I’ve done well so I’m giving back” idea, but rather a networked entrepreneurship, a continuous exploration of “How can my idea involve the unique work of that awesome ?”

Mikeal Rogers, organizer of NodeConf, described Portland with admiration:

There are times and places, great cities, that are remembered much more
fondly than they were thought of at the time. Paris in the twenties when it
was home to Hemingway, Picaso and Fitzgerald. Or San Francisco in the 50s
when the beat generation migrated from New York to call it home. So
impressionable were those years that the echo fuels future, lesser,
generations for decades.

When you're in Portland, Oregon you can't help but think it might be such
a place and the time. For what it will be remembered I have no idea, but
everyone seems to be bursting with creativity and no one is content until
they have done more.

Spending time in Portland pierced the shell of cynicism that I’d developed, and rekindled my motivations to build something real and lasting, to improve my community. Portland provides the inspiration; New York has provided the confidence and skillset to do something about it.

So once again Meaghin and I find ourselves surrounded by boxes and preparing for a big move. In a month I’ll be a resident of Portland. I can’t wait to find out what the future will bring.

Preaching the gospel of Node can be a tough job: how do you express your excitement for the technology without coming across as a fanboy? And how do you deal with people that view it as a cancer? I had to answer these questions when crafting a presentation last week introducting Node.js as part of a developer tech talk. Given the radically divergent opinions that Node has elicited, I wanted to address the FUD head on. I tried to focus on the gut reaction most people tend to have towards Node (“WHO would want to code their server in JS?!”), and I think the talk went fairly well. I’m embedding the slides here if anyone would like to take a look. Speakerdeck is a fantastic resource and I was happy to finally put it to use as a producer, instead of just a reader.

For even more insight into the developing world of “JavaScript everywhere”, check out this video from Joe McCann, principal architect of subPrint Interactive, on “End to End JavaScript”.

January’s been a long month; not in a bad sense, just sort of interminable. Looking through my links collection in notational velocity today, I found a good twenty plus from the last six weeks. I’ve culled them down a bit to the true standouts:

• An Object is not (Always) a Hash — Great post from Guillermo Rauch about the perils of using Javascript Objects as Hashes. Definitely some tricky gotchas here that you may not have thought about.

• Hot Node Reloading — Also from Guillermo, a solution to the annoying problem of continuously restarting your node app during development: Staying Up with Node.JS

• Shedding Light on SPDY — If you haven’t been hearing a great deal about SPDY lately, you will be soon. SPDY, if you couldn’t tell from the name, serves as a next-generation protocol to speed up web communication. Nodejitsu posts a helpful introduction to SPDY.

• Improving your Git Logs — Git log, out of the box, is so wordy that it proves fairly worthless. Check out Oli Studholme’s aliases for taming the git log beast.

• Hack Learning — One of the key lessons I’ve learned as a developer is that getting good at something requires hours, and hours, and hours, and hours of work, and many of these hours will be exhausting displays of ignorance which you’ll want to quickly forget. The good news is that you can pick up nearly any skill with enough effort and time. Jack Kinsella provides a hack for cutting back on how many hours separate “terrible newb” from “fairly competent practitioner”. His Janki Method may sound like a get rich quick scheme, but it’s really just a simple approach to improving your effiency at learning. Maybe you can cut those 10,000 hours of practice to a more managable… 6,000.