ImportJS 3.0 Overview

Since I’ve updated ImportJS to version 3.0 recently (and even released a library based on it) I wanted to write a little bit about what ImportJS is and why I love using it.

Let me begin by mentioning that ImportJS is something I created a few years ago because I wanted a way to write modular code that felt familiar to me, and I wasn’t impressed by the alternatives at the time. I’m not going to spend much of this article bashing popular modular loaders, but I will at least say that many of them just seemed to over-complicate what I thought a module system needs to do. I’m sure there are many reasons why certain decisions were made about these systems, but I’m going to present my personal take on writing modules for the web.

ImportJS Is About Modules

In case you are unfamiliar with what a module is, think of it as a standalone section of code (usually one module per file) that contains a small part of your application. They may also be called “packages” in other languages, but for simplicity’s sake let’s just stick with JavaScript terminology. Modules can reference other modules by importing them, as well as expose their own properties and functions so that other modules can use them. Well-designed modules are designed to have very few dependencies, in that they don’t rely on too many other modules to work properly. Below is an example of a very simple module pattern in Node.js/CommonJS syntax:

/* my-module.js */

//Import other modules as you please
var SomeOtherModule1 = require('./some-other-module1');
var SomeOtherModule2 = require('./some-other-module2');

//Set up what you want to expose
var MyModule = function () {};

MyModule.prototype.foo = 0;
MyModule.prototype.increment = function () { this.foo++; };

//Export what you want to expose to other modules
module.exports = MyModule;

/* my-app.js */

//Use a module like so
var MyModule = require('./my-module');

var instance = new MyModule();
console.log("foo is: " + instance.foo); //Prints "foo is: 0"
instance.increment();
console.log("foo is: " + instance.foo); //Prints "foo is: 1"

This is a very commonly used coding pattern among server-side and desktop JavaScript applications, but it doesn’t work all that well in the web browser. On a command-line Node.js application, scripts don’t have to be downloaded so require() can run synchronously. On the web, you would have to bundle your code in advance in order to simulate this feature using a build tool like webpack or Browserify (or perhaps use an asynchronous loader like RequireJS). The upcoming version of JavaScript known as ES6 has a decent solution, but until the specification is implemented in all browsers we’ll be hopping around different build tools for quite some time.

ImportJS Is Like CommonJS

ImportJS resembles CommonJS syntax, but for the browser:

/* my-module.js */
ImportJS('MyModule', function (module, exports) {
  // this.import() for first-party dependencies
  var SomeOtherModule1 = this.import('SomeOtherModule1');
  var SomeOtherModule2 = this.import('SomeOtherModule2');
  // this.plugin() for third-party dependencies
  var $ = this.plugin('jquery');

  var MyModule = function () {};

  MyModule.prototype.foo = 0;
  MyModule.prototype.increment = function () { this.foo++; };

  module.exports = MyModule;
});

/* my-app.js */

//Load files before starting app
ImportJS.preload({
  baseUrl: 'js/',
  packages: ['my-module.js'],
  ready: function () {
    var MyModule = ImportJS.unpack('MyModule');

    var instance = new MyModule();

    console.log("foo is: " + instance.foo); //Prints "foo is: 0"
    instance.increment();
    console.log("foo is: " + instance.foo); //Prints "foo is: 1" });
  }
});

As you can see above it’s not truly the same as CommonJS because its versions of require(), the import() and plugin() functions, have a different usage.

In ImportJS, modules are stored via ImportJS.pack(). They can be retrieved later with ImportJS.unpack() while outside of a module, or this.import() while inside one. You can use any string you’d like for a module name, however I highly encourage you use Reverse Domain Name Notation if you plan to use the asynchronous loading capabilities of the library. ImportJS has a static preload() function that can load dependencies asynchronously, and it will parse the function body of each dependency dynamically as they load to determine what other files to load. The file names map from Reverse Domain Name Notation to the actual name of the file (so “Module” maps to “./Module.js”, and “com.AnotherModule” would map to “./com/AnotherModule.js”, etc). All of this is always relative to the root path you set across all files regardless of depth, which means you no don’t have to type out relative dependency paths in a project 😉

Code imported via this.plugin() actually does the same thing as import(), however it is reserved for asynchronous dependency loading. While fetching files dynamically, if ImportJS encountered a this.plugin(“jquery”) call it would search in a “plugins” folder at the root of your project for “plugins/jquery/jquery.js”. This creates a separate load context for ImportJS, so that the plugin resolves its own dependencies before allowing any load completion callbacks to trigger. The plugin’s dependencies have an entirely separate namespace, so duplicate module names will not conflict with your main project files. The purpose of this is to separate the concerns for your third-party dependencies from your own, and I have found this pattern to be useful across many projects I’ve worked on.

Finally, we have the ImportJS.compile() function which is a fancy way of “unpacking” all of your modules at once. ImportJS doesn’t actually run the code in your modules until you extract it with unpack(). In the above example you could technically remove the compile() function and ImportJS would still automatically execute your module before unpack() starts. This auto-unpacking includes any modules imported via import(), though it is recommended to use the compile() function to add developer intent that the application is ready to be unpacked and executed.

ImportJS Is Not Barely A Module Loader

Say what??? After all that I had just said this might sound a bit odd considering what most module loaders are made for, but ImportJS at its core really isn’t a system for loading modules. It would be more accurate to describe it as a system for storing modules. While it does have features to fetch dependencies asynchronously, it can work just fine when used purely as a tool for namespacing your code. You can comfortably write your source code in separate files and bundle them together for deployment without worrying about a dependency graph, since you control when to actually start unpacking all of your modules.

At this point you might be wondering “isn’t this just a glorified global object used for namespacing?”. The short answer to that question is yes, it’s a global object you can use to store and retrieve other objects. However, the key feature I’ve yet to cover is injecting logic between the post-execution and  pre-initialization phase of an application. I’ll explain what I mean by this shortly, but first let me give some background into what led up to this.

Concern#1 – Asynchronous Loading

So the first obstacle I ran into when starting to write JS full-time was managing dependencies that may not all be loaded simultaneously. This issue doesn’t exist in most non-web platforms since all of the code comes bundled in one package. For example, I come from an Flash/ActionScript background which has a packaging system that’s very similar to Java. As such, dependencies are all taken care of at compile time and you don’t have to worry about if everything has been loaded.

When you bundle all of your code together for a JavaScript application into one file these problem tend to go away, although the solution almost always involves using a module loader that is able to parse a dependency graph from your code before it can spit out a bundled file. Depending on the bundler, you can sometimes experience slow compilation times because of this.

On a side note, one thing I find odd about the idea of asynchronous dependencies in JavaScript is that there are module loaders out there that are designed for loading modules asynchronously, yet we often chose to bundle the code into once package anyway. It seems like the preferred choice has always been bundled code, since it means fewer HTTP requests and less worrying about dependency availability. This was one of the things I definitely wanted to address with ImportJS.

Concern#2 – Module Organization

The second obstacle I’ve found in JavaScript is finding the “best” method to write modular code. In some other languages such as Java or ActionScript, you’re forced to use a class-based structure for your code with one specific entry point for your application. In JavaScript anything goes, so there has never really been a specific pattern that everyone “must” use. I actually consider this a great thing since it’s part of what makes JavaScript so flexible, but in my opinion the moment you perform logic within the root scope of your module you’ve already made a mistake. So while it’s awesome that JavaScript doesn’t boss you around, I feel it leaves the developer open to some poor coding patterns that hurt scalability.

My Module Principles

I’ve narrowed down my main thoughts on solving the above to the following 3 principles I now use for myself when writing JavaScript applications. These principles obviously don’t fit the needs of every application, but I’ve used them frequently enough where I can comfortably say they should cover the overwhelming majority of use cases:

1. A JavaScript application should have a single entry point.
2. External (third-party) JavaScript dependencies should always be resolved, loaded, and executed before your application code.
3. Internal JavaScript dependencies should be resolved and loaded prior to any logic execution /entry point.

Principle #1 in particular is a common feature of many module loaders which need to know how to initialize your application once its dependencies are resolved.

As for Principle #2, I’ve seen most module loaders handle this in such a way that you must “shim” third-party modules that don’t fit the loader’s criteria. This way you can reference those third-party dependencies as if they were native to your application, and the load order can be determined while resolving the dependency graph. I’ve found in my experiences that a big benefit of doing that is when you need the ability to include multiple versions of the same third party library in your application, because you can create aliases for each unique version. Though if you do find yourself needing multiple versions of a library in your application, you might want to think twice about how you’re implementing things. In any case, I came to the conclusion that it’s far cleaner and straightforward if all third party dependencies were already resolved before your core application code, which is one of the things that ImportJS handles naturally.

Then finally Principle #3 is where ImportJS strays away from typical module loaders. Let’s take a look at how.

 Your Typical Module Loader

1. Load a config
2. Load the entry point source file
3. Parse AST of loaded source file for required dependencies
4. For each dependency in this source file that has not been loaded yet, start loading it
5. Repeat Steps 3 through 5 until all dependencies have been met for a particular source file’s code
6. Execute modules immediately the moment their dependency requirements are met (entry point is naturally last)

Note: The above list is for asynchronous loading. If the code was bundled in one package then similar logic is used at compile time, but you can assume the fetch time for a dependency would become negligible.

ImportJS Module Loading

1. Load a “config” (optional, may not even be necessary)
2. Load all of your module files in whatever order (If bundled it’s almost instant, if async it loads dependencies as it finds them via Regex )
3. Execute all of your modules now that they have all been loaded
4. Initialize application entry point

This might seem confusing at first, but the key difference between these two approaches is specifically step #6 from typical module loaders. But in order to understand this, I want to draw a clear distinction between loading a module, executing a module, and initializing part of a module.

•  loading a module means downloading the source code, but not yet evaluating it
• executing a module means evaluating (running) the module’s source code
• initializing part of a module means executing specific logic within a module (whether that be “new Thing()”, or calling “doSomething()”.

With your typical module loader, the loading and execution steps are extremely coupled, and this is because the completion of loading a module’s dependencies will result in the immediate execution of the code inside of that module. What bothers me about this is that I like to think of an application as a single entity. Yes that entity might be made of smaller parts, but I don’t want those smaller parts to do anything until I know they have all finished loading.

The example I’m going use to demonstrate why this is an issue is about the dreaded circular dependency concept, which requires ugly work-arounds in some loaders. Now before you get alarmed, I am not here to promote circular dependencies. However I think it’s import to acknowledge why this can’t be handled easily in other module syntaxes, and it’s the perfect way to demonstrate how ImportJS naturally solves this if you follow some basic guidelines. Let’s see a simple circular dependency with AMD syntax using a “brother” and “sister” object that depend on each other’s existences:

define('main', ['brother', 'sister'], function (brother, sister) {
  console.log('RequireJS: Loaded main');

  return {
    toString: function () {
      return "RequireJS: main, " + brother.name() + ", " + sister.name();
    }
  };
});
define('sister', ['brother'], function (brother) {
  console.log('RequireJS: sister is ready', brother);

  return {
    brother: brother,
      name: function () {
          return "RequireJS: sister [" + brother + "]";
      }
  };
});
define('brother', ['sister'], function (sister) {
  console.log('RequireJS: brother is ready', sister);

  return {
    sister: sister,
    name: function () {
      return "RequireJS: brother ["+ sister + "]";
    }
  };
});
require(['main'], function (main) {
  //Entry point for RequireJS
  console.log(main.toString());
});

If you executed the above code with RequireJS, you’re going to get a nice pretty “RequireJS: sister is ready undefined” in your browser console. Why is this? It’s because both “brother” and “sister” require each other as dependencies, but AMD syntax cannot return one of the module references until the other has finished executing. As a result, regardless of the load order of “brother” and “sister” only the one that was loaded first will have access to its sibling.

To elaborate – if “brother” was to be loaded first, it needs “sister” before it can enter its execution state. This forces “sister” to enter the execution state before “brother” gets a chance to complete, resulting in “sister” containing an undefined reference to her sibling.

Now let’s talk about why this is a non-issue with ImportJS using the same brother-sister example:

ImportJS.pack('main', function (module, exports) {
  var brother = this.import('brother');
  var sister = this.import('sister');

  module.exports = {
    toString: function () {
      return "ImportJS: main, " + brother.name() + ", " + sister.name();
    }
  };
  console.log('ImportJS: loaded main');
});

ImportJS.pack('sister', function (module, exports) {
  var brother;
  this.inject(function () {
    brother = this.import('brother');
    console.log('ImportJS: sister is ready', brother);
  });

  module.exports = {
    name: function () {
      return "ImportJS: sister [" + brother + "]";
    }
  };
});

ImportJS.pack('brother', function (module, exports) {
  var sister;
  this.inject(function () {
    sister = this.import('sister');
    console.log('ImportJS: brother is ready', sister);
  });

  module.exports = {
    name: function () {
      return "ImportJS: brother [" + sister + "]";
    }
  };
});

ImportJS.compile();
console.log(ImportJS.unpack('main').toString());

You will not get a single undefined anywhere in your console output. The reason for this is what I call deferred dependency injection. In simpler terms, this means delaying dependency resolution until after the execution step, but before the initialization step. This takes place through the “this.inject()” mechanism in ImportJS, which is nothing more than a function callback that gets executed only at the time you call ImportJS.compile() (i.e. pre-unpack()). Without this feature, you’d get the same issue as the AMD example.

The compile() function will go through all of the modules you have stored, and execute them one by one. Any import() calls outside of an inject() function will be resolved immediately. But any code within the inject() function is delayed until every single module has safely completed its execution step. The end result of this is having completely resolved dependencies at the beginning of application run-time without any hacks!

But despite this feature, please don’t go littering your applications with circular dependencies! They are still considered an anti-pattern in the dev community and should be used sparingly.

Conclusion

Now, there is one last thing to keep in mind with all of this and that’s the fact that none of this will be useful without following some sort of guidelines. So here’s how I prefer/recommend writing modules under ImportJS (and in general really):

1. Modules should export 1 “class”, and 1 class only (can be a function or object, though a function is preferable)
2. Modules should never execute any logic outside of the class’s definition (Let the functions in the class do the setup, not the module)
3. Module dependencies should always be resolved in the inject() phase (except in the case of a parent-child relationship, where the child needs the parent to be extended)
4. Write your modules in individual files for development and concatenate them to test rather than loading them through the preload() feature
5. Have a single entry point for the application, preferably inside of a class constructor if you want to follow a more traditional practice
6. Separate concerns for 3rd party dependencies by either loading them separately into the global namespace, or creating ImportJS wrappers/plugins.

If you follow these above steps and keep your modules nice and compact, writing JavaScript with ImportJS should start to resemble that of a more traditional application. I think the patterns used here have been heavily battle-tested over the years in other platforms, which to me warrants giving it a shot in the JS world. I certainly hope that you’ll consider giving ImportJS a try in projects of your own 🙂