Anatomy of JS module systems and building libraries!

Lately, we have all been stuck with these giant words “JavaScript Modules”. Each one of us wonders what we can do with them, how they even play a vital role in our daily life and so on.

So what the heck is JS module System?

As JavaScript development gets more and more common, namespaces and dependencies get much more difficult to handle. Different solutions were developed to deal with this problem in the form of module systems.

Why understanding JS Module System is important?

When we say an application is modular, we generally mean it’s composed of a set of highly decoupled, distinct pieces of functionality stored as modules. As you probably know, loose coupling facilitates easier maintainability of apps by removing dependencies where possible. When this is implemented efficiently, it’s quite easy to see how changes to one part of a system may affect another.

Story Time!

Telling stories is as basic to human beings as eating. More so, in fact, for while food makes us live, stories are what make our lives worth living – Richard Kearney?

Why am I even talking about all this?

So, the next step was to publish the library. Huh???

This was the toughest part because there were a bunch of things striking my head like:

1. How to make it tree shakeable?
2. What JS module systems shall I target (commonjs, amd, harmony)?
3. Shall I transpile the source?
4. Shall I bundle the source?
5. What files to publish?

Every one of us has these kinds of questions bubbling in our heads while creating a library. Right?
I’ll try to address all the above questions.

Different Types of JS Module Systems?

1. CommonJs

• Implemented by the node.
• Used for server-side when you have modules installed.
• No runtime/async module loading.
• import via “require”.
• export via “module.exports”.
• When you import you get back an object.
• No tree shaking because when you import you get an object.
• No static analyzing as you get an object so property lookup is at runtime.
• Always get a copy of an object so no live changes in the module itself.
• Poor cyclic dependency management.
• Simple Syntax.

2. AMD: Async Module Definition

• Implemented by RequireJs.
• Used for client-side (browser) when you want dynamic loading of modules.
• import via “require”.
• Complex Syntax.

3. UMD: Universal Module Definition

• Combination of CommonJs + AMD i.e Syntax of CommonJs + async loading of AMD.
• Can be used for both AMD/CommonJs environments.
• UMD essentially creates a way to use either of the two, while also supporting the global variable definition. As a result, UMD modules are capable of working on both client and server.

4. ECMAScript Harmony (ES6)

• Used for both server/client side.
• Runtime/static loading of modules supported
• When you import you get back the binding’s value (actual value).
• import via “import” and export via “export”
• Static analyzing — You can determine imports and exports at compile time (statically) — you only have to look at the source code, you don’t have to execute it.
• Tree shakeable because of static analyzing supported by ES6
• Always get an actual value so live changes in the module itself.
• Better cyclic dependency management than CommonJs

That was all about different types of JS module systems and how it has evolved. Although the ES Harmony module system is supported by all the tools and modern browsers, we never know while publishing libraries how our consumers will utilize it. Hence, we must always ensure that our libraries should work in all environments. Let’s dive deeper and design a sample library to answer all the questions related to publishing a library in a proper way.

I’ll build a small UI library, source on GitHub, and then share all my experiences and explorations for transpiring, bundling and publishing it.

Here we have a small UI library which has 3 components Button, Card and NavBar. Let’s transpile and publish it step by step.

Best practices before publishing?

1. Tree Shaking?

• Tree shaking is a term commonly used in the JavaScript context for dead-code elimination. It relies on the static structure of ES2015 module syntax, i.e. import and export. The name and concept have been popularized by the ES2015 module bundler rollup.
• Webpack and Rollup both support Tree Shaking but for that, we need to keep certain things in mind, so that our code is tree shakeable.

2. Publish all module variants

• We should publish all the module variants i.e UMDand ES, because we never know which browser/webpack versions our consumers might use this library/package in.
• Though all the bundlers like Webpack and Rollup understand the ES module if our consumer is using Webpack 1.x then it cannot understand the ES module.

// package.json
{
"name": "js-module-system",
"version": "0.0.1",
...
"main": "dist/index.js",
"module": "dist/index.es.js",
...
}

The main fieldof package.json file is usually used to point to the UMDversion of the library/package.

How can I release the ES version of my library/package?

The module fieldof package.json is used to point to the ES version of the library/package. Previously many fields were used like js:next ,js:mainbut module is now standardized and used by bundlers as a lookup for ESversion of the library/package.

Less known fact: Webpack uses resolve.mainfields to determine which fields in package.json are checked.

Performance Tip: Always try to publish ES version of your library/package as well because all the modern browsers now support ES modules. So you can transpile less and ultimately you’ll end up shipping less code to your users, hence your applications performance boosts up.

So now what’s next? Transpilation or Bundling? What tools to use?

Ah, here comes the trickiest part! Let’s dive in. ?

Webpack vs Rollup vs Babel?

These are all the modern day tools we use in our day to day life to ship our applications/library/packages. I cannot imagine the modern generation web development without them *BLESSED*. Hence we cannot compare them, therefore it’s a wrong question to ask! ❌

Each tool has it’s own benefits and serves different purpose based on the needs.

It’s a wrong question to ask. Each tool has it’s own benefits and serves different purpose based on the needs.

Webpack: Webpack is a great module bundler ? that is widely accepted across the globe and mostly used for building SPAs and gives you all the features out of the box like code splitting, async loading of bundles, tree shakingetc. It uses CommonJS module system.

PS: Webpack-4.0.0 alpha is already out ?, hopefully with the stable release it will become the universal bundler for all types of module system.

RollupJS: Rollup is also a module bundler similar to Webpack but the main building block of rollup is that it follows new standardized format for code modules included in the ES6 revision, so you can use this to bundle ES module variant of your library/package. It doesn’t supportsasync loading of bundles.

Babel: Babel is a transpiler for JavaScript best known for its ability to turn ES6 version into code that runs in your browser (or on your server) today. Remember it just transpiles and not bundles your code.

Use Rollup for libraries and Webpack for apps.

Transpile(Babelify) the source or Bundle it?

Again there’s a story behind it. ?

Most of my time was spent to figure out answer to this question when I was building this library. I started digging out my node_modules to lookup all the great libraries and check out their build systems.

After looking at different libraries/packages build output, it gave me a clear picture of what different strategies and purpose the authors of these libraries think before publishing their library/package. Below are the observations.

As you can see the above image, I’ve divided them into 2 groups based on their characteristics:

1. UI Libraries (styled-components, material-ui)
2. Core Packages (react, react-dom)

If you’re a good observer ? you might have figured out the difference between these two groups

1. UI Libraries

• There is a dist folder that has the bundled and minified version for ESand UMD/CJS module system as a target.
• There is a lib folder that has the transpiled version of the library.

2. Core Packages

• There is just one folder which has bundled and minified version for CJS orUMD module system as a target.

But why there is a difference in build output of UI libraries and Core Packages?

UI Libraries

• Imagine if we just publish the bundled version of our library and host it on CDN our consumer will use it directly in a<script/> tag. Now if my consumer wants to use just the <Button/> component he has to load the entire library. Also in a browser there is no bundler which will take care of tree shaking and we’ll end up shipping the whole library code to our consumer. ❌

<script type="module">
import {Button} from "https://unpkg.com/uilibrary/index.js";
</script>

• Now if we simply transpile the src into lib and host the lib on a CDN, our consumers can actually get whatever they want without any overhead.

“Ship less, load faster”. ✅

<script type="module">
import {Button} from "https://unpkg.com/uilibrary/lib/button.js";
</script>

Core Packages

• Core packages are never utilized via <script/> tag, as they need to be part of main application, hence we can safely release the bundled version (UMD, ES) for these kind of packages and leave up to the consumers build system. Example they can use the UMD variant but no tree shaking or they can use the ES variant if the bundler is capable to identify and get the benefits of tree shaking.

// CJS require
const Button = require("uilibrary/button");
// ES import
import {Button} from "uilibrary";

But…

Transpile(Babelify) the source or Bundle it?

UI Library

1. Transpile the source with babel with es module system as a target, place it inlib. We can even host the lib on a CDN.
2. Bundle and minify the source using rollup, for cjs/umd module system and es module system as a target. Modify the package.json to point to the proper target systems.

// package.json
{
"name": "js-module-system",
"version": "0.0.1",
...
"main": "dist/index.js",      // for umd/cjs builds
"module": "dist/index.es.js", // for es build
...
}

Core Packages

1. We don’t need the lib version in this case.
2. Bundle and minify the source using rollup, for cjs/umd module system and es module system as a target. Modify the package.json to point to the proper target systems same as above.

Tip: We can host the dist folder on CDN as well, for the consumers who are willing to download the whole library/package via <script/> tag.

How to Build?

We should have different scripts for each target system in package.json . You can find the rollup config in the in the GitHub repo.

// package.json
{
...
"scripts": {
"clean": "rimraf dist",
"build": "run-s clean && run-p build:es build:cjs build:lib:es",
"build:es": "NODE_ENV=es rollup -c",
"build:cjs": "NODE_ENV=cjs rollup -c",
"build:lib:es": "BABEL_ENV=es babel src -d lib"
}
...
}

What to Publish?

• License
• README
• Changelog
• Metadata(main , module, bin) — package.json
• Control through package.jsonfiles property

In package.json"files" field is an array of file patterns that describes the entries to be included when your package is installed as a dependency. If you name a folder in the array, then it will also include the files inside that folder.

We will include lib and dist folder in "files" field in our case.

// package.json
{
...
"files": ["dist", "lib"]
...
}

Finally the library is ready to publish just type npm run build command in the terminal and you can see the following output. Closely look at the dist and lib folder. ?

Ready to publish ?

Wrap up!

Wow! Where does the time go? That was a wild ride, but I sincerely hope it gave you a better understanding of JavaScript Module system and how you can create your own library and publish it by ensuring following things:

1. Make it Tree Shakeable. ?
2. Target at least ES Harmony and CJS module systems. ?
3. Use Babel and Bundlers for libraries. ?
4. Use Bundlers for Core packages. ?
5. Set module field of package.json to point to the ES version of your module (PS: It helps in tree shaking). ?
6. Publish the folders which has transpiled as well as bundled versions of you module. ?

Trending?

1. Webpack-V4 alpha released. ?
2. ParcelJs: Blazing fast, zero configuration web application bundler. ?
3. Turbo: 5x faster than Yarn & NPM, and runs natively in-browser ?

Thanks to Juho Vepsäläinen and Lakshya Ranganath for their reviews & feedbacks, Sean T. Larkin and Tobias Koppers for sharing the insights of webpack at ReactiveConf, Addy Osmani for sharing workings of different JS module Systems in “Writing Modular JavaScript With AMD, CommonJS & ES Harmony”.