Extreme Javascript Minimization

Compressing javascript source code is a tricky undertaking. To my knowledge, the only domain specific javascript compressor I’ve found is JSZap, which is a hefty undertaking in terms of tree-teardowns. There’s also the proposals for Javascript Diffing, which would save tons of bandwidth every day for GMAIL, but that’s a very generic algorithm. Most of the web seem content to simply rely on GZIP for compression; Which seems fine when your JS is under 15k; But for a 25mb JS file, you’ve got problems.

BTW, I quickly want to take a second and discuss some terminology :

• Minification is the act of reducing data such that it can be consumed without processing by the underlying system
• Compression is the act of reducing data length such that it requires processing to recover it to a state in which the system can operate on it.

GZIP is a compressor; it reduces the data from its’ functional form to an interpreted form

They may seem similar, but as I’ll show in this article, the ramifications in mind-set between the two are quite large.

Automated Javascript Minifiers

At the top of our chart are the automated set of minifiers that fit into your production pipeline. Tools like UglifyJS, JSMin, YUI, rJSMin, Dojo ShrinkSafe, Ajax Minifier, and Closure. Much like we saw with CSS minifiers, there’s no dominant, one-size-fits-all JavaScript minifier either. Most of these systems work by compiling your Javascript into some sort of Abstract Syntax Tree representation, and then recovering from that doing various scope renaming and variable minimization. Although these automated minimizers do their job, they have recently been exposed to lack advanced optimizations that stupid robots have no idea how to do. Although I’m sure there’s a whole separate discussion about if those hand-made adjustments hit the gzip threshold or not; but let’s skip that for now.

The JS Demo Scene

With the addition of WebGL to the HTML5 stack, it’s opened up a new door for graphics-programming-hackers to bring their love of the demoscene to the web. If you’re unfamiliar, DemoScene applications attempt to pack as much excellence into as few bites as possible. One of the most popular ones is .kkreiger who fits an entire First Person Shooter, complete with weapons, particles, textures, enemies, animations, post processing, lens flares, sound, music and physics into a single 64k executable. This is such an amazing task, especially considering that the video showing off the game is larger than the actual game itself.

This combination of hacking, graphics, and feedback speaks to the core of many engineers, in fact, in various parts of Europe, there’s even a notion of a DemoScene Party which is taken pretty seriously; awarding prizes and swag to the best demo submitted under a byte threshold.

Once these hackers set their attention towards javascript, fun things were bound to happen. Dominantly these demos extract their awesome from source-code level hacks that allow them to squeeze as much functionality into as few bytes as possible, which usually includes using procedural algorithms to generate game content.

Note that the demo scene doesn’t accept browser-based GZIP as a contribution; the raw Javascript file must be under the threshold specified. This means in order for demo hackers to meet their quota, they have to push custom compressors onto their javascript and include the decompressor in the javascript too.

This has given rise to an awesome set of inline extractors for javascript which address a unique orthogonal need : Compress the source as much as possible, with the least number of bytes for the compressor.

JsCrush, jsPacker, Packify round up the top of these extractors, effectively applying a longest-string replacement process to their source (somewhat like an LZ77). More advanced versions like jsEXE, jsSFX3, and CrunchMe all produce more advanced versions of these, mostly due to advanced compression codecs and the ability to pack javascript as GZIP compressed PNG data, inline.

This has brought around a whole new development experience; how low can you go

Note how these systems are true  Kolmogorov measurements because the encoder exists alongside the data stream

Extreme minification

The demo scene folks have done a great job of compressing their JS data through various means, however there’s a new generation of JS hackers that push past those automated compression methods into hand-generated minification, which often times goes farther than any automated process.

As an example, I here is a Base64Encoder in 140bytes of javascript:

function(a,b,c,d,e,f,g){for(g=c=e="";(d=a.charCodeAt(g))&&(c+=/.{8}$/(1e7+d.toString(2))),f=c.substr(g++*6,6);)e+=b[parseInt(f,2)];return e}

Not impressed? Fine, here’s a 140byte music synth (try it here):

function(f){for(var t=0,S='RIFF_oO_WAVEfmt '+atob('EAAAAAEAAQBAHwAAQB8AAAEACAA')+'data';++t<3e5 data-blogger-escaped-eval="" data-blogger-escaped-f="" data-blogger-escaped-return="" data-blogger-escaped-s="" data-blogger-escaped-span="" data-blogger-escaped-tring.fromcharcode="">

Most of these demos adhere to a new style of minmification, that, by my eyes, seems damn near impossible to automate.

For example, take this presentation on JSGolf lessons, which present how to create a 140byte tetris game :

function(a,b,c,d){d+=c;    return[d<0 a="<br" amp="" b="" class="kix-line-break" d="">    parseInt((a|b<    .toString(d=32,b=new Date&2|1)    .replace('v',''),d):    a|b<

Granted, there’s some parts missing, the above code assumes that you’ve got some canvas and other data available, but the idea is there, and pretty cool. The autor from aem1k.org also lists some great resources for packing numbers in JS as well.

Extreme obfuscation

I know it’s not in the same boat, but I feel like I should mention some really awesome javascript hacks out there that you should play with. Recently, it’s been shown that there’s a transform which will take Javascript and represent it in only 6 symbols; Note that the represented version is still working and valid. Simply pass it to the browser, and things will execute as normal. Three known transforms are JJencode, AAencode and JSFuck, which all encode things using various methods. In general, these provide an interesting obfuscation method for javascript, and bring up some interesting questions about compression.

Large compression wins occur when you can reduce the number of potential symbols (ie histogram reduction) alongside the number of symbol frequencies. These obfuscators clearly achieve the first method; reducing all javascript down to only 6 characters. However the inflation caused by this process violates the 2nd process of compression; post-compressing the transformed data does yield some amazing compression ratios, however the inflation is so large that the overall file size is still larger than the original.

How is any of this helpful?

Although 60% of the web are images, Javascript maintains it’s spot as 2nd largest resource; And things will become increasingly difficult for Javascript file sizes, especially with technologies like Emscripten around. For example, the Javascript files for Unreal3’s HTML5 port are around 25MB of data; Post GZipped to 5mb. That’s insanely huge; and by my estimates, as web-apps become more powerful on mobile, we’ll see these types of footprints becoming more popular over time.

~Main

You can find Colt McAnlis here: