Thank you esbuild
SQL Frames has several goals in terms of how it is distributed. As a library that can run within the browser or in the backend using Node or within an electronjs app. In addition, it should also be delivered as esm module or umd and cjs modules. Initially much of this bundling was managed with Rollup. In the initial stages when the entire codebase was in pure JavaScript the build time was around 2 seconds. Moving to TypeScript increased it to around 9 to 10 seconds when targeted to ESNext and almost double when targeted to older ES standards. This was simply killing productivity which was unacceptable. Yarn workspaces and esbuild is how this has been solved.
Most people working on the frontend technologies should be knowing that JavaScript has been evolving over the last few years with newer language proposals each year. It's possible to completely avoid these new and exciting features and stick to years old standards or use the latest and in some cases not even finalized features but in order to make them work universally on most browsers, the code has to be transpiled.
When the codebase was still in JavaScript, the choices were rollup.js and babel plugin to transpile to older ES versions. For example, SQL Frames extensively uses JavaScript private member variables (not necessarily for its runtime security/isolation but for proper abstraction and performance) and these were not supported in Safari (till 14.1) and still not in Firefox (only available in Developer build). Chrome on the other hand has private member variables support for sometime and the performance difference with and without transpiling is obvious. Babel was used for the transpilation.
After migrating to TypeScript, the tsc compiler itself has options to specify which ES standard to target and accordingly takes care of generating the appropriate JavaScript code. This essentially means it is possible to get rid of babel. However, I had multiple targets and interestingly what worked best (lower build time) is to target ESNext with TypeScript and to target older versions with Babel which also supports TypeScript but without typechecking (which was OK because one of the targets was doing the type checking).
rollup.js has caching mechanism and TypeScript has caching and incremental build features. In spite of using these features the build time was still not fast enough. While some of these techniques helped, there was still a lot of difference in build time. Coming from using golang on a large project prior to this, the build time latency and the resulting productivity loss was very obvious and something had to be done.
That's when I discovered mono-repos and yarn workspaces and how to split the code to reduce build time. Note that the code is already nicely structured into several packages (directories) but still it was a bit spaghetti as JavaScript itself doesn't impose any structure. With rollup.js occasionally there were circular dependencies while TypeScript seems to be much smarter to avoid some of those circular dependencies.
After careful consideration, the entire codebase had been once again completely restructured to make use of yarn workspaces and create explicit packages. Incidentally TypeScript compiler even supports this concept and it is possible to build across these isolated packages using --build flag. With this, the build times have improved as only the changed packages and packages that depend on them are compiled and not the overall codebase. While this solved the compilation and bundling timings the build times for targets requiring transpilation still suffered.
Then I chanced upon esbuild and after reading the project and trying it, I must say, it is a fantastic project. The bundling and transpiling times have gone down to sub-second! So, while incremental compilation time could occasionally spike up to a few seconds, most of the time the entire compilation to transpiling and bundling is all done within a second.
After moving to esbuild, there is no need to use rollup.js and babel at all. Incidentally esbuild is written in golang and without hesitation it is the fastest JavaScript bundler in the world.
There is one problem with esbuild that may or may not be tolerable for your use case. It has a single pass design which avoids certain types of optimizations (for example, private fields may be converted to WeakMap based fields in some scenarios). Hence, if performance is critical to the application, then it may be better to adopt a dual strategy where the dev builds are based on esbuild and the production builds based on rollup.
I suggest you give it a try. Even if you have a complex build process, figure out a way to isolate parts that can be done with esbuild to shave off several seconds of the build time during development to boost dev productivity.