Thank you Monaco Editor
SQL Frames provides the ability to create calculated fields by writing JavaScript code. I deliberated whether to provide a much simpler formula authoring system like Excel keeping in mind the goal of providing a low-code platform but in the end it was decided to not have any proprietary scripting language that someone has to learn and instead just rely on JavaScript which is already one of the most popular languages. This meant, the need for a simple online IDE that allows authoring JavaScript.
In addition to authoring calculated fields, SQL Frames has a companion app called SF REPL that provides an integrated experience to work with SQL Frames API. This also required authoring JavaScript.
Initially, I chose Ace Editor, Amazon's Cloud 9 editor. And it worked out great for both use cases. While picking the online IDE, I also stumbled across the Monaco Editor from Microsoft. Monaco editor is a factored out code out of the Visual Studio which I already use and like. So, while I used Ace editor I was in a dilemma and put it on roadmap to perhaps switch to Monaco later on.
One of the things I wanted to provide in the REPL is auto-suggest capability of the SQL Frames API. Ace Editor has some auto-suggest capability and I believe it is even possible to customize it. So, auto-suggest was in the roadmap.
A few months later, the entire SQL Frames code base moved to TypeScript and that's when I figured that I already have type definitions and so it should be possible to directly use those as the auto-suggest API rather than having to do something else that I should be creating. But as I researched on Ace editor's ability to automatically support this, I realized a few things.
- Ace editor has support for many languages but the code snippets are not parsed using a proper parser.
- Instead individual lines are parsed and for JavaScript it uses JSHint to provide auto-suggest.
This essentially meant that I had to do lot of custom code to get the auto-suggest working. So, time for a break and go back and evaluate this in the context of other roadmap item which is to perhaps using Monaco Editor. Interestingly, Monaco Editor has a completely different architecture and it comes with actual language servers that run as Web Workers. This meant that it can truly parse TypeScript and JavaScript languages. It also has the feature to load existing TypeScript API to use in auto-suggest. Exactly what I was looking for.
So, one final day I decided to move from Ace Editor to Monaco Editor. The rest is history and the time spent in making the switch was worth it.
There are a few challenges I ran into while integrating the Monaco Editor. It comes with it's own loader. I already make use of requirejs for loading rest of the dependencies. So, I had to carefully workout a way to load Monaco Editor along with rest of the dependencies. Overall it wasn't all that hard to do it. The end result is a very polished user experience as can be seen in this website.
Below are a couple of examples of the REPL with Monaco editor.
Any value
SQL Frames REPL contains three components, STDIN, STDOUT and STDERR, just like any standard REPL environment. Monaco editor is used for STDIN. The various types of data that can be created using SQL Frames such as the DataFrames, charts and basic data types like arrays and objects all have visualizations that STDOUT can display. Each REPL instance can be thought of as an isolated function and every time it is executed, it reexecutes the entire code replacing the STDOUT with the results that are explicitly returned by the code written in the editor.
Animation
JavaScript has the concept of generators that can yield a stream of data. SQL Frames REPL has native support for such generators with the ability to pause and resume the stream being displayed in the STDOUT. This is meant to provide support for animation with the visuals.
As SQL Frames expands its functionality within the core layers or within the plugins, their built-in visualizations will be integrated into the SF REPL to provide an integrated experience.