Jelly: A Tale of Two Phases...

[SlightlyLoony]

Long post today, folks, on a topic that confuses many. We begin with a story:

Not so very long ago, in a government bureaucracy that was part of a state famous for its ineffective and infuriating bureaucracies, a very junior functionary named Oscar invented something that made his bureaucracy faster and more efficient. This, of course, doomed Oscar's bureaucratic career, which until that moment was quite promising — in another ten years or so, he expected he'd be personally denying and obstructing the bureaucracy's customers, just like his seniors were doing today.

Oscar's job was very simple. He ran the "forms room", which was the place every bureaucrat went to get information. For example, let's say that Senior Obstructionist McNot (early photo at right) needed information about citizen Jones to deny process a building permit. McNot would come to Oscar's window and rudely demand (honing his infuriation skills) Form 283.65(a)/12.3, filled out for citizen Jones. Oscar then went through these steps:

First Oscar got "the book" for the form in question. Each of these books had elaborate instructions for the form Oscar was to draft. That's right, Oscar had to draw each of these forms from scratch! The instructions took into account the role of the requesting bureaucrat — a Senior Obstructionist might be allowed to see the citizen's street number, whereas a mere Obstructionist could not. Likewise, a Senior Principal Obstructionist might be allowed to see the last four digits of the citizen's Social Security Number, whereas the Senior Obstructionist could not. On a single form there might be hundreds of instructions for things like this. Then there were the appearance-related rules. The Obstructionist's form was laid out in an ugly grid, using only black ink — but the Vice President of Obstructionism's form, well, that was something else altogether! That form was a splendorous thing, ornate, colorful, and downright weighty. Poor Oscar's fate was to wade through the morass of rules, figure out what this particular requester's form should look like, and then draft the thing. Sometimes this drafting took hours. Finally, when the drafting was finished, Oscar would fill in all the blanks he'd just created — a task that generally took only a few minutes.

One morning after Oscar had been there for a few years, it occurred to him that he often had the same bureaucrat asking him for the same form repeatedly. He supposed this had something to do with their job function, but that didn't really matter. On this particular morning, Junior Principal Infuriationist Mayer stopped by to ask for Form 83.12(b1)/59a (electrical outlet inspection reports) for the sixth time that day. On each of the previous occasions, Oscar drew the entire form from scratch, with the identical result each time — and then he filled in all the blanks with the data for the particular building in question. That's when he had his brainstorm: he could create that form full of blanks just once, then copy it on the fancy copying machine sitting idle in the corner. Then he'd fill in the blanks on the copy and give it to the bureaucrat. He'd then take the original and file it away under the requesting bureaucrat's name, in order by the form number. Then the next time Junior Principal Infuriationist Mayer asked for Form 83.12(b1)/59a, Oscar would simply look in that file, find the original, copy it, and fill in the blanks. Instead of hours of painstaking labor interpreting the rule book and drafting the form, he'd be done in just a few minutes.

Of course, they fired Oscar later that day — as increasing efficiency is nearly the only thing that could get a bureaucrat fired, and the bureaucracy defends itself swiftly. Oscar found himself out on his ear. Poor Oscar.

Now back to Jelly-land, where Oscar's insight is put to work as well. The Jelly templates are the equivalent of Oscar's rule books: they contain the elaborate instructions (sometimes hundreds of lines long) that tell the Jelly engine exactly how to build a form (page). These forms are first constructed with blanks (we call this Jelly Phase 1) and they are filed away (cached) in this state — and they are stored separately for each user. Jelly does this for the same reason that Oscar wanted to: the time-consuming, complex task of building the basic form-with-blanks need only be done once for each user and form combination. Then when a ServiceNow user actually requests a form (i.e., visits a particular page), the Jelly engine first looks in that cache to see if the form-with-blanks has already been prepared, and if so, reuses it. Then the form-with-blanks is filled in with actual data (a much faster operation that we call Jelly Phase 2). All of this is accomplished via a single Jelly template for a particular form — specially crafted so that some things happen in Phase 1, and some in Phase 2.

Here's an example of such a Jelly template — SlightlyLoony's custom listing of Configuration Items:

<?xml version="1.0" encoding="utf-8"?>
<j:jelly>
    <div>
        <g:evaluate>
            var user = gs.getUser();
            var username = user.getFirstName() + ' ' + user.getLastName();
        </g:evaluate>
       
Slightly Loony's Configuration Items for ${username}
        <g2:evaluate>
            var gr = new GlideRecord('cmdb_ci');
            gr.addNotNullQuery('name');
            gr.orderBy('name');
            gr.setLimit(25);
            gr.query();
        </g2:evaluate>
        <table>
            <tr>
                <td>Name</td>
                <td>Class</td>
            </tr>
            <j2:while>
                <tr>
                    <td>$[gr.name]</td>
                    <td>$[gr.sys_class_name.getDisplayValue()]</td>
                </tr>
            </j2:while>
        </table>
       
First 25 Configuration Items only...
    </div>
</j:jelly>

Most of this should look familiar to you by now. There are two new things in here: the tags that start with j2: and g2:, and the JEXL expressions that use square brackets instead of curly braces (like $[gr.name]). Both of these indicate steps that will happen in Phase 2 — they represent the "filling in the blanks" part of what Oscar did.

Let's go through this in a little more detail. What's going to happen in Phase 1? The Jelly engine knows which Phase it's processing, and in Phase 1 it will actually perform all the Phase 1 tags (those without the "2" in them) and all the Phase 1 JEXL expressions (the ones with curly braces). The Phase 2 tags and JEXL expressions simply get copied to the output of Phase 1. Here's what the Phase 1 output of the example above looked like when I executed it:

<?xml version="1.0" encoding="utf-8"?>
<j:jelly>
    <div>
       
Slightly Loony's Configuration Items for Mostly Loony
        <g2:evaluate>
            var gr = new GlideRecord('cmdb_ci');
            gr.addNotNullQuery('name');
            gr.orderBy('name');
            gr.setLimit(25);
            gr.query();
        </g2:evaluate>
        <table>
            <tr>
                <td>Name</td>
                <td>Class</td>
            </tr>
            <j2:while>
                <tr>
                    <td>$[gr.name]</td>
                    <td>$[gr.sys_class_name.getDisplayValue()]</td>
                </tr>
            </j2:while>
        </table>
       
First 25 Configuration Items only...
    </div>
</j:jelly>

So what happened here? Well, the g:evaluate tag at the beginning is gone — that's because Jelly evaluated it in Phase 1, and it isn't needed in Phase 2 at all. The JEXL expression in curly braces was evaluated, and the text it produced ("Mostly Loony") is now embedded in the title paragraph. The HTML tags, square bracketed JEXL expressions, and the Phase 2 tags just got passed straight through. This is what the Jelly engine caches, just like Oscar filed away his forms with blanks.

In Phase 2 the Jelly engine starts with the cached Phase 1 results (like above), and then evaluates all the Phase 2 tags and square bracketed JEXL expressions. It also strips away the outer j:jelly tag. In our case, here's what we end up with (showing just the first few lines from my system):

<div>
       
Slightly Loony's Configuration Items for Mostly Loony
        <table>
            <tr>
                <td>Name</td>
                <td>Class</td>
            </tr>
                <tr>
                    <td>Acrobat</td>
                    <td>Software</td>
                </tr>
                <tr>
                    <td>Activity Monitor</td>
                    <td>Software</td>
                </tr>
                <tr>
                    <td>Address Book</td>
                    <td>Software</td>
                </tr>
                <tr>
                    <td>Adobe Flash Player</td>
                    <td>Software</td>
                </tr>
                <tr>
                    <td>Adobe Reader</td>
                    <td>Software</td>
                </tr>
        </table>
       
First 25 Configuration Items only...
    </div>

You can see that in Phase 2, we processed the j2:while tag, building all the rows in our output table. You can also see that the square bracketed JEXL expressions got resolved into data fields from the CMDB. That's Phase 2!

Understanding the Phase 1/Phase 2 Jelly process is key to understanding the rest of what we're going to talk about with Jelly, not to mention your own Jelly projects. Make sure you are absolutely clear on this topic, and by all means fire questions at me (in the comments, please) if any of what I've said here isn't doing the job.

But back to Oscar: there's good news after he was fired. After Bruno (the bureaucrat's security guy) tossed him out the front door, Oscar picked himself up and spent the next 2 weeks plotting his revenge. This involved Twitter, Facebook, compromising photos, and press releases — culminating in a lovely demonstration that the Governor ended by firing all the bureaucrats (which, of course, were quickly replaced from the apparently bottomless pool of bureaucrat wannabes). Then Oscar started his own company, making forms preparation software — and sold it to MultiMegaSoftware, Inc. He retired at the ripe old age of 28, and is at this very moment sipping a mai tai, served up by a decorative bikini-clad serving wench on an island in Indonesia. Which he owns...