Data Dictionary Strategies

This article will introduce a few techniques for working with data dictionaries in ServiceNow. A Data Dictionary is a description/data model of a piece of information (Object). They are Object Literals containing nothing but data. Often times, the SNow community refers to these Objects as JSON.

Four methods will be introduced:

1. Object written as part of larger code or Stand Alone Script Include
2. Retrieving from Properties
3. Retrieving from Message
4. • 2 and 3 will then be refactored into one function
5. Retrieving from Table 

After introducing the four methods above, the last method, Retrieving from a Table, will be refactored for scale-ability.

Method 1:
Object written as part of larger code or Stand Alone Script Include

This is nothing more than an Object Literal hosting some data. 

var BasicDataOictionary = {
    name: 'MyName',
    age: 40,
    country: 'Country Namme',
    city: 'City Name',
    street: 'Street',
    houseNumber: '245C'

}

Method 2:

Retrieving from sys_property

The same example from Method 1 extracted from code and placed in a system property as JSON string.
Property values are limited to 4000 Characters.
This approach is for simple objects, and is often useful as an entry point for a larger application's configuration settings.

Extracting the JSON string and parsing it into an Object Literal

var BasicDataDictionarySysProperty = (function () {

    var basicDictionary = gs.getProperty('basic.data.dictionary');

    basicDictionary = JSON.parse(basicDictionary);

    return basicDictionary;
})();

Method 3:

Retrieving from UI Message

Same approach as Method 2 yet stored as a sys_ui_message
This method is a bit more extensible as the value of messages is 8000 characters long.

Extracting from sys_ui_message and parsing into Object Literal

var BasicDataDictionarySysUIMessage = (function () {

    var basicDictionary = gs.getMessage('basic.data.dictionary');

    basicDictionary = JSON.parse(basicDictionary);

    return basicDictionary;
})();

Given that Method 2 and Method 3 share precise behavior, refactoring two functions into one makes sense.

var BasicDataDictionaryRetriever = function (isSysProperty, name) {

    var basicDictionary = ((isSysProperty) ? gs.getProperty(name) : gs.getMessage(name));

    basicDictionary = JSON.parse(basicDictionary);

    return basicDictionary;

}

Method 4 and Last Approach

Building Object from a GlideQuery/Retrieving from Table

This method will use the Object Literal within the body of the function that will act as the dictionary template/format. Because it was extracted from an inline piece of code, it has to remain the exact format to be used by other code expecting it remain unchanged... as if it hand never changed.

It will also use de-facto SNow practices by using GlideRecord within the body of a business process function (a practice that is detrimental to scale-ability and maintenance).

var BasicDataDictionaryTable = (function () {

    var basicObject = {

        name: null,
        age: null,
        country: null,
        city: null,
        street: null,
        houseNumber: null

    }

    var dictionary = new GlideReord('table_name');
    dictionary.addQuery('identifier', 'basic');

    dictionary.query();

    if (dictionary.next()) {

        basicObject.name = dictionary.getDisplayValue('name');
        basicObject.age = dictionary.getDisplayValue('age');
        basicObject.address.country = dictionary.getDisplayValue('country');
        basicObject.address.city = dictionary.getDisplayValue('city');
        basicObject.address.street = dictionary.getDisplayValue('street');
        basicObject.address.houseNumber = dictionary.getDisplayValue('houseNumber');

    }

    return dictionary;

})();

Thoughts regarding Method 4

One of the characteristics of scale-able and robust code is separation of concerns. Method 4 should be divided to address the mixture of concerns that prevent the various parts from being shared.

Mixture of concerns:

1. Template should be stand alone for easy sharing
2. DB CRUD operations should be in a Database Layer to be shared. Sitting inside the function above helps only itself 
3. Logic for filling the values of the Data Dictionary should be in a class to be used to build any type of Object Literal
   
   

Lastly, because the table structure has a column for each data key, the table is inflexible. A more robust approach would be to re-engineer the table.

 
Table fields are:
      name, age, country, city, street, house_number

Refactoring Method 4 to address inefficient structure

Step 1:

Define the template format as its own object.
Note that this can be as a sys_properties, a sys_ui_message, or a table as demonstrated above.  For the benefit of this article, it will be assumed that the template is hosted as a Script Include.

 By placing template BASIC inside an Object, it allows future templates to be hosted at this location independently of one another.

var BasicDataDictionaryTemplates = {

    BASIC: {
        name: null,
        age: null,
        country: null,
        city: null,
        street: null,
        houseNumber: null

    }
}

Step 2:
Introduce a Factory to fill the data dictionary

Note that this is a very basic factory that does not address
nested children Data Dictionaries

A factory is functionality that builds one and only one Object
given a few parameters

var BasicDataDictionaryFactory = {

    getDictionary: function (record, dataTemplate) {
        var dictionary = {};

        while (record.next()) {

            var name = record.getDisplayValue('name');

            if (dataTemplate.hasOwnProperty(name)) {
                dataTemplate[name] = record.getDisplayValue('value');

            }

        }

        return dictionary;
    }
};

Step 3:
Introduce a Business Access Layer(BAL) or Business Logic Layer (BLL) from which all other business processes wishing to use DataDictionaries can tap into.

In a mature system, this layer might simply be a Mediator: (traffic cop) pointing to other functionality to carry out logic.

In this article, the BAL will have only one method: entry point to obtain the data dictionary

var DataDictionaryTableBAL = {

    getBasicDataDictionary: function () {
        var basicObject = DataDictionaryTableDAL.getBasicDataDictionary('basic'); //get data dictionary by identifier
        var dictionary = BasicDataDictionaryFactory.getDictionary(basicObject, BasicDataDictionaryTemplates.BASIC);

        return dictionary;
    }
};

Setp 4:
Introduce a Database Access Layer (DAL) to host any CRUD operations related to DataDictionaries. No one else should directly access DataDictionary table. 

Code looking for data from this table must come through this DAL. such as DataDictionaryTableBAL.geteBasicDataDictionary()

Again, for demo intentions, this is a very basic DAL that in an advanced architecture would itself not use GlideRecords, instead it would delegate the job to a BaseDAL.

var DataDictionaryTableDAL = {

    table: 'table_name_goes_here',

    getBasicDataDictionary: function (key) {

        var dictionary = new GlideRecord(DataDictionaryTableDAL.table);
        dictionary.addEncodedQuery(DataDictionaryEncodedQuery.BASIC_DICTIONARY + key);

        dictionary.query();

        return dictionary;

    }
};

Final Step: 
Create another Data Dictionary Script Include to host BasicDataDictionaryTable encoded quieries:

Note that Encoded Query Object that can be refactored later with the same strategy Methods 1 through 4 in this article.

var DataDictionaryEncodedQuery = {

    BASIC_DICTIONARY: 'active=true^template.identifier=',

}

Thoughts on Re-Engineering a new table to introduce full scale-ability

Steps (2):

Step 1. Create a new table to host templates with five fields

TableName: DataDictionaryTemplate

fields:
1. name {string} template name
2. template {string - 8K+ data size} JSON describing the template
3. description {string} Notes about this template
4. active {boolean}
5. identifier {string} unique identifier for this key


Step 2. Create supporting table that contains key/value "pairs" with five fields

TableName: DataDictionary

fields:
1. key {string} key identifier in key/value pair
2. value {string} value in a key/value pair
3. Template {reference} DataDictionaryTemplate
4. description {string} notes about this record
5. active {boolean} to retire this record

Table DataDictionaryTemplate will host JSON strings intended to be the format of the template. The format can be of any complexity. 

Table DataDictionary will have a reference to DataDictionaryTemplate. Between the two, all of the information required to fill the values of any imaginable Object Literal/Data Dictionary, will be available. 

It would then be up to the a cleverly refactored BasicDataDictionaryFactory to fill the values of the Template from the GlideRecord result.

Summary:

Loosely couple architectures should be the goal of every integration, or customization. Removing, adding, scaling, whether upwards or sideways, should be a trivial task. By learning various techniques available to programmers, ServiceNow can also be turned into a scalable platform. 

The ideas above are tried and proven in the field of software engineering. Finding how to apply them to ServiceNow is key for decreased maintenance and refactoring, time to delivery, and more importantly cost. By cleanly separating concerns, as a system grows, so will the libraries that will help others spend less time rebuilding already built code, while increasing their efficiency.

It should be the goal of every quality implementation to use patterns to prevent typical inadequacies.    

Imagine that the use comes to change the format of the template. Without concern, the DataDictionaryTemplate changes and all that's required is a piece of code (adapter) sitting between BasicDataDictionaryFactory and BasicDataDictionaryBAL (the sole object that uses it) to return the expected format as part of the new Data Dictionary for backwards compatibility. Very minimal places to change code, all else will be new functionality...