CMDB Identification and Reconciliation

[richardbrounste]

The Configuration Management Database (CMDB)

The CMDB is a powerful single system of record of configuration items (CIs).   It is not just a database with information on all the CIs such as hardware, software, services, and more.   It provides a critical resource for essential IT Processes such as:

• IT Service Management
• Operations Management
• Asset Management
• Configuration Management

CMDB Problems with Inaccurate Data

The most efficient way to populate the CMDB is with an Automated Discovery Mechanism.     Yet, many CMDBs suffer from having inaccurate data.   If your CMDB data is inaccurate, the reports and processes which depend on the CMDB can fail and your organization will be at risk.

Two common inaccuracies that exist in a CMDB are:

• Duplicate CIs in the CMDB where one item exists in the data center
• Compression: A single CI in the CMDB that represents multiple items in the data center

Both of these problems occur when the attributes used to identify configuration items do not uniquely represent the identity of the CI. Consider the following scenarios.   The Data Center has two Linux Servers.   One Linux Server is running a single instance of a Tomcat Application Server. The other Linux Server is running multiple instances of Tomcat.   When written to the CMDB, the Tomcat Servers are being identified by their class and the name of their primary configuration file.   On the machine where 2 Tomcat servers exist, the name of the config files are identical so even though there are two CIs, only 1 entry is made in the CMDB for that one Tomcat Application Server on that particular Linux Server.  This is Compression.

Or consider the scenario where the version number is used to uniquely identify the Tomcat Application Server.   The version number can be a unique identifier as long as it stays the same. But once Tomcat is upgraded to a new version, it is then recognized as a new CI and a duplicate entry is made in the CMDB even though only one instance of Tomcat exists in the data center. These are the kinds of CI identification challenges that CMDB managers must deal with.

Choosing the Right Identification Methods

Just as your drivers license or passport is a document that identifies you from other people, configuration items need their own unique identifiers. Identifiers are made up of attributes that are part of each configuration item.   For an identifier to be effective, it must satisfy two criteria:

1. The identifier must be unique to that configuration item
2. The identifier must not change for the life of the configuration item.

Let's consider a typical identifier for a virtual or physical computer. Networks identify computers by their IP address.   IP addresses are unique.   In fact, if a computer joins a network with the same IP address as another computer, a warning is usually issued and that computer is not allowed to join the network. You might think that an IP address is the optimum choice as a unique identifier.   Yet it is not because IP addresses can easily change as a configuration setting in any computer's TCP/IP settings so they are not a reliable unique identifier in the CMDB.   The MAC (Media Access Control) address could be a better choice since MAC addresses are uniquely assigned to network cards and are burned into the hardware.   Yet, with virtual hardware, MAC addresses can now be assigned at random and can be duplicated throughout the network.

Since ServiceNow Geneva release, a facility exists to determine the attributes which are used to uniquely identify any CI type for hardware and software. Serial numbers make a much better choice as they are generated as unique ids for physical and virtual hardware and tend to not change for the life of a configuration item.   That is why ServiceNow Discovery uses a serial number as the default mechanism to identify hardware items.   However, it is possible that a serial number is not available for a piece of hardware. So, a fallback mechanism exists where the machine name or a combination of the IP/MAC Addresses can be used if the serial number does not exist.

It is important for the configuration management group in any organization to be familiar with the hardware and software in order to best determine which attributes should best be used to uniquely identify these components.   These are configured by class in a simple user interface the "CI Identifiers" section.

CI Identifier Rules for Hardware.   Sometimes the Serial Number may be blank so different identifiers are allowed

Identification Methods for Software Applications

Software applications don't necessarily have a unique serial number that can be applied as their identifier.   They can best be identified by the individual process that they run. However, since many applications allow for multiple instances of themselves to be run on a single machine, there may exists multiple different processes.   So we also want to identify an application by the "installation directory", which should be different for each instance.   Multiple instances of the same software would have to be installed into different directories so this makes an effective identification method.

CI Identifier Rule for Applications

Configuration Item Reconciliation

Just as an intelligent human is educated from multiple sources of knowledge and wisdom, your organization's CMDB may not completely serve your use cases by being populated from just one source. When multiple use cases are served by a CMDB, multiple discovery methods are often needed.   Consider that ServiceNow Discovery is designed to gather information on all desktops, servers and network devices in your data center. Yet an asset discovery tool that recognizes licensable software programs from installed utilities may be better suited to identify the inventory of software on servers and desktops.   So, by having both discovery mechanisms running, the CMDB is more accurate.

This becomes a problem when different sources try to write information to the same records.   This has to be managed or changes to the CIs that are tracked in the CMDB history tables will negatively affect the change management process.

In order to detect a change that was made in the data center, ServiceNow compares discovered values exactly against the current value in the CMDB.   Data that has the same value may appear to be different.   For example, one discovery tool may identify the memory size as 2048 mb whereas another tool may identify the memory size as 2 gb. They are the same value but will be seen differently. The ServiceNow CMDB will record the values as different and will note that a change was made to that CI when in the data center no changes were actually made.

In order to prevent false change reports, we must prevent different automated discovery tools from overwriting the same configuration item fields in the CMDB. This means we must have reconciliation rules based on the different sources assigned to different configuration items.   For this to work, a single source of truth must be identified not just for each configuration item but for each attribute in each configuration item.

Example of multiple sources of automated discovery updating the ServiceNow CMDB

Multiple sources of automated discovery assigned to individual attribute fields

Single Source of Truth or Multiple Sources of Truth

The CMDB is the single source of truth of information about the data center. For it to work effectively as a source of truth, each Configuration Item can only be populated by in the CMDB by a trustable source.   More accurately, each attribute (field) of the CI must have a primary source of truth.   This is a meant to be a source that we trust to populate the right values in the database.

While there should be a single source of truth, there may actually be multiple sources that can update the CI attribute.   If a discovery source is designated as the only source that can create or update CI fields, we are limiting our CMDB maintenance. What if a CI does not exist in the CMDB yet and a source that is not as trusted but is still valued provides the info on the new CI?   We want that update to happen because we don't want to wait until the next scheduled discovery scan of the fully trusted source.   That means we need to allow for multiple trusted discovery sources but need to make sure that some sources have higher precedence than others.

CI Reconciliation in ServiceNow

In ServiceNow, starting with the Geneva release, there exists a simple facility to define CI Reconciliation.   In other words, there exists a facility that allows users to decide which attributes for which CI class a particular discovery Data Source is allowed to update after it has collected such data.

Definition of CI attributes allowed to be updated by ServiceNow Discovery for Windows Servers

Data Source Precedence

As indicated in the above paragraph, we need to set an order of precedence for the discovery data sources.   A simple example is with ServiceNow Discovery and ServiceWatch (Service Mapping) discovery.   Setting the precedence simply under the "Datasource Precedences" for CI types allow you to define the Order in which different discovery sources are allowed to create and update CIs.   What this means is that if a discovery source of a lower precedence tries to insert a CI or update a field of a CI, it will only be allowed if the CI does not exist or, in the case that the CI does exist, it will only update the field of the CI if it is blank.

In this Example, the ServiceNow Discovery Data source has a higher precedence than ServiceWatch (Service Mapping)

Starting in the ServiceNow Helsinki release, changes to configuration items are tracked by individual changes to the attributes.   This is done in the Data Source History (cmdb_datasource_last_update) table.   This is how ServiceNow knows which attribute was updated by which discovery mechanism.

Testing CI Reconciliation

One way to test that the CI Reconciliation is working is to run a script that tries to insert/update a new CI pretending to be one discovery source and then try to overwrite it with a different discovery source.   Below is an example of such a script.   Note the execution of the method call "createOrUpdateCI" that takes in a parameter of the particular discovery source.   The below example script simulates two updates from two different discovery sources.

After running the script, a server is created.   Note the Discovery Source with the last update and the CPU Name only gets updated by ServiceWatch and CPU type is only updated by ServiceNow