SnowMirror vs ODBC driver

Casey12 · ‎12-10-2018

My company is experiencing performance issues with the ServiceNow ODBC driver. As I understand it, this driver is not a real driver -- it is merely a wrapper around the out-of-the-box (OOTB) ServiceNow SOAP API, which is the cause of its poor throughput. It was recommended to me to look into SnowMirror to resolve our throughput problem. As I understand it, SnowMirror also uses the OOTB ServiceNow SOAP API. But, somehow, it has better performance than the ODBC driver (or so it's claimed). Is this true? If so, how can this be, given that they both use the OOTB ServiceNow SOAP API (and are, therefore, effectively the same thing)?

pavelmuller · ‎12-12-2018

Hi,

Please let me explain why SnowMirror is able to achieve the highest possible data throughput. We've been developing SnowMirror for more than 6 years and we have fine-tuned the replication algorithms on hundreds of production ServiceNow instances.

You are right that there is no magic way how to access an instance. Just the SOAP or REST APIs. But the key thing is to use the API and the whole instance the most efficient way.

Here are the most important things making SnowMirror faster than other solutions:

Incremental load - SnowMirror downloads just the deltas; only the new and updated records and it remove the deletes. This enables us to work even with the largest tables.
Smart pagination. SnowMirror uses a special algorithm for handling batches. It downloads a batch of keys first and then it downloads smaller chunks of actual data.
Concurrency. SnowMirror is able to download a single table in multiple threads which is increasing performance significantly.
Instance utilization - and this is the most important point for bigger instances. SnowMirror is able to maintain sessions with individual ServiceNow nodes which boosts performance and data throughput dramatically.

Please read more about the node identification and load concurrency in our article:

http://www.snow-mirror.com/snowmirror-4-performance-boost/

I know that SnowMirror is not the cheapest solution but it is a proven, enterprise-ready solution and we guarantee that it stays compatible with new ServiceNow releases. Maintenance costs of custom solutions is a very common underestimation. It takes a lot of effort to keep such a solution up to date.

Kind Regards,
Pavel Muller
SnowMirror Consultant

View solution in original post

SanjivMeher · ‎12-10-2018

The advantage you have with SnowMirror

Your reporting will be faster. The reason it is faster, because it pulls data in advance.

It also pulls the reference fields display value, which is an advantage, which is not the same with ODBC Driver. In ODBC Driver, you will have to create database views to get the reference's field display values.

You can connect any reporting tool such as Tableau to generate report out of it.

Disadvantage

You have to pay for it. They have Standard and Enterprise package. I think it depends on the size of the data you choose standard or enterprise.

The reporting wont be real time. Because a scheduler runs every day and will update the snow mirror database on a daily basis. So you will have the data only till yesterday.

You will need another database server to store these data. So additional maintenance of a server.

Please mark this response as correct or helpful if it assisted you with your question.

Slawek_Radziewi · ‎12-10-2018

We used to use snowmirror but it is too expensive.

We switched to solution that we've added scheduled exports of needed tables which we send to local server.

These exports are imported to our local server using MIDServer and than used for reporting.

Ryan153 · ‎12-11-2018

We also had similar issues and tackled in two ways.

1. We use SSIS (with the ODBC connection) for the data extraction and were experiencing a lot of timeouts and jobs that ran for a very long time. The first thing we did was to break each table extract into manageable chunks based on the width of the table. For example if we knew we had 30000 metric_instance records we might configure the ETL to pull 6 chunks of 5000 records each. For the Incident table we pulled only 2000 at a time because the table has a lot more records. We were running daily extracts for about 90+ SN tables and this made a big difference in our ability to collect records.

2. We began to use Perspectium Data Integrations which was a game changer, and is how we are handling all SN data extracts now. With Perspectium we are able to move records into a SQL Server DB very quickly and then use that DB as the source for SSIS ETLS which loads into the DW. End to end we can get an entire day's worth of records for 90+ tables from SN into the DW in 45 mins. In some cases we are even using real time extracts for some specialized reporting. I would estimate this model cut our time in half, provided additional capabilities for data access and cut down our on-call issues drastically.

We had also done a POC with CrossFuze but they were unable to successfully move the SN data we needed into our environment. That was several years ago so I would hope they have things figured out by now.

Hope this helps

Casey12 · ‎12-11-2018

As far as I know, all of these third-party extract tools (e.g., Perspectium, SnowMirror) all use the ServiceNow SOAP API. But, so does the ODBC "driver". So, how is it that there are performance boosts with these third-party extract / replication tools relative to the ODBC "driver"? I understand that ETL with the ODBC "driver" can be tuned, but I'd like to know if there is any inherent performance boost with these third-party tools, and, if so, how they achieve those boosts (given that they're all SOAP API-based).

ServiceNow Community servicenow community

SnowMirror vs ODBC driver