Database Syncing – When One Must

February 7, 2018 — Brian Higgins

One of our clients has the interesting predicament of two systems of record for the documentation of electric infrastructure. The legacy system of record is Milsoft’s WindMil. WindMil’s primary focus is quick and accurate modeling of the electric system, and not much of the functionality of Esri’s ArcGIS. SSP’s initial task was to convert a WindMil export into the Electric Multispeak model as an Esri Enterprise geodatabase.

Even though the project’s plan was to migrate to ArcGIS, the client’s plan did not include the abandonment of WindMil data maintenance either. In fact…it was desired be able to digitize/maintain both systems until the GIS is in a trusted state and then be the singular system of record. At that time in the future, exports would then be made from GIS and imported into WindMil.

For the record, it is generally not advisable to digitize in two different systems. This not only duplicates the level of effort, but can lead to data inconsistencies. However, let’s say we want to do it anyway because we have a justifiable reason or just want to be contrary and not follow advise. How do we maintain two different systems and keep them in sync? Here is one way.

Milsoft Export

As a first step, a nightly export from WindMil was scheduled to the Esri File Geodatabase format. This export is titled “Milsoft_Data_NEW.gdb”. A second scheduled task leverages SSP’s Maintenance (formerly Night Batch Suite) to compare the exported Geodatabase to the previous night’s export (“Milsoft_Data_OLD.gdb”). This comparison is looking for deltas (i.e. new and modified features).

Database Mapping

SSP worked with our client to develop a feature class migration scheme from Milsoft to the Esri Electric Multispeak model. The scheme would not only be utilized for the initial Enterprise Geodatabase construction, but also be leveraged in the migration of the deltas. This makes it so that only changes in the mapped fields are tracked by the scheduled task. The nightly scheduled task creates a database version off Default and inserts the code-determined deltas. Once the code successfully runs, the previous night’s export (“Milsoft_Data_OLD.gdb”) is renamed based on the date of the comparison, and stored in an archive folder while the newly exported database (“Milsoft_Data_NEW.gdb”) is renamed to “Milsoft_Data_OLD.gdb”. This allows users to review any exports as well as enables the two scheduled tasks to run every night without any issues.

Manual Enterprise Database Clean-Up

It was determined by the client that it was best to manually interact with the created versions instead of automated reconcile and post. Quite frankly, Milsoft does not care about the integrity of the geometric network. Automation of the task may adversely impact GIS data integrity.

These new/modified features are easily identified through a combination of daily log file review and Editor Tracking field query. Editor tracking is turned on for all feature classes resulting in creation/modified date field population upon applicable events.

Deletes Identification

As stated above, the first efforts addressed the new and modified features. What about deleted features? If a feature was deleted in Milsoft, we want that feature identified in the GIS and potentially deleted as well (if desired).

We did not want to bring over deleted features in a version, because…well…they are deleted features and adding them seemed illogical. This tool was broken down into two components. We first wanted to notify the GIS user that deletes have occurred in the Milsoft export. This initial notification was accomplished as a third scheduled task, once again utilizing NBS. The deletes identifier task looks for features within the Enterprise database that are not contained within the Milsoft export. If, at any point, a feature is found that exists in the Enterprise but not the export, an email notification is sent to certain individuals that have been configured for such alerts.

The next day, the GIS user gets to the office and a notification email is present stating that there are deletes present. That user can then run a secondary SSP-created tool from within ArcMap. It cycles through the feature classes contained within the Table of Contents and identifies “Milsoft” features, based on the mapping that was created. It then does a comparison, looking for any features that exist in the Enterprise database and not in the Milsoft export, and adds those features to a selection set. The user can then use the Esri OOTB tools to identify those features and delete as applicable.

As stated above, it is never good to maintain two different systems. Time will tell if this maintenance procedure is justified, but we feel that it is a good start.

We Wrote the Book

The Indispensible Guide to ArcGIS Online

Download It for Free

Brian Higgins

Solution Architect

What do you think?

Leave a comment, and share your thoughts

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>


This site uses Akismet to reduce spam. Learn how your comment data is processed.