You are here

The Big State 0 Theory

Up until now State 0 has been nothing but a theory, or simply a subject that has been avoided most of the time due to its complexity. SSP is very happy to announce that we have successfully proven the State 0 theory, which was explained by Skye Perry on “The Elusive State 0 – Can We Get There?” article.

We would like to thank Duane Holt – Director of GIS (and general rock star) at Intermountain Rural Electric Association (IREA) who believed in our solution when it was just a theory, and Schneider Electric for their constant support throughout this process.

What drove us to get into this journey?

A simple request from a client can make a huge difference in creating innovative solutions. Don’t ever be afraid to request something, even if it sounds crazy. There is always a chance you will find crazy-idea lovers like us who are willing to give it a shot!

The request from IREA was to re-project their data from NAD 1927 to NAD 1983 without losing their versions. As you can see it's a very simple request, but the complexity was challenging to say the least.

A combination of the SSP Nightly Batch Suite, the SSP All Edits Tool, several white-board sessions, and a creative vision made it all possible.

How does the process work?

  1. We use a new configurable SSP Nightly Batch Suite (NBS) application to extract all versions into the SSP All Edits tables and feature classes. These are effectively the same tables and feature classes used by the SSP All Edits Report when extracting an individual version for re-visualizing it at a later point in time.
  2. Delete all versions and compress the database. Your versioning lineage should look something like this:

    State Zero
  3. If required, drop all relationships, all geometric networks, and un-version the GIS Data.
  4. Do whatever you need to do once you are at State 0 (In IREA’s case we had to re-project their data from NAD 1927 to NAD 1983).
  5. Use a second configurable SSP NBS application to recreate all versions extracted in step 1. This application creates the versions using the exact same name and then replays all of the edits including inserts, updates, and deletes.
  6. If you are a Schneider Electric Designer™ shop, then there is one additional SSP NBS application that is used to re-synchronize each design’s XML which includes references to the versioned object ids.
  7. Celebrate!

The following diagram will help you understand the flow a little better:

AEZero Workflow

After recreating the versions and verifying data, IREA’s versioning lineage looked like this:

Versioning Lineage

Close up shot:

Versioning Lineage Close Up

It truly is a nice thing to have your versioning lineage as displayed above; normally you can’t even find where SDE.Default is within this view.

Now that you have a general idea of how the whole process works, there are probably a few questions popping up in your head: How long did it take to run? Was it a few hours, or days? How long was production shut down? etc., etc.

Production was shut down only for a single day (Friday) and the process took us 2 ½ days to get the whole project completed - including the re-projection which was time-consuming as well:

  • Thursday afternoon – Preparation time. Installing code, updating config files and kicking off the first SSP NBS  application “WriteVersionsToDB”.
  • Friday – Deleting versions, relationships, dropping network, un-versioning GIS Data, re-projecting Electric and Land Dataset, rebuilding the database (recreate relationships, network, version GIS Data) and kicking off the second SSP NBS application “CreateVersionsFromDB”.
  • Saturday – Validating all versions were recreated successfully, running the final SSP NBS application “UpdateDesignXML” to update each design's XML with the new object ids and full system testing.

The following statistics were gathered from running the test process in our office with a total 299 versions and from running the production process at IREA’s site with a total 242 versions.

SSP's Office

 SSPs Stats

IREA's Office

 IREA Stats

The difference in timing between our office and IREA are mainly due to system infrastructure (i.e. memory, processors, etc.) but I hope it gives you an idea as to what to expect if/when you consider getting to State 0 without losing any of your versions. You should be able to use your version- and/or edit-count from your geodatabase to come up with some approximate processing times within your environment.

The processing time at IREA took about 12 hours and 29 minutes (give or take) for 242 versions, but what if you have thousands of versions? Would you need to shut down production for more than couple of days? The answer is no - our solution is scalable. We recently made some changes so that you can process a range of versions within a multi-threaded implementation of the SSP NBS.

For example:

Let’s say you have 3,000 versions. We would set up 3 SSP NBS processes to run concurrently, each processing 1000 versions at a time.

  • NBS1 will process versions 1-1,000.
  • NBS2 will process versions 1,001-2,000.
  • NBS3 will process versions 2,001-3,000.

This will decrease the time it takes to process the entire geodatabase by splitting up the workload. The processing time may go up slightly per each edit/version because there is only a single geodatabase supporting the multiple processes, but this approach will tremendously reduce the overall down time within your production environment.

Additional NBS processes can be added as long as you have the hardware and database resources to support running the processes concurrently (i.e., additional application servers and a well-performing geodatabase).

What did this project accomplish for IREA, and how do they feel about the project?

We asked IREA’s Duane Holt for some feedback for this article and here is what he sent us:

The AEZERO project provided IREA the ability to reach State 0 and re-project the geodatabase to a NAD83 coordinate system.  Without going through this process, the antiquated NAD27 geodatabase would be restricted by “on-the-fly” projection limitations that are present in Web Mapping Services. 

Completing the AEZERO project allows IREA to progress into the future of GIS and deploy more web GIS technologies such as ArcGIS Online.  Successful completion of the project instills confidence that the same process can be repeated if necessary to reach State 0 again.

The AEZERO project employed a process that utilized two of SSP’s out-of-the-box products with a little customization and specific configuration.  The thorough development and testing completed by SSP and IREA resulted in a succinct step-by-step process.   Much like following a recipe to bake a cake, the AEZERO project at IREA stepped through exactly as planned in shorter time than expected. 

Working side by side with SSP developer Dennise Ramirez, IREA’s Duane Holt was able to work through the project and at the same time gain the knowledge and confidence that the two companies had created a scalable, reusable method to reach State 0 in a versioned database without losing the versions; a far-reaching idea for any GIS with a versioned SDE geodatabase.  I am continually satisfied and surprised at how easy project rollouts have been working with SSP Innovations.

Conclusion

There you have it, reaching State 0 is no longer just a theory; the issue is now a thing of the past. You can get to State 0 without losing your versions; just leave the magic to us.

If you are wondering whether or not we hosted the celebration party after the project was completed… we sure did!