Ever felt confused with all the database options for pipeline GIS management? What’s the difference between them all? If I operate regulated and non-regulated pipe, can I store both my distribution and transmission features in the same schema and network?
I run into these questions on a weekly basis and even though the industry has been utilizing GIS for transmission pipeline management for years, questions still exist around data storage and best practices. Managing the entire lifecycle of a pipeline asset in GIS is hard enough, but it gets even more complicated when we look at centralized data storage options and corresponding system architectures. As it stands today, there are currently four implementation choices in the transmission pipeline market and a potential 5th on the way (PODS ALRP/Next Generation?).
If you were implementing a system today, the four current options are the Pipeline Open Data Standard (PODS), PODS Spatial, ArcGIS Pipeline Data Model (APDM) and Esri's Utility Pipeline Data Model (UPDM). Hopefully this post can provide some insight into your organization's direction or simply provide some differences between these options.
If you are handling everything internally with out-of-the-box Esri tools…go with a geodatabase implementation and utilize Esri's editing and versioning capabilities. Otherwise, I’ve always recommended to select one of these database implementations based on the desired software tools and your operational needs. Your GIS business needs, software functionality and implementation practices should drive the model decisions, not the other way around. Choose the best suite of applications for your regulatory reporting or data needs and simply work with the vendor’s standard model of choice.
Each vendor in the space utilizes best practices around data management and has developed software against industry standard databases that all provide schemas to store your transmission pipeline data. Since all of these database options are open, you can always extend the schema to accommodate new features, relationship classes or company specific attribution.
When purely looking at the data models, the decision of which model to deploy can be difficult. While comparing the PODS relational versus geodatabase implementations, there is actually a big difference. This difference lies in the implementation patterns, core feature structures and how data is edited. Even in this comparison, the most important considerations are often less about the model/schema and more about how the data is being utilized and the software sitting on top of it. Sometimes, existing IT guidelines and system architectures dictate how the database should be deployed.
If IT requires a geodatabase implementation and/or versioning, then PODS Spatial, APDM or UPDM should be selected. If there are no certain IT deployment requirements, the PODS Relational implementation is another option and Esri or third party applications can be used to synchronize the relational records with its spatial representation. If at all possible, fit the desired tools and consumption of data to the model…don’t force fit the model to the tools.
Again, all of the described models are considered open and can be extended based on operator needs. There are core features and datasets in each that must remain, but every operator extends these models to accommodate company specific attributes or in order to integrate the database with external applications and/or systems. Let's now take a deeper dive into each option...
The Pipeline Open Data Standard (PODS) provides a relational database schema pipeline operators can use to store assets, regulatory and analytical data for the transmission system. PODS Relational manages this data as linearly-referenced, tabular records that are synchronized geospatially and visualized in the Esri GIS platform.
The PODS relational implementation inherently stores data in the RDBMS environment with a route and location reference. The record will store the unique attributes for the point or linear event, what route it resides on and the location. The location can be stored based on the linearly referenced location on the pipeline (station/measure) or by x/y coordinates.
Spatial representations are synchronized by Esri or third party software tools specifically designed for transmission data management. The spatial updates will either create the feature at an X/Y location and derive the measure or the relational event is overlaid on the linearly referenced centerline (think display route events) based on a route ID and measure or stationing.
Edits against a relational database like PODS can often be more efficient, but it is typically a wash once the time needed to update the spatial features is accounted for. Typically, users can immediately update the spatial representation, or if there are a large number of edits, schedule batch updates. Scheduled updates allow the user to bulk load large datasets or make a number of edits before kicking off the more intensive process related to spatial synchronization.
The diagram below shows the components of a PODS Relational implementation for transmission and distribution assets. Note that the distribution assets and geometric network are managed in a separate database schema. Updates and edits from both models can ultimately be shared or viewed in any native Esri environment. If you are purely a transmission pipeline operator, your configuration would ignore the distribution components on the far right.
Three other database options can be deployed utilizing Esri geodatabase implementations. These are PODS Spatial, APDM and UPDM. For these deployments, the tabular and spatial data are managed by one system where a native Esri database stores the GIS features and it sits on top of an RDBMS, such as Oracle or SQL Server. Since a geodatabase is being used, versioning can be utilized and out-of-the-box Esri tools can be used for version management, approving or reviewing edits and ultimately posting to production (SDE.Default).
The PODS Spatial and APDM models are designed for implementation as an object-relational Esri geodatabase. Both these models have the same core classes and manage them the same way. These core classes define the model centerline features, stationing attributes, and supporting model elements.
Edits can be performed directly against the features and will instantly be available within the corresponding feature to view after finalizing a create, update or retire task. The differences between these two geodatabase options comes down to the schemas and continual support. APDM inherently is a template that is extended based on software and operational needs and PODS Spatial contains a robust pipeline schema with the same features, attribution and data storage that is found in PODS relational.
Lastly, support for APDM would need to come through software/service providers or Esri services. There will not be updates to the model from Esri moving forward since the focus is on UPDM (described below). PODS Spatial, and PODS relational, are still fully supported by the PODS technical committee and we can expect updated versions to be released moving forward. As mentioned earlier, a new ‘Next Generation’ geodatabase PODS model is also being discussed that will define a spatial standard for PODS and improve on some previous issues with PODS interoperability.
The diagram below shows the components of a PODS Spatial or APDM geodatabase implementation for transmission and distribution assets. Note that the distribution assets and geometric network are managed in a separate database schema. Updates and edits from both models can ultimately be shared or viewed in any native Esri environment. If you are purely a transmission pipeline operator, your configuration would ignore the distribution components on the far right.
Esri's Utility and Pipeline Data Model (UPDM) is a geodatabase schema template initially designed for operators of gas pipe networks that have both distribution and transmission assets. UPDM is a moderately normalized data model that represents each physical component of a gas pipe network. UPDM evolved due to combined utilities needing one data model to manage both distribution and transmission data.
That said, the model can be used to only store transmission assets and therefore utilize tools being developed by Esri. The ArcGIS Pipeline Referencing toolset, formerly ALRP, is specifically being developed to manage linearly referenced transmission assets stored in UPDM. This is a big step in the industry since this tool will allow operators to manage these assets like they have in the past, but within a combined data model and also have these edits persist in the geometric network (which now contains both transmission and distribution pipe).
The goal here is to remove dual edits across two database implementations and presumably, two editing application suites. Other benefits include one system of record, only having to integrate external systems to one database and the ability to analyze one managed network for traceability and outage management.
Moving forward, Esri will also support distribution network edits utilizing a facility network management tool, all working against UPDM. Since this is a geodatabase implementation, all events in this model are stored and edited as features and a connectivity/network model is used to maintain the physical assets in the gas system.
Like previously described models, core linear referencing features and tables are used to maintain calibration points, series, route/system hierarchy and associated on-line events. Typical implementations would only store one linear reference mode, but values can be translated to either stationing or measure (based on the primary managed value and corresponding station series).
The diagram below shows the components of a UPDM geodatabase implementation for transmission and distribution assets. As you can see, we now have one database that stores all assets and the editing tools work directly against this one model.
I hope this overview of database options for managing transmission assets was helpful. At a minimum, its enough information to make you dangerous! As your organization continues to explore new workflows, reporting, applications and potential migrations to UPDM, feel free to reach out and hear how SSP can help you through the process.
It may seem insurmountable at times, but there are new tools on the horizon and we can help to optimize existing workflows or help transition you to the next phase of your pipeline GIS management.
Clarke Wiley – Director of Pipeline; email@example.com