This year, we surveyed our readers to learn what you would like to see more of in our blog posts.  One of the most common requests was for us to write more about Python scripting.  You asked, we’re listening!

For those of you interested in learning more about how Python can be used to script repeatable procedures for GIS users, read on!

In the interest of full disclosure, I am not a programmer.  The last time I spent more than half of my time writing code, I was sitting in front of a SPARCstation 20 writing Arc Macro Language (AML) scripts for automated mapping and data processing.

I was running my scripts against Arc/INFO 6.0. All of the sudden, out of nowhere, the ArcGIS Geoprocessing Framework and his object-oriented buddies clobbered me over the head with a Windows XP desktop machine.

You might laugh, but those things were big and quite heavy.  I awoke from my daze and immediately realized that I was destined to live out the rest of my days as a project manager, product manager, and consultant.  I had been overtaken by people that called themselves “developers” and “software engineers”.

Fast forward to the present day…

A few months ago, we were approached by one of our clients who needed to make a large number of data model changes to the geodatabases that they use to manage their electric and gas transmission and distribution assets.

The organization is quite large, and they have hundreds of users that either interact directly with the GIS or use mapping and analysis products that are generated using the GIS.  They have multiple business units in multiple territories, and do their best to maintain consistency in their business processes.

They have fully embraced GIS and have brought it into the core of their business activities.  When they make changes to their GIS and other integrated systems, the changes are applied in the develpment, testing, and production environments.

Once a change has been made for one of the business units or territories, it is then applied to the others.  For this organization, change management is no trivial pursuit.

The list of changes to the data model was extensive, and it was to be applied to multiple versioned Oracle geodatabases with geometric networks.  It was imperative that the changes be applied in a perfectly consistent manner.

The changes would be applied to the development environment, and a round of testing and verification would follow.  Problems would be reported to us, the problems would be fixed, and then the changes would be applied to the test environment.

Only after everything was tested and verified would the changes be applied to the production environment.  This was a perfect use case for a scripted solution, and it was decided that Python (specifically ArcPy) would be used to make the data model changes.  I was selected to write the scripts.

The types of changes that had been requested fell into these categories: adding domains, adding new codes to domains, adding object classes, adding fields, parsing text into new numeric fields, removing fields, setting subtype fields, adding subtype codes, setting default subtypes, assigning domains to fields by subtype, assigning defaults to fields, adding relationship classes, and resetting privileges to datasets and object classes.

If you are familiar with the ArcToolbox, you know that there are tools that will do most of these changes.  Almost every tool available in the ArcToolbox is also available in the ArcPy site package of Python, and they take the same parameters.

The ArcToolbox tools that can be deployed using ArcPy are perfect for making the data model changes themselves, but the text parsing (in a versioned geodatabase), is much more easily handled with SQL.

Since our client uses Oracle to store their data and the data model changes were to be deployed using scripts, we decided that the SQL statements should be formatted for SQL Plus.  The SQL updates would be applied to both the base table and the associated “a” table for each feature class being updated.

A master batch file was written for the purpose of setting variables for the SDE connections, the Oracle connection strings, and the locations of the scripts and the log files, as well as for executing the Python scripts and SQL Plus scripts in the proper order.

Previous posts on this site give instructions on how you can use configuration files, logging, and other nifty Python gadgets to improve the way your scripts perform.

The Python Best Practices Part 1 and Part 2 are particularly helpful. So, rather than presenting you another article on how to write Python scripts, I just want to present you some of the common tools that I used for this project.  All of them come from the group named Data Management Tools in ArcToolbox, which makes sense since the project’s goals all involved data model changes.

I will give examples of the tools in the order that makes sense, when one is making changes to a data model.

First, here is the syntax and an example of the Create Domain tool:

• CreateDomain_management (in_workspace, domain_name, domain_description, field_type, {domain_type}, {split_policy}, {merge_policy})
• arcpy.CreateDomain_management(connectionFile, “Anode_Manufacturer_G”, “Anode_Manufacturer_G”, “TEXT”, “CODED”, “DUPLICATE”, “DEFAULT”)

Next, here is the syntax and an example of the  Add Coded Value to Domain tool:

• AddCodedValueToDomain_management (in_workspace, domain_name, code, code_description)
• arcpy.AddCodedValueToDomain_management(connectionFile, “Anode_Manufacturer_G”, “CP”, “Cathodic Protection”)

• SetValueForRangeDomain_management (in_workspace, domain_name, min_value, max_value)
• arcpy.SetValueForRangeDomain_management(connectionFile, “Pressure_Rating_G”, 1, 1500)

Here is the syntax and an example of the Create Table tool:

• CreateTable_management (out_path, out_name, {template}, {config_keyword})
• arcpy.CreateTable_management(connectionFile, “CPAnode”, “”, “”)

Here is the syntax and an example of the Add Field tool:

• AddField_management (in_table, field_name, field_type, {field_precision}, {field_scale}, {field_length}, {field_alias}, {field_is_nullable}, {field_is_required}, {field_domain})
• arcpy.AddField_management(workingDataset + “CPAnodeBed”, “MANUFACTURER”, “TEXT”, “”, “”, “25”, “Manufacturer”, “NULLABLE”, “NON_REQUIRED”, “Anode_Manufacturer_G”)

Here is the syntax and an example of the Delete Field tool:

• DeleteField_management (in_table, drop_field)
• arcpy.DeleteField_management(workingDataset + “GasMain”, “PIPETESTPRESSURE”)

Here is the syntax and an example of the Set Subtype Field tool:

• SetSubtypeField_management (in_table, field)
• arcpy.SetSubtypeField_management(workingDataset + “Riser”, “SubTypeCD”)

Here is the syntax and an example of the Add Subtype tool:

• AddSubtype_management (in_table, subtype_code, subtype_description)
• arcpy.AddSubtype_management(workingDataset + “Riser”, “1”, “Steel”)

Here is the syntax and an example of the Set Default Subtype tool:

• SetDefaultSubtype_management (in_table, subtype_code)
• arcpy.SetDefaultSubtype_management(workingDataset + “Riser”, “4”)

Here is the sytax and an example of the Assign Domain to Field tool:

• AssignDomainToField_management (in_table, field_name, domain_name, {subtype_code})
• arcpy.AssignDomainToField_management(“GIS.GasMain”, “PIPETESTMEDIUM”, “Pipe_Test_Medium_G”, “1: Steel”)

Here is the syntax and an example of the Assign Default to Field tool:

• AssignDefaultToField_management (in_table, field_name, default_value, {subtype_code})
• arcpy.AssignDefaultToField_management(“GIS.CPAnode”, “SYMBOLROTATION”, “0”)

Here is the syntax and an example of the Create Relationship Class tool:

• CreateRelationshipClass_management (origin_table, destination_table, out_relationship_class, relationship_type, forward_label, backward_label, message_direction, cardinality, attributed, origin_primary_key, origin_foreign_key, {destination_primary_key}, {destination_foreign_key})
• arcpy.CreateRelationshipClass_management(originTable, destinationTable, outRelClass, “SIMPLE”, “CP Anode”, “Gas Main”, “NONE”, “ONE_TO_MANY”, “NONE”, “OBJECTID”, “GASMAINOID”)

Lastly, here is the syntax and an example of the Change Privileges tool:

• ChangePrivileges_management (in_dataset, user, {View}, {Edit})
• arcpy.ChangePrivileges_management(connectionFile + “\\” + arcTable, “GASEDIT”, “GRANT”, “GRANT”)

If you are looking for more information on Python scripting, there are other posts on the SSP Innovations website that you may be interested in.  Simply search for “Python”, or click on the Python hyperlink under the Blog Categories header to get a listing of previous articles that have been written on the subject.