Fixing Your Data Sources With Python

May 7, 2012 — Jeff Mertz

During a recent customer upgrade from ESRI 9.3 to 10SP3, we uncovered a last minute “gotcha” regarding over 1,000 map files floating around several share drives. Since the new production environment was now using a brand new Sql Server installation, with a different server name, all of the data sources that were attached to the map files would now be invalid.

We identified three scenarios to handle this situation:

First, whenever a user opens the map, they can manually repoint the data source. This seems reasonable at first, but requires the user to perform an additional task which they weren’t expecting to do.

The second scenario is similar to the first, but to hire a temporary worker to open all the maps and fix the data source. Very tedious work, but what the heck, that’s why they are a temp!

The third scenario involves automating the opening and fixing of the map file through a Python script. This should be relatively easy to do, after all, as ESRI has some pretty good information in their Resource Center here

There were a few small challenges to get a script to process a bunch of map files located in various shared directories. The example scripts in the Resource Center all involved changing a single file. With a little Python magic, we were able to process a whole directory tree at once by using the os.walk function. To make sure we only process map files, we make sure only look at files that have an “.mxd” extension. When working with the layers, we noticed that not all layers have a “workspacePath”, so a check needs to be made before trying to manipulate that using the arcpy “supports” function.

Also, you will need to make sure that there is a new data source file on the disk. Just setting a new source path does not work and will throw an exception. In reality, this new “.sde” file would live on a shared directory, so all of the map files could reference it. For this example, we will just use a local one.

Here is the script!

# Import system modules
import arcpy, os

#find all the MXD’s in the directory tree
for root, subFolders, files in os.walk(r”C:\Temp\Maps”):
for filename in files:
fullpath = os.path.join(root, filename)
basename, extension = os.path.splitext(fullpath)
if extension.lower() == “.mxd”:
print “——————————”
print filename
#open the map document
MXD = arcpy.mapping.MapDocument(fullpath)
#get all the layers
for lyr in arcpy.mapping.ListLayers(MXD):
#get the source from the layer
if lyr.supports(“workspacePath”):
source = lyr.workspacePath
print “%s -> %s” % (lyr, source)
basename, extension = os.path.splitext(source)
if extension.lower() == “.sde”:
#This is the NEW SOURCE that you want to point to
datapath = r”C:\temp\osa@dc@newSqlServer.sde”
#replace the old path wih the new
lyr.findAndReplaceWorkspacePath(source, datapath)
#save your changes
del MXD
The final thing to do is have some good exception handling. This sample script provided does not account for any errors that may occur. We encountered a corrupted map file and the corresponding exception that was thrown was not useful at all:

Fixing your data sources - Python exception

While there are several resources for performing this task available online, we hope that this little consolidated version will get you on your way to updating your data sources en masse.

We Wrote the Book

The Indispensible Guide to ArcGIS Online

Download It for Free

Jeff Mertz is the Director of Technology for SSP Innovations and leads the company in advancing new technologies within our solutions. He has been working with GIS for more than half of his 25+ years of experience in software development. Jeff has focused on implementing custom GIS, WMS, and OMS Read more


What do you think?

Leave a comment, and share your thoughts

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>