You are here

Fixing Your Data Sources With Python

During a recent customer upgrade from ESRI 9.3 to 10SP3, we uncovered a last minute "gotcha" regarding over 1,000 map files floating around several share drives. Since the new production environment was now using a brand new Sql Server installation, with a different server name, all of the data sources that were attached to the map files would now be invalid.

We identified three scenarios to handle this situation:

First, whenever a user opens the map, they can manually repoint the data source. This seems reasonable at first, but requires the user to perform an additional task which they weren’t expecting to do.

The second scenario is similar to the first, but to hire a temporary worker to open all the maps and fix the data source. Very tedious work, but what the heck, that’s why they are a temp!

The third scenario involves automating the opening and fixing of the map file through a Python script. This should be relatively easy to do, after all, as ESRI has some pretty good information in their Resource Center here

There were a few small challenges to get a script to process a bunch of map files located in various shared directories. The example scripts in the Resource Center all involved changing a single file. With a little Python magic, we were able to process a whole directory tree at once by using the os.walk function. To make sure we only process map files, we make sure only look at files that have an ".mxd" extension. When working with the layers, we noticed that not all layers have a "workspacePath", so a check needs to be made before trying to manipulate that using the arcpy "supports" function.

Also, you will need to make sure that there is a new data source file on the disk. Just setting a new source path does not work and will throw an exception. In reality, this new ".sde" file would live on a shared directory, so all of the map files could reference it. For this example, we will just use a local one.

Here is the script!

# Import system modules
import arcpy, os

#find all the MXD's in the directory tree
for root, subFolders, files in os.walk(r"C:\Temp\Maps"):
    for filename in files:
        fullpath = os.path.join(root, filename)
        basename, extension = os.path.splitext(fullpath)
        if extension.lower() == ".mxd":
            print "------------------------------"
            print filename
            #open the map document
            MXD = arcpy.mapping.MapDocument(fullpath)
            #get all the layers
            for lyr in arcpy.mapping.ListLayers(MXD):
                #get the source from the layer
                if lyr.supports("workspacePath"):
                    source = lyr.workspacePath
                    print "%s -> %s" % (lyr, source)
                    basename, extension = os.path.splitext(source)
                    if extension.lower() == ".sde":
                        #This is the NEW SOURCE that you want to point to
                        datapath = r"C:\temp\osa@dc@newSqlServer.sde"
                        #replace the old path wih the new
                        lyr.findAndReplaceWorkspacePath(source, datapath)
            #save your changes
            MXD.save()
            del MXD

 

The final thing to do is have some good exception handling. This sample script provided does not account for any errors that may occur. We encountered a corrupted map file and the corresponding exception that was thrown was not useful at all:

Fixing your data sources - Python exception

While there are several resources for performing this task available online, we hope that this little consolidated version will get you on your way to updating your data sources en masse.

Author Information

  • Jeff Mertz

    Jeff Mertz

    Jeff Mertz is the Director of Technology and a Principal Consultant for SSP Innovations and leads the company in advancing new technologies within our solutions. He has been working with GIS for more than half of his 25+ years of experience in software development. Jeff has focused on implementing custom GIS, WMS, and OMS solutions for SSP with a specialty in systems integration and web technologies, including ArcGIS Server and ArcGIS Online.

    See all items created by this author >

    Connect with me on:

Comments

Thanks, so much, this is just what we needed to accomplish.

This code does not work if you have grouped layers in your MXD. Please advise.

Thank you! Your script help me very much.

Add new comment