Using JupyterLab for Database Schema Discovery

November 7, 2022 — Aaron Leese

One of my favorite things about working at SSP is that I get to help people use data to make informed decisions about their operations. Python has been my go-to language for doing almost all my data related tasks for the past 12 years. Let me tell you, it’s come a long way! The things we can do today in a few lines of code are nothing short of amazing in my opinion. Building a very useful app to answer a specific question inside a notebook can be done in min or hours. Once I got my head wrapped around using Jupyter/JupyterLab, and the pandas, numpy, and ipywidgets libraries, I became many orders of magnitude more effective at almost every aspect of my job, and you might too! What I’ll be showing today is a very simple example of a notebook that will be easy to understand for anyone with a bit of python under their belt.

What are we building today?

The notebook we’ll be building has one purpose, to be able to quickly show field properties of a specific column of geodatabase table, including the allowable coded value domain values if the field is controlled by a domain. Is this groundbreaking use of technology JupyterLab? Absolutely not, but if you wanted to go review field types and allowable domain values, this would be quite useful. As it will display all the information in a simple interface without any clutter, and it will display the information very quickly compared to navigating around within ArcGIS Pro or ArcCatalog.

The Notebook (no… not the movie… the interactive python notebook!)

I like using JupyterLab over Jupyter notebook, they’re similar, but JupyterLab has more bells and whistles. The fine folks over at ESRI have put together some documentation about using JupyterLab within the ESRI python ecosystem if you need any help getting that setup. Within JupyterLab I’m going to create a new notebook and then get to coding. If you’re experienced in python scripts but new to python notebooks, the concept of cells of code might be strange, but if you use python to handle data tasks, I’ve got a feeling you will be in love with them soon. Cells separate your code into groups so it can be executed independently of the rest of the code in the notebook. If you’re doing lots of data processing you can run a cell block, examine the results, and if you don’t like the results, and need to tweak something, you can make your adjustment and then rerun the cell (provided you aren’t overwriting the data in your variables anyway). In my first cell I typically have all my import statements (unless I’m making something big, and I want to add sections of markdown (but markdown is out of scope for this blog post!))

In my next cell I’ll declare the variables that I may need to change at some point (I’m using a copy of the ESRI UPDM UN asset package data model for this example)

In the third cell of the notebook, we’re going to make two dictionaries and populate them with information about our tables and our domains so we can quickly access the data. Pretty plain arcpy coding stuff here, nothing fancy.

Finally in the last cell I’ll create some selection widgets, make a function that shows the field properties and domain values, and make an interactive widget, which produces an output widget, then I’ll adjust some UI stuff, then display the UI and we’re done!

OK, OK, I’ll show ya the results

When you start it’s just a table list, with no fields:

But as soon as we click on a table in the selector it shows us the fields for the table and the properties of the first field:

And if we click on a field that is controlled by a coded value domain, it shows us the allowable domain values for that field:

It’s probably not the most spectacular thing you’ve ever seen, but if you were in the mood to easily review the allowed coded values of some fields in some tables, this would be the bee’s knees!

One more thing for the seasoned veterans!

One of the things I’ve found frustrating in the past when making notebooks with a UI full of widgets is that you can accidentally scroll down in a notebook while trying to scroll to the bottom of a selection widget and end up with something like this on your screen

I just recently was trying to adjust some sizes of the widgets on my screen for another project using the dev tools in chrome, and I discovered that there’s a CSS style of “scrollPastEnd” that’s causing that white space:

That white space is great when you’re writing code, but for using the interactive widgets, it just causes heartache. But since we’re doing neat stuff here today using ipywidgets, let’s go overwrite that “block” display with “none” and make it go away!

If you add this code to an existing cell, or a new cell, and run it, that white space will go away

Now if I try to scroll to the bottom of the notebook, I can’t scroll down into whitespace land.

Word of warning, it’ll blast the white space for all the notebooks you have open, even in other tabs too (if you need it back just change the “none” back to “block” and it’ll come back (remember what I said earlier about being able to re-run cells😊)

Wrapping it up

If you’d like to play around with the code I’ve written for this, I’ve dropped the notebook here. I hope it serves you well and gets your creative juices flowing. Using ipywidgets to explore data and database schemas has made me much more effective at the tasks I do on a regular basis, and I like to share what I know so we can all grow and do even cooler stuff in the future. If you liked this post let me know and I’ll try to do more posts like this one and we’ll see where it goes.

We Wrote the Book

Understanding & Implementing the Esri Utility Network

Download It for Free

Solution Engineer

What do you think?

Leave a comment, and share your thoughts

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes:

<a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>


This site uses Akismet to reduce spam. Learn how your comment data is processed.