This is another topic that has garnered a lot of interest from various utilities that currently use either Telvent ArcFM™ or Telvent Designer™. The goal has long been to capture some objective key performance indicators (KPI’s) on the work performed by your editors within the GIS that would allow management to gauge how well they are performing in general and to show trends over time.
If you are using ArcFM™ Session Manager or Designer™ Workflow Manager you are already familiar with working in a “Session” or a “Design”. Sessions and Designs allow users to perform edits within an Esri versioned geodatabase without having to know about how versioning works behind the scenes. They instead focus the user on the business requirements behind the work that has to occur within the geodatabase.
Sessions are used for general data cleanup, as-builting ad-hoc or blanket work orders or any other types of general editing. Designs are used to enter planned work requests into the GIS to establish a design drawing and a cost estimate for the future work.
In both cases, there is a workflow associated with these edit sessions. For the purposes of this article we will focus on sessions going forward, but the concept is the same for both sessions and designs. The following screenshot shows Session Manager with a few sample Sessions. Each of these corresponds to an Esri Version in the geodatabase where the edits are performed:
At the simplest level, a Session will have a workflow similar to this where the gray boxes represent the statuses of the Session as it moves through its lifecycle:
So a new Session is created by an editor, edits are performed and then the Session is submitted to an approval officer who will review the edits and either approve and post the edits or reject the edits back to the editor for correction. Each time any of these transitions occur within Session Manager, the history of the transaction is saved to the database. This transactional data is typically kept with the Session until the Session is posted (completed) and then all of the data is cleaned out of the database.
SSP saw an opportunity to harvest this transactional data to drive several editing KPI’s:
- Type of Work Performed: When a new Session is created the editor designates the Session Type as Electric, Gas, Water, Land, Fiber, etc (a configurable list of types.) We can use this data to statistically track the type of work a user performs over time as compared to their assigned work.
- Average Days Worked per Session: Based on the transactional data we can extract how long each session is being worked by the editor before being submitted to the approval officer. Most utilities have benchmarks set for the turnaround of their sessions (as-built work orders) so this can be a very useful statistic.
- Number of QA Rejections: Each time a Session is rejected back to the editor for corrections, the rejection is logged as a transaction. With this data we can display the number of rejections per Session and more importantly, the number of rejections over a period of time in relation to the number of sessions worked.
- Quality Performance Rating: We can then combine the above statistics into a single rating that combines: the total number of sessions completed, how quickly an editor is completing the work and how many times the work is being rejected. This provides an overall quality rating for the editor who has performed the work.
This is all numeric statistical data, but as with many numbers like this, the value to the organization goes up if it is displayed in a graphical and interactive format. Like many of our tools, we put this into a web control format which can be plugged into most configurable websites, because that maximizes the availability of the data to the organization.
The interface starts with allowing the manager to choose a specific user and the timeframe to extract the KPI’s. This would typically be monthly, quarterly, annually, etc depending on the goals of the analysis:
The system then performs the analysis and renders the data using a combination of text, graphs, and dials. Preset ranges identify the target (green), warning (orange) and poor (red) performance brackets on the charts:
The bottom panel includes all of the raw data on the Sessions that was included in the analysis, which is very useful if further inspection is needed. The detail includes each Session ID, the description of the work performed, the days worked, the number of QA rejections and the full transactional history for the Session. The detail is organized in a collapsible hierarchy to make it easy to review.
Managers will often compare different time periods of work for the same editor to establish trends in the quality of work for that user – ie., whether they’re improving, holding steady, or decreasing. They may also run it on different users during the same time period for comparison.
The KPI’s collected from this report give management an objective view of individual editors and the editing organization as a whole, and can help them to address quality issues through further training, goal setting, and/or other corrective actions.
To collect this data over time, our solution is deployed with a few custom Telvent PX Subtasks that take care of capturing and archiving this data to a set of data warehousing tables. This allows for trends to be established over many years while not clogging up the underlying PX tables used by the Telvent software.
As you might imagine, the KPI’s are only limited by the workflow and data kept within your organization. As you add additional fields to your Session or Design screens within the Telvent software, or add additional statuses and transitions to the workflow, the performance analysis module can be easily updated to account for almost any valuable KPI you can think of. Plug the KPI’s into a web dashboard and you’ll have more management visibility to the performance of your GIS group than you’ve ever had before.
I hope this article has sparked some thoughts on how you can extract some new KPI’s out of your editing group. We’ve looked at doing a similar implementation on top of Esri WFM (previously JTX) but have not had the push from a customer as of yet. Almost all of our utility customers use Telvent Session Manager and Designer™ Workflow Manager so this has been the focus thus far. As always, if you have thoughts on the topic we’d love to hear from you.