Lab 6 Digitization and Editing with Kart
Written by Paul Pickell
Lab Overview
Version control is an important concept to software development that has recently been adopted and adapted for use in GIS. The ability to track data changes, manage versions, and collaborate on data editing are significant advances for managing geospatial data assets in an enterprise environment.
In this lab, you will learn about version control using both Git and Kart software. Git is a widely used distributed version control software system for tracking changes to files. Kart simply extends this distributed version control to geospatial vector data including points, lines, polygons, and point cloud datasets. You will also practice digitizing and editing datasets in QGIS and ArcGIS Pro.
Learning Objectives
- Practice using distributed version control software systems Git and Kart
- Digitize and edit geospatial features in QGIS and ArcGIS Pro
- Commit edits to a local repository
- Push edits to a local repository
- Manage merge conflicts and diffs between working copies of a repository
Deliverables
Answers to the questions posed throughout the lab (20 points)
Map response to the question posed at the end of Task 4 (10 points)
Data
All data for this lab are accessible via the UBC PostgreSQL server, the public MGEM Data Store, and the public MGEM Geoserver. Instructions for connecting to the server and data store are given in the tasks below and prior labs.
Task 1: Configure Git, install Kart, and create your first repository
At the core of version control with Git, each change made to a file is tracked with a message by a user, so we need to configure Git with a user name and e-mail address to sign our commit messages with. For the purpose of this lab, you can choose any username or e-mail address. You are not actually creating any account information and the user name and e-mail address you enter (real or fake) will only be stored on your local machine for this lab. We are not using GitHub for this lab, so you can just copy and paste the code in the steps below. However, if you will eventually start working with a hosted service like GitHub, then you will want to configure your local Git software with the user name and e-mail address associated with your GitHub account.
Step 1: Search for and open “Git Bash” in Windows.
Step 2: Enter the following command git config --global user.name "student" and press enter. Next, enter the following command git config --global user.email "you@example.com". Git is now configured and you can close the Git Bash.
Step 3: Open QGIS and install the Kart plugin using the Plugin Manager from the top toolbar. If you have not already installed the latest version of Kart on your computer, then you will be prompted to download and install the latest version.
Step 4: Create a directory on your computer where you can store your first Kart repository, which we will call ubcv. For example, C:\Kart\ubcv.
Step 5: From the QGIS navigation bar at the top, select “Plugins” > “Kart” > “Repositories…”. This opens the Kart repositories pane.
Step 6: From the Kart repositories pane, right-click “Repositories” and select “Create new repository…”. Add the path to the directory you created in Step 2, ensure “Storage Type” is set to Geopackage and click “OK” to initiate the Kart repository.
Expand your repositories list and you should see your new repostiory. If you expand “Datasets”, you will see nothing there. What we have done is basically told Kart to watch this directory for any changes to the files. If you navigate to the folder in your file system, you will see some new files KART_README.txt, .git, and .kart that Kart has added, which enables version control in this new repository. Now let us add some data and visualize it in QGIS.
Step 7: Right-click the repository C:\Kart\ubcv [main] and then select “Import dataset from database…”. We are going to connect to the ubcv database on the UBC PostgreSQL server.
Step 8: Add the parameters to connect to the UBC PostgreSQL server. Be sure to switch to the “Basic” tab for credentials and add the student credential that was provided to the class. Once you have added the credential, click the “Load Tables” button to so a soft connection to the database. If the connection was successful, then select “ubcv_campus_trees” from the drop-down menu and then click “OK” to add the table to your Kart repository.
We will be comparing this point dataset to a recent orthophoto of UBC Vancouver campus and then making some edits.
Step 9: Add the most recent UBC orthophoto to your QGIS project. Navigate to the MGEM Orthophoto Data Store and then right-click the most recently dated “UBC_ORTHOPHOTO_YEAR_COG.tif” file listed there and then select “Copy Link”. Return to your QGIS project, open the Data Source Manager, select “Raster” from the left navigation bar, change “Source Type” to “Protocol: HTTP(S), cloud, etc.”, and then paste the link that you copied into the “URI” field and press “Add”.
UBC orthophotos are publicly available datasets that are also encoded as Cloud Optimized Geotiffs (COG), which enables fast retrieval and zoom levels for cloud-hosted raster datasets. If you find that the file hosted on the server is not very responsive, you can also download the geotiff to your computer and add it as a regular file. Note that these orthophotos are large files (~7 GB) and ensure you have enough disk space in the location you want to save it.
Step 10: Drag and drop the “ubcv_campus_trees” dataset into your map from the Kart repository pane. Your QGIS project should look something like what is shown below.
Step 11: Zoom in to inspect the trees and the orthophoto.
Task 2: Edit a layer in QGIS with version control
Since the campus trees do not perfectly align with the locations in the orthophoto, we will practice editing some of them and then use the version control capabilities of Kart to inspect and manage the changes.
Step 1: From your QGIS toolbar, click the “Toggle Editing” icon to start an editing session.
Step 2: Once an editing session is started, you will be given the option to add new points or edit the vertices of existing points. Click the “Vertex Tool” icon to edit the existing point dataset.
Step 3: Zoom into an area where you can easily see where some points are not perfectly intersecting with the bottom of the tree trunk in the orthophoto. In the example below, we are looking at some trees just outside the Forest Sciences Centre at UBC on Agronomy Road. Right-click on the point you want to edit to show its current coordinate and it will highlight red in the map. Then left-click the same point and your cursor will now have a red “x” to indicate you are ready to place the new coordinate. Left-click again to move the point to the location of your cursor at the base of the tree in the orthophoto. If you are not satisfied with the location, just left-click the point and then left-click again to place it where you want. Repeat this step for 3-5 more points.
Step 4: Add a missing tree by clicking the “Add Point Feature” icon. Your cursor will now have a cross-hair and you simply left-click again anywhere on the map canvas to add the new point.
Step 5: A dialogue window should appear that prompts you to add values for the attributes. Since we do not really know any of these values, just let Kart autopopulate most of the values. Scroll down to “notes” and add a short message to this field then click “OK”. Add at least one point this way and then click the “Save Layer Edits” on the toolbar.
Step 6: Right-click the repository from the Kart repositories pane and select “Show working copy changes…”. This will open the Diff viewer that allows you to inspect the changes you just made to the layer. In the screenshot below, you can see that we added one point and modified four others. You can click the “Geometries” tab to view the change you made to the geometry of the feature, which is really handy.
Step 7: Right-click the repository again and this time select “Commit working copy changes…”. You will be prompted to enter a commit message. Commit messages help you and your collaborators understand what is happening in the changes that you just made. For this commit message, we are going to indicate that we edited four point locations and added one point.
You can think of committing as a way of saving your work progress in your repository. Every commit can be inspected and/or reverted if needed, so it is good practice to commit changes that are related to each other so that the commit messages can be concise and informative. Avoid committing every single edit separately as this can make navigating and interpreting the commit history difficult.
Step 8: Right-click the repository again and this time select “Show log…”. This will open the commit history for the repository and you should see the most recent commit you just made at the top along with the commit message you just entered in the last step. You can right-click any of the commits and select “Show changes introduced by this commit…”, which is another way to inspect the changes to attributes and geometries of the features with the diff viewer.
Task 3: Working on and merging different branches
Step 1: Right-click the repository and select “Show log…” to display the commit history.
Step 2: Right-click the commit you made in Task 2 and select “Create branch at this commit…”. You will be prompted to enter a branch name, name it “mybranch”. You should now see both “main” and “mybranch” noted on the top commit.
Step 3: Right-click the top commit again and select “Switch to branch ‘mybranch’…”.
Step 4: Now working on “mybranch”, start an editing session, pan around the orthophoto and add 5-10 more missing tree points. Do not change anything about the attributes, just click “OK” to dismiss after creating the point. Be sure to save your edits to the layer and then commit the change.
Step 5: Open the commit history of the repository. You will see now that “mybranch” is ahead of “main” by the commit you just made.
Step 6: Switch the branch back to “main” by right-clicking the commit tagged with “main” and then digitize 5-10 different tree points. Do not change anything about the attributes, just click “OK” to dismiss after creating the point. Commit your edits to the repository.
Step 7: Right-click the repository and select “Merge into current branch…”. In the dialogue window that appears, select “mybranch” from the drop-down menu for “Branch” and click “OK”. Since we are currently on the “main” branch, this will merge the edits/commits in “mybranch” into “main”.
Step 8: You should see an error message appear warning you about a merge conflict between the two branches. Inspect the merge conflict by right-clicking the repository and selecting “Resolve conflicts…”. Inspect the conflicting commits then answer the question below. You may need to close the Merge Conflicts window and inspect the ubcv_campus_trees attribute table to really appreciate what is going on here.
Q3. Describe why the merge conflict occurred. (3 points)
Step 9: Once you have answered the question above, open the Merge Conflicts window again. Click each feature one-at-a-time listed in the left then click “Use modified feature”. This is going to force Kart to recognize the most recent edits we made in “mybranch” and overwrite the conflicts in the “main” branch (our current branch). Once you have accepted all the modified feature edits, the Merge Conflict window will automatically close.
Q4. What is the difference between the “modified feature” and “ancestor feature”? (1 point)
Step 10: Open the commit history for the repository. You should now see the commit history between both branches, and we are now back to a single up-to-date branch (“main”).
In this example, we simulated two different branches to explore these tools. We generally try to avoid merge conflicts rather than produce them intentionally! Branches are really helpful for managing collaborative work on different aspects of the same project.
For example, we can allocate specific work to a specific branch, like a “digitizetrees” branch might be for digitizing trees only and a “identifytrees” branch might be for changing the attribute values for the tree species of the points that are made from the other branch. Then that leaves the “main” branch as the merge point for all the work. In this way, it is possible to collaboratively edit a large dataset with multiple people without causing merge conflicts between branches.
Task 4: Digitize some buildings in ArcGIS Pro
Step 1: Start a new ArcGIS Pro project.
As of the writing of this lab, ArcGIS Pro does not natively support adding a remote data source via a URL like you can do in QGIS. To get around this, we will create a reference to the remote COG by converting it to a virtual raster (VRT) format in QGIS.
Step 2: In QGIS, select “Raster” from the top menu bar, then “Conversion”, and finally select “Translate (convert format)…”. The input file will be the UBC orthophoto COG that you added to your QGIS project earlier and scroll down to “Converted”, then navigate to a directory where you want to save the VRT on your machine. Your ArcGIS Pro project folder is a sensible choice. Name the output something like “UBC_ORTHOPHOTO_2022.vrt” and ensure that the file extension is VRT. Click “Run” and this tool should complete in a couple seconds.
Step 3: Navigate to the location of the VRT in ArcGIS Pro and add it to your project or just drop and drag it into your project map. It may take a few minutes to calculate statistics, but you can cancel that to immediately display the COG in ArcGIS Pro.
In the following steps, we are going to digitize a building on campus. To do so in ArcGIS Pro, we need to first create a new feature class that will hold the polygon we are going to digitize.
Step 4: In a Catalog window pane, expand “Databases” and then right-click the geodatabase for your project. Select “New” and then “Feature Class”. Name it “my_ubc_buildings” and make sure the type is set to “Polygon”. Click “Next” to view fields. Add two fields: “Name” (data type is text) and “Vertices” (data type is double). Click “Next” to view the spatial reference. Probably the default is set to WGS 1984. Under “Projected Coordinate System” find NAD 1983 UTM Zone 10N. You can continue clicking “Next” to view more properties that you can set, but we will leave these as the defaults, so you can click “Finish” when you are done. You should now have an empty polygon feature class in you map.
Step 5: Click the “Edit” tab and click the “Create” button to start creating new features. The “Create Features” pane will open. Click on “my_ubc_buildings” to highlight it and start the editing session. Now pick a building and zoom in so that you can clearly see its borders, then start left-clicking to add vertices along the building perimeter. Be sure to work either in a clockwise or counter-clockwise pattern otherwise you will criss-cross the boundary. Once you are happy, you can simply double left-click in the last position to finish the edit or press F2. Ta-da! You have now digitized your first building.
Step 6: At the top ribbon on the “Edit” tab, click “Attributes” to open the attribute editor for the selected feature. Change the name to the name of the building and click “Apply”.
Step 7: Digitize four more buildings on campus and add the building name to the attribute table. Once you are done, be sure to click “Save” on the top ribbon under the “Edit” tab to save your work to the feature class in the geodatabase.
Next, we are going to validate and compare your digitized buildings against the official building dataset that is produced by UBC Campus + Community Planning.
Step 8: Add the official “ubcv_buildings” data to your map from the ubcv database on the PostgreSQL server. For information about connecting to the server, refer to Lab 1. Since these data are read-only from the PostgreSQL server, we need to export the feature class to our local geodatabase so that we can make some calculations. We also need to change the coordinate system from WGS 1984 to our preferred projected coordinate system of NAD 1983 UTM Zone 10N. To do this, we can use a single tool “Project”. Save the output with a name of “ubcv_buildings_utm” to your geodatabase.
Step 9: Add the local copy of “ubcv_buildings_utm” to your map and compare your digitization to the official record. Make some observations and answer the questions below.
Q5. Describe why your boundary does not align with the official dataset? Give three possible sources of error. (3 points)
Q6. What do you observe about the location of the official building dataset compared with the apparent location of the buildings in the orthophoto? (3 points)
Q7. How could relief displacement have impacted your digitization? (2 points)
Step 10: Open the attribute table of the “ubcv_buildings_utm” layer and add a field called “Vertices” with double data type. Save the change then return to the attribute table, find the new field, right-click it, select “Calculate Geometry” and then under the “Property” drop-down, select “Number of vertices” and click “OK”. Repeat this step for “my_ubc_buildings”.
Step 11: Inspect the attribute tables of “ubcv_buildings_utm” and “my_ubc_buildings” and compare the “Shape_Area”, “Shape_Length”, and “Vertices” fields then answer the questions below.
Q8. Which building that you digitized had the largest deviation of Shape_Area compared with the official buildings dataset? Give the value difference in square meters and discuss why you think this building had the worst area error. (2 points)
Q9. Which building that you digitized had the largest deviation of Shape_Length compared with the official buildings dataset? Give the value difference in square meters and discuss why you think this building had the worst perimeter error. (2 points)
Your last deliverable for the lab will be a mapped response to the following question: Did you digitize more or fewer vertices than the official UBC dataset? Discuss the implications of more or fewer vertices on positional accuracy, area accuracy, and perimeter accuracy. Your map should show a single building exemplar and should be annotated with supporting text and other elements (e.g., arrows, text boxes, etc.) to answer the question and illustrate your understanding of the relationship between digitization and accuracy measures.
Summary
By now, you should appreciate that digitization is an imperfect process. There are many sources of errors that can accumulate, especially when you are working on a crowd-sourced or collaborative editing project. It takes time to develop and practice good digitization skills. You might want to take a moment to reflect on the question, “which data set is right?” Invariably, the answer is the one with “authority”.
Return to the Deliverables section to check off everything you need to submit for credit in the course management system.