{
"cells": [
{
"cell_type": "markdown",
"id": "53aec29b",
"metadata": {},
"source": [
"# Working With Geometries\n",
"\n",
"\n",
"\n",
"In the course of working with legislative redistricting data, it is inevitable that\n",
"we will have to work with files that contain geometries. Most often, these geometries\n",
"come in the form of shapefiles, which, while nice in theory, can be a bit of a pain to\n",
"work with in practice. For now, we will focus on the basics of working with geometries,\n",
"but the interested reader is encouraged to explore our partnered library,\n",
"[maup](https://github.com/mggg/maup#readme), which is specifically designed to\n",
"help fix tricky geometry problems. Specifically, in the event that you are working with\n",
"a shapefile and run in to an error of the flavour:\n",
"\n",
"```console\n",
"UserWarning: Found overlaps among the given polygons.\n",
"Indices of overlaps: {(887, 892), (893, 915), (892, 914), (887, 893)}\n",
"```\n",
"\n",
"or\n",
"\n",
"```console\n",
"UserWarning: Found islands (degree-0 nodes). Indices of islands: {2552, 3107}\n",
"\"Found islands (degree-0 nodes). Indices of islands: {}\".format(islands)\n",
"```\n",
"\n",
"then you should consider consulting the\n",
"[maup documentation](https://github.com/mggg/maup/wiki/)\n",
"to see if it can help you out."
]
},
{
"cell_type": "markdown",
"id": "388c42c3",
"metadata": {},
"source": [
"## Loading and Running a Plan\n",
"\n",
"\n",
"
\n",
"\n",
"For this example, we will make use of a Minnesota GeoJSON file that contains\n",
"the geometries of the state's precincts (you will need to unzip the above\n",
"folder to get to the file -- it's a bit large). We will follow a similar workflow to\n",
"what we already covered in the [ReCom section](./recom.ipynb), but with an eye\n",
"towards some of the conveniences afforded by `GeographicPartition` objects. As always,\n",
"we'll start with the imports:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "7ca2bb6b",
"metadata": {},
"outputs": [],
"source": [
"import matplotlib.pyplot as plt\n",
"from gerrychain import (\n",
" Partition,\n",
" Graph,\n",
" MarkovChain,\n",
" updaters,\n",
" constraints,\n",
" accept,\n",
" GeographicPartition,\n",
")\n",
"from gerrychain.proposals import build_recom_proposal_fn\n",
"from gerrychain.tree import bipartition_tree\n",
"from gerrychain.constraints import contiguous\n",
"import pandas"
]
},
{
"cell_type": "markdown",
"id": "54d9466e",
"metadata": {},
"source": [
"And now we load the graph from the GeoJSON file"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "75562c5b",
"metadata": {},
"outputs": [],
"source": [
"import zipfile\n",
"\n",
"with zipfile.ZipFile(\"MN.zip\") as z:\n",
" z.extractall()\n",
"\n",
"graph = Graph.from_file(\"MN_precincts.geojson\")"
]
},
{
"cell_type": "markdown",
"id": "1f8e8169",
"metadata": {},
"source": [
"as well as create our chain, with its initial partition and updaters"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d32ee9f0",
"metadata": {},
"outputs": [],
"source": [
"recom_chain = MarkovChain(\n",
" total_steps=20,\n",
" rng=42,\n",
")\n",
"\n",
"recom_chain.initial_partition = GeographicPartition(\n",
" graph,\n",
" assignment=\"CONGDIST\",\n",
")\n",
"\n",
"recom_chain.add_updaters(\n",
" {\n",
" \"population\": updaters.Tally(\"TOTPOP\", alias=\"population\"),\n",
" \"cut_edges\": updaters.cut_edges,\n",
" \"perimeter\": updaters.perimeter,\n",
" \"area\": updaters.Tally(\"area\", alias=\"area\"),\n",
" }\n",
")"
]
},
{
"cell_type": "markdown",
"id": "dcb95e58",
"metadata": {},
"source": [
"The observant reader will notice that we have added two new updaters, `perimeter`,\n",
"and `area`, [^1] and we are now using the `GeographicPartition` class instead of the\n",
"`Partition` class. The `GeographicPartition` class is a subclass of the\n",
"`Partition` class that allows us the capability of working with geometries throughout\n",
"our Markov chain, and the `perimeter` and `area` updaters are examples of such a\n",
"geometric updater that was previously unavailable to us. These updaters are necessary for\n",
"monitoring things like geometric compactness and area via metrics such as the Polsby-Popper\n",
"test. [^2]\n",
"\n",
"And now it is time for one of the first conveniences of the `GeographicPartition` class:\n",
"we can plot our map and see the initial partition!"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "6b751bc3",
"metadata": {},
"outputs": [],
"source": [
"recom_chain.initial_partition.plot()"
]
},
{
"cell_type": "markdown",
"id": "9c9a1a1b",
"metadata": {},
"source": [
"Of course, this isn't very pretty, so let's pass it some additional arguments to\n",
"things a bit nicer:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5f14bd43",
"metadata": {},
"outputs": [],
"source": [
"fig, ax = plt.subplots(figsize=(8, 8))\n",
"ax.set_yticks([])\n",
"ax.set_xticks([])\n",
"ax.set_title(\"Initial Partition in MN\")\n",
"recom_chain.initial_partition.plot(ax=ax, cmap=\"tab20c\")"
]
},
{
"cell_type": "markdown",
"id": "55ebd1c3",
"metadata": {},
"source": [
"Under the hood, the `plot` method is using the `geodataframe.plot` method from\n",
"[geopandas](https://geopandas.org/) to plot the geometries, and all of this is\n",
"built on top of `matplotlib`, so most of the standard methods for modifying a\n",
"`matplotlib` plot will work here as well.\n",
"\n",
"The chain already knows where it starts and what to track, so all that is left is to tell it how\n",
"to propose a plan, which plans are valid, and when to accept one:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "f3a61816",
"metadata": {},
"outputs": [],
"source": [
"ideal_population = sum(recom_chain.initial_partition[\"population\"].values()) / len(\n",
" recom_chain.initial_partition\n",
")\n",
"\n",
"recom_chain.proposal_fn = build_recom_proposal_fn(\n",
" pop_col=\"TOTPOP\",\n",
" pop_target=ideal_population,\n",
" epsilon=0.01,\n",
")\n",
"recom_chain.add_constraint(contiguous)\n",
"recom_chain.acceptance_fn = accept.always_accept"
]
},
{
"cell_type": "markdown",
"id": "774f7278",
"metadata": {},
"source": [
"The next cell builds an interactive viewer for watching the chain. Its Back and Forward\n",
"buttons run entirely in the browser, so they also work in the rendered documentation."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "d7933105",
"metadata": {},
"outputs": [],
"source": [
"from io import BytesIO\n",
"\n",
"from matplotlib.animation import ArtistAnimation\n",
"from PIL import Image\n",
"from IPython.display import HTML\n",
"\n",
"frames = []\n",
"district_data = []\n",
"\n",
"for i, partition in enumerate(recom_chain):\n",
" for district_name in partition[\"perimeter\"]:\n",
" district_data.append(\n",
" (\n",
" i,\n",
" district_name,\n",
" partition[\"population\"][district_name],\n",
" partition[\"perimeter\"][district_name],\n",
" partition[\"area\"][district_name],\n",
" )\n",
" )\n",
"\n",
" with BytesIO() as buffer:\n",
" fig, ax = plt.subplots(figsize=(10, 10))\n",
" partition.plot(ax=ax, cmap=\"tab20\")\n",
" ax.set_xticks([])\n",
" ax.set_yticks([])\n",
" fig.savefig(buffer, format=\"png\", bbox_inches=\"tight\", pad_inches=0)\n",
" frames.append(Image.open(buffer).copy())\n",
" plt.close(fig)\n",
"\n",
"df = pandas.DataFrame(\n",
" district_data,\n",
" columns=[\"step\", \"district_name\", \"population\", \"perimeter\", \"area\"],\n",
")\n",
"\n",
"fig, ax = plt.subplots(figsize=(8, 8))\n",
"ax.axis(\"off\")\n",
"fig.subplots_adjust(left=0, right=1, bottom=0, top=1)\n",
"artists = [[ax.imshow(frame, animated=True)] for frame in frames]\n",
"animation = ArtistAnimation(fig, artists, interval=500)\n",
"plt.close(fig)\n",
"HTML(animation.to_jshtml())"
]
},
{
"cell_type": "markdown",
"id": "c37101d3",
"metadata": {},
"source": [
"The dataframe collected the population, perimeter, and area for every district at each step."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4d8fc858",
"metadata": {},
"outputs": [],
"source": [
"df.head(5)"
]
},
{
"cell_type": "markdown",
"id": "0eeae947",
"metadata": {},
"source": [
"[^1]: The `area` attribute is added when the graph is built from the GeoDataFrame. The\n",
" `perimeter` updater computes district perimeters from the graph's geometries.\n",
"[^2]: The Polsby-Popper test is part of `gerrychain.metrics`."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
}
},
"nbformat": 4,
"nbformat_minor": 5
}