{ "cells": [ { "cell_type": "markdown", "id": "58f1e441", "metadata": {}, "source": [ "# Getting Started With GerryChain\n", "\n", "\n", "\n", "This guide will show you how to start generating ensembles with GerryChain." ] }, { "cell_type": "markdown", "id": "4ca7cc40", "metadata": {}, "source": [ "## What You'll Need\n", "\n", "Before we can start running Markov chains, you'll need to:\n", "\n", "* Install `gerrychain` from PyPI. See the [installation guide](./install.md) for instructions.\n", "* Download [this example json of Pennsylvania's VTDs](https://github.com/mggg/GerryChain/blob/main/docs/_static/PA_VTDs.json)\n", " (about 20 MB; use the download button on that page).\n", "* Open your preferred Python environment (e.g. JupyterLab, IPython, or a `.py` file\n", " in your favorite editor) in the directory containing the `PA_VTDs.json` file\n", " that you downloaded.\n", "\n", "Throughout this guide, we will assume that the user is working with a jupyter notebook\n", "or a python script file (a \\*.py file). We will also assume that the end user is working\n", "through this guide is familiar with, but still relatively new to Python.\n", "\n", "Also please note that, while this guide makes use of the \"propose_random_flip\" proposal\n", "function, this is not the only proposal function available in GerryChain (in fact, it is\n", "the one that we use the least). We are only using it here because it provides for\n", "a simple example of how to run a Markov chain.\n", "\n", "> Note\n", ">\n", "> If you are having any trouble getting the materials on this tutorial to work, please\n", "> consult the [installation guide](./install.md) first. Working through its\n", "> [installation checks](./install.md#checking-your-installation) will confirm that your\n", "> environment is set up the way this guide expects, and will usually pinpoint the problem.\n", ">\n", "> If you're still having issues, please\n", "> [open an issue](https://github.com/mggg/GerryChain/issues) so we can fix the guide for\n", "> the next person, or send us a message at code[at]mggg[dot]org." ] }, { "cell_type": "markdown", "id": "f33d75ef", "metadata": {}, "source": [ "## Setting Up the Chain\n", "\n", "
\n", " Download PA File\n", "
\n", "
\n", "\n", "\n", "The central object of GerryCahin is the `MarkovChain`, so we shall begin there. A chain needs to\n", "know how many steps to take and, if you care about reproducibiliy, it requires either a random\n", "seed or a random number generator object.\n", "\n", "Once these are specified, we can update the central `MarkovChain` object to add the rest of the \n", "information necessary to run the chain:\n", "\n", "1. Initial Partition\n", "2. Proposal function\n", "3. Updaters\n", "4. Acceptance constraints\n", "\n", "\n", "> Note\n", ">\n", "> Older versions of GerryChian required the reverse paradigm: build all of the components for \n", "> running the chain and then use them to construct the `MarkovChain` object. Users reported that\n", "> having to remember all of the components that are needed before making the chain as a bit \n", "> confusing. The latest version should also produce meanigful warnings that inform the user of\n", "> any components that are missing." ] }, { "cell_type": "code", "execution_count": null, "id": "ff094ebb", "metadata": {}, "outputs": [], "source": [ "from gerrychain import Graph, MarkovChain, Partition\n", "from gerrychain.updaters import Tally, cut_edges\n", "\n", "chain = MarkovChain(\n", " total_steps=1000,\n", " rng=2024, # a seed for the random number generator\n", ")\n", "\n", "# Load the graph in from the provided json file\n", "graph = Graph.from_json(\"./PA_VTDs.json\")\n", "\n", "# The partition the chain starts from\n", "chain.initial_partition = Partition(\n", " graph,\n", " assignment=\"2011_PLA_1\",\n", ")\n", "\n", "# Quantities we want tracked on every partition the chain produces\n", "chain.add_updaters(\n", " {\n", " \"population\": Tally(\"TOT_POP\", alias=\"population\"),\n", " \"cut_edges\": cut_edges,\n", " }\n", ")" ] }, { "cell_type": "markdown", "id": "725c92c2", "metadata": {}, "source": [ "Here's what's happening in this code block.\n", "\n", "* We create the `MarkovChain` first, giving it only `total_steps` and `rng`. Everything else is\n", "optional at this point and can be filled in as we go. Building the chain up this way keeps each\n", "piece of configuration next to the explanation of what it does, and it means you never have to\n", "assemble every argument before you can start.\n", "\n", "* The `Graph.from_json()` classmethod creates a GerryChain `Graph` of the precincts. \n", "\n", " Note that this JSON file was previously created by creating a GerryChain `Graph` from a \n", " shapefile (if you don't know what a shapefile is, don't worry), and then writing that\n", " graph out as a JSON file. We used `Graph.from_file()` on the shapefile obtained from\n", " [mggg-states/PA-shapefiles](https://github.com/mggg-states/PA-shapefiles#metadata) .\n", " By default, the `from_file()` method copies all of the data columns from the shapefile's attribute \n", " table to the `Graph` object as node attributes. We then used `Graph.to_json()` to create\n", " the JSON file. \n", "\n", "* We then assign the chain's `initial_partition`, the districting plan the chain starts from. The\n", "`Partition` class takes two arguments here:\n", "\n", " graph\n", " : A graph.\n", "\n", " assignment\n", " : An assignment of the nodes of the graph into parts of the partition. This can be either\n", " a dictionary mapping node IDs to part IDs, or the string key of a node attribute that holds\n", " each node's assignment. Here `assignment=\"2011_PLA_1\"` tells the `Partition`\n", " to assign nodes by their `\"2011_PLA_1\"` attribute.\n", "\n", "* Finally, `chain.add_updaters()` registers the quantities we want computed for every partition the\n", " chain visits. Updaters can be added before or after the initial partition is assigned; the chain\n", " applies them either way, and every partition it generates inherits them.\n", "\n", " updaters\n", " : `Tally(\"TOT_POP\", alias=\"population\")` sums the `\"TOT_POP\"` node attribute within each district\n", " and stores it under `\"population\"`. `cut_edges` records the edges that cross a district\n", " boundary." ] }, { "cell_type": "markdown", "id": "a296d853", "metadata": {}, "source": [ "With the `\"population\"` updater configured, we can see the total population in\n", "each of our districts. The initial partition is available on the chain as\n", "`chain.initial_partition`." ] }, { "cell_type": "code", "execution_count": null, "id": "ebdf5be1", "metadata": {}, "outputs": [], "source": [ "for district, pop in chain.initial_partition[\"population\"].items():\n", " print(f\"District {district}: {pop}\")" ] }, { "cell_type": "markdown", "id": "a2a7818e", "metadata": {}, "source": [ "Notice that `partition[\"population\"]` is a dictionary mapping the ID of each district to its total\n", "population (that's why we can call the `.items()` method on it). Most updaters output values in this dictionary format.\n", "\n", "And that is it! From here, you can move on to configuring and running the chain." ] }, { "cell_type": "markdown", "id": "bcacd7fe", "metadata": {}, "source": [ "## Running the Chain\n", "\n", "Our chain has now been configured to know where it starts and what to track, but it \n", "does not yet know how to propose a new\n", "plan, which plans count as valid, or when to accept one. Those are the last three pieces." ] }, { "cell_type": "code", "execution_count": null, "id": "b7998f3d", "metadata": {}, "outputs": [], "source": [ "from gerrychain.accept import always_accept\n", "from gerrychain.constraints import single_flip_contiguous\n", "from gerrychain.proposals import propose_random_flip\n", "\n", "chain.proposal_fn = propose_random_flip\n", "chain.add_constraint(single_flip_contiguous)\n", "chain.acceptance_fn = always_accept" ] }, { "cell_type": "markdown", "id": "30d0a8a2", "metadata": {}, "source": [ "A chain needs five things in total, and we have now supplied all of them.\n", "\n", "* proposal_fn\n", "\n", " : A function called as `proposal_fn(current_state, rng=chain.rng)` that returns a new `Partition`.\n", " Here we've used the `propose_random_flip` proposal, which proposes that a random node on the\n", " boundary of one district be flipped into the neighboring district.\n", "\n", "* constraints\n", "\n", " : Binary constraints (functions that take a partition and return `True` or `False`) that together\n", " define which districting plans are valid. Here we've added just one, `single_flip_contiguous`,\n", " which checks that each district in the plan is contiguous. This particular constraint is\n", " optimized for the single-flip proposal function we are using (hence the name). \n", " \n", " Call `add_constraint` again, or `add_constraints` with several at once, to require more. Each\n", " constraint is checked against the initial partition as soon as it is added, so a starting plan\n", " that violates one is reported immediately rather than when the chain runs.\n", "\n", "* acceptance_fn\n", "\n", " : A function called as `acceptance_fn(proposed_state, rng=chain.rng)` that returns `True` or\n", " `False`. Here `always_accept` accepts every valid proposal.\n", "\n", "* initial_partition\n", "\n", " : The plan the chain starts from, which we assigned above.\n", "\n", "* total_steps\n", "\n", " : The number of steps to take, which we passed when we created the chain.\n", "\n", "We also gave the chain `rng=2024` at the start. Each chain owns its randomness, so passing an\n", "integer makes the run reproducible. See [Reproducibility](../topics/reproducibility.md) for\n", "details.\n", "\n", "Note that the order in which you assign attributes to your chain does not matter. \n", "The chain validates the whole configuration before it starts iterating to produce\n", "successive plans, so you can assign these in whichever sequence is convenient.\n", "\n", "To start running your chain, just enumerate it:" ] }, { "cell_type": "code", "execution_count": null, "id": "0092b770", "metadata": {}, "outputs": [], "source": [ "for i, partition in enumerate(chain, start=1):\n", " if i <= 10:\n", " print(f\"Step {i} population for district 1: {partition['population'][1]}\")\n", "\n", "print(f\"... {i - 10} more steps omitted\")" ] }, { "cell_type": "markdown", "id": "e3cae0cd", "metadata": {}, "source": [ "Congratulations! You've run a Markov chain!" ] }, { "cell_type": "markdown", "id": "9a19298b", "metadata": {}, "source": [ "### Working With Elections\n", "\n", "Of course, `gerrychain` was build for analyzing districting plans, so it seems\n", "like it would be important to be able to analyze election results. We can do this\n", "by adding an `Election` object to our `Partition` as an updater. To do this, we'll need\n", "to import the `Election` class\n", "and change around our initial partition a bit." ] }, { "cell_type": "code", "execution_count": null, "id": "42835433", "metadata": {}, "outputs": [], "source": [ "from gerrychain import Election\n", "from gerrychain.constraints import contiguous\n", "\n", "# Set up the election updater. We give the election a name (\"SEN12\") and tell it\n", "# which node attribute in our graph holds the Democratic vote totals (\"USS12D\")\n", "# and which column holds the Republican vote totals (\"USS12R\").\n", "election = Election(\"SEN12\", {\"Dem\": \"USS12D\", \"Rep\": \"USS12R\"})\n", "\n", "chain_2 = MarkovChain(\n", " total_steps=1000,\n", " rng=2024,\n", ")\n", "\n", "chain_2.initial_partition = Partition(\n", " graph,\n", " assignment=\"2011_PLA_1\",\n", ")\n", "chain_2.add_updaters(\n", " {\n", " \"population\": Tally(\"TOT_POP\", alias=\"population\"),\n", " \"cut_edges\": cut_edges,\n", " \"SEN12\": election,\n", " }\n", ")\n", "\n", "chain_2.proposal_fn = propose_random_flip\n", "chain_2.add_constraint(contiguous)\n", "chain_2.acceptance_fn = always_accept" ] }, { "cell_type": "markdown", "id": "3760f025", "metadata": {}, "source": [ "The new chain is built exactly like the first one, with one extra updater. The election is\n", "registered under the name `\"SEN12\"`, so its results are available as `partition[\"SEN12\"]` on every\n", "partition the chain produces. We told `gerrychain` that the Democratic vote share, which we call\n", "`\"Dem\"`, is stored in the `\"USS12D\"` attribute of our file, and likewise that the Republican vote\n", "share, which we call `\"Rep\"`, is stored in the `\"USS12R\"` attribute.\n", "\n", "This chain also swaps the constraint: `contiguous` checks contiguity for any proposal, rather than\n", "assuming the single-flip proposal the way `single_flip_contiguous` does.\n", "\n", "Now we can run it and print off some election data." ] }, { "cell_type": "code", "execution_count": null, "id": "3a237ed5", "metadata": {}, "outputs": [], "source": [ "for i, partition in enumerate(chain_2, start=1):\n", " if i <= 10:\n", " print(\n", " f\"Step {i} Democratic vote share for district 1: \"\n", " f\"{partition['SEN12'].percents('Dem')[1]:0.4f}\"\n", " )\n", "\n", "print(f\"... {i - 10} more steps omitted\")" ] }, { "cell_type": "markdown", "id": "2463a7d9", "metadata": {}, "source": [ "> Coding note\n", ">\n", "> The `:0.4f` in the above code is a formatting string that tells Python to print\n", "> the preceding number with four decimal places. This is just a formatting string, and is\n", "> not specific to `gerrychain`. Also, we have split the string onto different lines for\n", "> readability since Python automatically concatenates adjacent strings." ] }, { "cell_type": "markdown", "id": "dca4cd3c", "metadata": {}, "source": [ "### Using DataFrames to Collect Information\n", "\n", "Printing out data is nice, but it's not very useful for analysis. Instead, it would\n", "be good if we could collect data from our Markov chain in a list\n", "and then convert it into a pandas `DataFrame` for analysis.\n", "\n", "Here we collect the percentages of Democratic voters computed by our Election updater \n", "and store them in a pandas `DataFrame`." ] }, { "cell_type": "code", "execution_count": null, "id": "ab1d6761", "metadata": {}, "outputs": [], "source": [ "import pandas\n", "\n", "d_percents = []\n", "for partition in chain_2:\n", " # We use the sorted function here to ensure that the data is in the same order\n", " # as the districts assignments\n", " d_percents.append(sorted(partition[\"SEN12\"].percents(\"Dem\")))\n", "\n", "data = pandas.DataFrame(d_percents)" ] }, { "cell_type": "markdown", "id": "47f4f5e9", "metadata": {}, "source": [ "> Coding note\n", ">\n", "> A more elegant way of achieving the same result is to use a list comprehension\n", "> instead of a `for` loop.\n", ">\n", "> ```python\n", "> data = pandas.DataFrame(\n", "> [sorted(partition[\"SEN12\"].percents(\"Dem\"))\n", "> for partition in chain_2]\n", "> )\n", "> ```\n", "\n", "> Important\n", ">\n", "> The above code will collect data from a different ensemble than the previous `for` loop.\n", "> Each time we iterate through the `chain` object, we run a brand new Markov chain\n", "> (using the same configuration that we defined when instantiating `chain`).\n", "\n", "The pandas [`DataFrame`](https://pandas.pydata.org/docs/reference/api/pandas.DataFrame.html)\n", "object also has many helpful methods for analyzing and plotting\n", "data. For example, we can produce a boxplot of our ensemble's Democratic vote percentage\n", "vectors, with the initial 2011 districting plan plotted in red, in just a few lines of code:" ] }, { "cell_type": "code", "execution_count": null, "id": "fbf48214", "metadata": {}, "outputs": [], "source": [ "import matplotlib.pyplot as plt\n", "\n", "ax = data.boxplot(positions=range(len(data.columns)))\n", "plt.plot(data.iloc[0], \"ro\")\n", "\n", "plt.show()" ] }, { "cell_type": "markdown", "id": "bb140440", "metadata": {}, "source": [ "(Before you over-analyze this data, keep in mind that this is a toy ensemble of just\n", "1000 plans created by single flips.)" ] }, { "cell_type": "markdown", "id": "d5e10a57", "metadata": {}, "source": [ "> Collecting data for long chains\n", ">\n", "> The dataframe method is convenient when running short (under 20,000 step) chains, but can become\n", "> cumbersome past that point. Often it is better practice to run and save the entire chain and then\n", "> to recollect the data on the saved chain. See \n", "> [https://gerrytools.readthedocs.io/en/latest/](https://gerrytools.readthedocs.io/en/latest/)\n", "> for more information on how to do this." ] }, { "cell_type": "markdown", "id": "8025fb58", "metadata": {}, "source": [ "## Next Steps\n", "\n", "To see a more elaborate example that uses the ReCom proposal, see\n", "[Running a chain with ReCom](./recom.ipynb).\n", "\n", "To learn more about the specific components of GerryChain, see the\n", "[API reference](../api.rst)." ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 5 }