{ "cells": [ { "cell_type": "markdown", "metadata": { "id": "mYr7XZoTuqBs" }, "source": [ "# Bulk Geocoding Addresses" ] }, { "cell_type": "markdown", "metadata": { "id": "hFKihKpEuv99" }, "source": [ "## Introduction\n", "\n", "Geocoding is the process of converting addresses to coordinates. It typically involves querying a web-based geocoding service that can understand and map the address to a real-world location. [GeoPy](https://geopy.readthedocs.io/en/stable/) is a Python library that offers an easy-to-use methods to use many popular geocoding services. In this tutorial, we will learn how to use GeoPy with Pandas and GeoPandas for bulk geocoding. We cover the following two methods for geocoding:\n", "\n", "* *Geocoding Method 1*: Pandas and GeoPy: Our recommended method that offers more flexibility and control over the whole process.\n", "* *Geocoding Method 2*: GeoPandas: A simpler method using GeoPandas' built-in [`geopandas.tools.geocode`](https://geopandas.org/en/stable/docs/reference/api/geopandas.tools.geocode.html) function.\n", "\n", "## Overview of the Task\n", "\n", "We will be working with an Excel sheet containing addresses of Hurricane Evacuation Centers in New York City and geocode these addresses to create a map.\n", "\n", "**Input Layers**:\n", "\n", "* `Hurricane_Evacuation_Centers.xlsx`: A spreadsheet containing names and addresses of hurricane shelters inn NYC.\n", "\n", "**Output**:\n", "\n", "* `hurricane_evacuation_centers.zip`: A zipped shapefile with geocoded locations of the hurricane evacuation centers.\n", "\n", "**Data Credit**:\n", "\n", "* [NYC Open Data Portal](https://data.cityofnewyork.us/Public-Safety/Hurricane-Evacuation-Centers-Map-/ayer-cga7)\n", "\n", "**Watch Video Walkthrough** " ] }, { "cell_type": "markdown", "metadata": { "id": "5F4UU8ky1_NM" }, "source": [ "## Setup and Data Download\n", "\n", "The following blocks of code will install the required packages and download the datasets to your Colab environment." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "8r-duS5kKuJ8" }, "outputs": [], "source": [ "%%capture\n", "if 'google.colab' in str(get_ipython()):\n", " !pip install leafmap mapclassify" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "LZY3cd9K1_NO" }, "outputs": [], "source": [ "import os\n", "import re\n", "import pandas as pd\n", "import geopandas as gpd\n", "import leafmap.foliumap as leafmap\n", "from geopy.geocoders import Nominatim\n", "from geopy.extra.rate_limiter import RateLimiter\n", "from zipfile import ZipFile" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "4SqUL0h81_NO" }, "outputs": [], "source": [ "data_folder = 'data'\n", "output_folder = 'output'\n", "\n", "if not os.path.exists(data_folder):\n", " os.mkdir(data_folder)\n", "if not os.path.exists(output_folder):\n", " os.mkdir(output_folder)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/" }, "id": "BilOp9-B1_NO", "outputId": "78204986-e9b2-46cc-eca0-db21a97b7039" }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Downloaded data/Hurricane_Evacuation_Centers.xlsx\n" ] } ], "source": [ "def download(url):\n", " filename = os.path.join(data_folder, os.path.basename(url))\n", " if not os.path.exists(filename):\n", " from urllib.request import urlretrieve\n", " local, _ = urlretrieve(url, filename)\n", " print('Downloaded ' + local)\n", "\n", "data_url = 'https://github.com/spatialthoughts/geopython-tutorials/releases/download/data/'\n", "\n", "download(data_url + 'Hurricane_Evacuation_Centers.xlsx')" ] }, { "cell_type": "markdown", "metadata": { "id": "GNO6nOET1_NO" }, "source": [ "## Procedure" ] }, { "cell_type": "markdown", "metadata": { "id": "GCC7mjMJ1_NP" }, "source": [ "Read the `Hurricane_Evacuation_Centers.xlsx` file using Pandas. This step requires the `openpyxl` package to be installed along with `pandas`. The dataset contains 64 addresses of hurricane evacuation centers in New York City." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 424 }, "id": "mjhj_ZJO1_NP", "outputId": "d4b23eea-74d4-458e-9f60-aa7e546514f5" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "summary": "{\n \"name\": \"address_df\",\n \"rows\": 64,\n \"fields\": [\n {\n \"column\": \"CITY\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 16,\n \"samples\": [\n \"Flushing\",\n \"Brooklyn\",\n \"Jamaica\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"EC_Name\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"P.S. 100 - Queens\",\n \"P.S. 268 - Queens\",\n \"J.H.S. 185 - Queens\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ADDRESS\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"111-11 118 Street\",\n \"92-07 175 Street\",\n \"147-26 25 Drive\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ZIP_CODE\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 526,\n \"min\": 10002,\n \"max\": 11433,\n \"num_unique_values\": 56,\n \"samples\": [\n 11354,\n 10022,\n 11358\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"BOROCODE\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1,\n \"min\": 1,\n \"max\": 5,\n \"num_unique_values\": 5,\n \"samples\": [\n 3,\n 5,\n 2\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"STATE\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"NY\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ACCESSIBLE\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"Y\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", "type": "dataframe", "variable_name": "address_df" }, "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
CITYEC_NameADDRESSZIP_CODEBOROCODESTATEACCESSIBLE
0FlushingJ.H.S. 185 - Queens147-26 25 Drive113544NYY
1BrooklynI.S. 258 - Brooklyn141 Macon Street112163NYY
2BrooklynI.S. 88 - Brooklyn544 7 Avenue112153NYY
3BronxWalton HS - X2780 Reservoir Avenue104682NYY
4BrooklynP.S./I.S.30 Mary White Ovington - Brooklyn7002 4 Avenue112093NYY
........................
59BronxI.S. 201 - Bronx730 Bryant Avenue104742NYY
60BronxP.S. 46 - Bronx2760 Briggs Avenue104582NYY
61Long Island CityAviation HS - Q45-30 36 Street111014NYY
62BronxP.S. 102 - Bronx1827 Archer Street104602NYY
63BronxMorris Heights Educational Complex - Bronx1780 Dr. Martin Luther King Jr. Blvd104532NYY
\n", "

64 rows × 7 columns

\n", "
\n", "
\n", "\n", "
\n", " \n", "\n", " \n", "\n", " \n", "
\n", "\n", "\n", "
\n", " \n", "\n", "\n", "\n", " \n", "
\n", "\n", "
\n", " \n", " \n", " \n", "
\n", "\n", "
\n", "
\n" ], "text/plain": [ " CITY EC_Name \\\n", "0 Flushing J.H.S. 185 - Queens \n", "1 Brooklyn I.S. 258 - Brooklyn \n", "2 Brooklyn I.S. 88 - Brooklyn \n", "3 Bronx Walton HS - X \n", "4 Brooklyn P.S./I.S.30 Mary White Ovington - Brooklyn \n", ".. ... ... \n", "59 Bronx I.S. 201 - Bronx \n", "60 Bronx P.S. 46 - Bronx \n", "61 Long Island City Aviation HS - Q \n", "62 Bronx P.S. 102 - Bronx \n", "63 Bronx Morris Heights Educational Complex - Bronx \n", "\n", " ADDRESS ZIP_CODE BOROCODE STATE ACCESSIBLE \n", "0 147-26 25 Drive 11354 4 NY Y \n", "1 141 Macon Street 11216 3 NY Y \n", "2 544 7 Avenue 11215 3 NY Y \n", "3 2780 Reservoir Avenue 10468 2 NY Y \n", "4 7002 4 Avenue 11209 3 NY Y \n", ".. ... ... ... ... ... \n", "59 730 Bryant Avenue 10474 2 NY Y \n", "60 2760 Briggs Avenue 10458 2 NY Y \n", "61 45-30 36 Street 11101 4 NY Y \n", "62 1827 Archer Street 10460 2 NY Y \n", "63 1780 Dr. Martin Luther King Jr. Blvd 10453 2 NY Y \n", "\n", "[64 rows x 7 columns]" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "excel_file = 'Hurricane_Evacuation_Centers.xlsx'\n", "excel_file_path = os.path.join(data_folder, excel_file)\n", "address_df = pd.read_excel(excel_file_path, skiprows=[0])\n", "address_df" ] }, { "cell_type": "markdown", "metadata": { "id": "FK57BFz7y4ey" }, "source": [ "The Nominatim geocoder expects the addresses with certain format. We fix our source addresses by making the street numbers as ordinal numbers.\n", "\n", "Examples\n", "\n", "- `45-30 36 Street` → `45-30 36th Street`.\n", "- `544 7 Avenue` → `544 7th Avenue`" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 424 }, "id": "gYHRCAbNus9X", "outputId": "ecf7a7c2-c838-452c-84ad-0f0309128cdf" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "summary": "{\n \"name\": \"address_df\",\n \"rows\": 64,\n \"fields\": [\n {\n \"column\": \"CITY\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 16,\n \"samples\": [\n \"Flushing\",\n \"Brooklyn\",\n \"Jamaica\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"EC_Name\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"P.S. 100 - Queens\",\n \"P.S. 268 - Queens\",\n \"J.H.S. 185 - Queens\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ADDRESS\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"111-11 118 Street\",\n \"92-07 175 Street\",\n \"147-26 25 Drive\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ZIP_CODE\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 526,\n \"min\": 10002,\n \"max\": 11433,\n \"num_unique_values\": 56,\n \"samples\": [\n 11354,\n 10022,\n 11358\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"BOROCODE\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1,\n \"min\": 1,\n \"max\": 5,\n \"num_unique_values\": 5,\n \"samples\": [\n 3,\n 5,\n 2\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"STATE\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"NY\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ACCESSIBLE\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"Y\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ADDRESS_FIXED\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"111-11 118th Street\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", "type": "dataframe", "variable_name": "address_df" }, "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
CITYEC_NameADDRESSZIP_CODEBOROCODESTATEACCESSIBLEADDRESS_FIXED
0FlushingJ.H.S. 185 - Queens147-26 25 Drive113544NYY147-26 25th Drive
1BrooklynI.S. 258 - Brooklyn141 Macon Street112163NYY141 Macon Street
2BrooklynI.S. 88 - Brooklyn544 7 Avenue112153NYY544 7th Avenue
3BronxWalton HS - X2780 Reservoir Avenue104682NYY2780 Reservoir Avenue
4BrooklynP.S./I.S.30 Mary White Ovington - Brooklyn7002 4 Avenue112093NYY7002 4th Avenue
...........................
59BronxI.S. 201 - Bronx730 Bryant Avenue104742NYY730 Bryant Avenue
60BronxP.S. 46 - Bronx2760 Briggs Avenue104582NYY2760 Briggs Avenue
61Long Island CityAviation HS - Q45-30 36 Street111014NYY45-30 36th Street
62BronxP.S. 102 - Bronx1827 Archer Street104602NYY1827 Archer Street
63BronxMorris Heights Educational Complex - Bronx1780 Dr. Martin Luther King Jr. Blvd104532NYY1780 Dr. Martin Luther King Jr. Blvd
\n", "

64 rows × 8 columns

\n", "
\n", "
\n", "\n", "
\n", " \n", "\n", " \n", "\n", " \n", "
\n", "\n", "\n", "
\n", " \n", "\n", "\n", "\n", " \n", "
\n", "\n", "
\n", " \n", " \n", " \n", "
\n", "\n", "
\n", "
\n" ], "text/plain": [ " CITY EC_Name \\\n", "0 Flushing J.H.S. 185 - Queens \n", "1 Brooklyn I.S. 258 - Brooklyn \n", "2 Brooklyn I.S. 88 - Brooklyn \n", "3 Bronx Walton HS - X \n", "4 Brooklyn P.S./I.S.30 Mary White Ovington - Brooklyn \n", ".. ... ... \n", "59 Bronx I.S. 201 - Bronx \n", "60 Bronx P.S. 46 - Bronx \n", "61 Long Island City Aviation HS - Q \n", "62 Bronx P.S. 102 - Bronx \n", "63 Bronx Morris Heights Educational Complex - Bronx \n", "\n", " ADDRESS ZIP_CODE BOROCODE STATE ACCESSIBLE \\\n", "0 147-26 25 Drive 11354 4 NY Y \n", "1 141 Macon Street 11216 3 NY Y \n", "2 544 7 Avenue 11215 3 NY Y \n", "3 2780 Reservoir Avenue 10468 2 NY Y \n", "4 7002 4 Avenue 11209 3 NY Y \n", ".. ... ... ... ... ... \n", "59 730 Bryant Avenue 10474 2 NY Y \n", "60 2760 Briggs Avenue 10458 2 NY Y \n", "61 45-30 36 Street 11101 4 NY Y \n", "62 1827 Archer Street 10460 2 NY Y \n", "63 1780 Dr. Martin Luther King Jr. Blvd 10453 2 NY Y \n", "\n", " ADDRESS_FIXED \n", "0 147-26 25th Drive \n", "1 141 Macon Street \n", "2 544 7th Avenue \n", "3 2780 Reservoir Avenue \n", "4 7002 4th Avenue \n", ".. ... \n", "59 730 Bryant Avenue \n", "60 2760 Briggs Avenue \n", "61 45-30 36th Street \n", "62 1827 Archer Street \n", "63 1780 Dr. Martin Luther King Jr. Blvd \n", "\n", "[64 rows x 8 columns]" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def make_ordinal(match):\n", " n = int(match.group(1))\n", " if 11 <= (n % 100) <= 13:\n", " suffix = 'th'\n", " else:\n", " suffix = ['th', 'st', 'nd', 'rd', 'th'][min(n % 10, 4)]\n", " return str(n) + suffix + match.group(2)\n", "\n", "def update_address(row):\n", " old_address = row['ADDRESS']\n", " pattern = r'(\\d+)(\\s+(?:Street|Avenue|Blvd|Drive))'\n", " result = re.sub(pattern, make_ordinal, old_address)\n", " return result\n", "\n", "address_df['ADDRESS_FIXED'] = address_df.apply(update_address, axis=1)\n", "address_df" ] }, { "cell_type": "markdown", "metadata": { "id": "dH_jpKja6dfd" }, "source": [ "We also create a new column containing the full address. The `CITY` column in the data are actually boroughs of NYC and adding it causing many geocoding requests to fail. Instead, we use the city name 'NYC' for all addresses." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 424 }, "id": "N3wLTfzc2HTP", "outputId": "f3887015-567a-4979-c66f-c6b2fe41a57d" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "summary": "{\n \"name\": \"address_df\",\n \"rows\": 64,\n \"fields\": [\n {\n \"column\": \"CITY\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 16,\n \"samples\": [\n \"Flushing\",\n \"Brooklyn\",\n \"Jamaica\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"EC_Name\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"P.S. 100 - Queens\",\n \"P.S. 268 - Queens\",\n \"J.H.S. 185 - Queens\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ADDRESS\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"111-11 118 Street\",\n \"92-07 175 Street\",\n \"147-26 25 Drive\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ZIP_CODE\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 526,\n \"min\": 10002,\n \"max\": 11433,\n \"num_unique_values\": 56,\n \"samples\": [\n 11354,\n 10022,\n 11358\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"BOROCODE\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1,\n \"min\": 1,\n \"max\": 5,\n \"num_unique_values\": 5,\n \"samples\": [\n 3,\n 5,\n 2\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"STATE\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"NY\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ACCESSIBLE\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"Y\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ADDRESS_FIXED\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"111-11 118th Street\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Full_Address\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"111-11 118th Street,NYC,NY,11420\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", "type": "dataframe", "variable_name": "address_df" }, "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
CITYEC_NameADDRESSZIP_CODEBOROCODESTATEACCESSIBLEADDRESS_FIXEDFull_Address
0FlushingJ.H.S. 185 - Queens147-26 25 Drive113544NYY147-26 25th Drive147-26 25th Drive,NYC,NY,11354
1BrooklynI.S. 258 - Brooklyn141 Macon Street112163NYY141 Macon Street141 Macon Street,NYC,NY,11216
2BrooklynI.S. 88 - Brooklyn544 7 Avenue112153NYY544 7th Avenue544 7th Avenue,NYC,NY,11215
3BronxWalton HS - X2780 Reservoir Avenue104682NYY2780 Reservoir Avenue2780 Reservoir Avenue,NYC,NY,10468
4BrooklynP.S./I.S.30 Mary White Ovington - Brooklyn7002 4 Avenue112093NYY7002 4th Avenue7002 4th Avenue,NYC,NY,11209
..............................
59BronxI.S. 201 - Bronx730 Bryant Avenue104742NYY730 Bryant Avenue730 Bryant Avenue,NYC,NY,10474
60BronxP.S. 46 - Bronx2760 Briggs Avenue104582NYY2760 Briggs Avenue2760 Briggs Avenue,NYC,NY,10458
61Long Island CityAviation HS - Q45-30 36 Street111014NYY45-30 36th Street45-30 36th Street,NYC,NY,11101
62BronxP.S. 102 - Bronx1827 Archer Street104602NYY1827 Archer Street1827 Archer Street,NYC,NY,10460
63BronxMorris Heights Educational Complex - Bronx1780 Dr. Martin Luther King Jr. Blvd104532NYY1780 Dr. Martin Luther King Jr. Blvd1780 Dr. Martin Luther King Jr. Blvd,NYC,NY,10453
\n", "

64 rows × 9 columns

\n", "
\n", "
\n", "\n", "
\n", " \n", "\n", " \n", "\n", " \n", "
\n", "\n", "\n", "
\n", " \n", "\n", "\n", "\n", " \n", "
\n", "\n", "
\n", " \n", " \n", " \n", "
\n", "\n", "
\n", "
\n" ], "text/plain": [ " CITY EC_Name \\\n", "0 Flushing J.H.S. 185 - Queens \n", "1 Brooklyn I.S. 258 - Brooklyn \n", "2 Brooklyn I.S. 88 - Brooklyn \n", "3 Bronx Walton HS - X \n", "4 Brooklyn P.S./I.S.30 Mary White Ovington - Brooklyn \n", ".. ... ... \n", "59 Bronx I.S. 201 - Bronx \n", "60 Bronx P.S. 46 - Bronx \n", "61 Long Island City Aviation HS - Q \n", "62 Bronx P.S. 102 - Bronx \n", "63 Bronx Morris Heights Educational Complex - Bronx \n", "\n", " ADDRESS ZIP_CODE BOROCODE STATE ACCESSIBLE \\\n", "0 147-26 25 Drive 11354 4 NY Y \n", "1 141 Macon Street 11216 3 NY Y \n", "2 544 7 Avenue 11215 3 NY Y \n", "3 2780 Reservoir Avenue 10468 2 NY Y \n", "4 7002 4 Avenue 11209 3 NY Y \n", ".. ... ... ... ... ... \n", "59 730 Bryant Avenue 10474 2 NY Y \n", "60 2760 Briggs Avenue 10458 2 NY Y \n", "61 45-30 36 Street 11101 4 NY Y \n", "62 1827 Archer Street 10460 2 NY Y \n", "63 1780 Dr. Martin Luther King Jr. Blvd 10453 2 NY Y \n", "\n", " ADDRESS_FIXED \\\n", "0 147-26 25th Drive \n", "1 141 Macon Street \n", "2 544 7th Avenue \n", "3 2780 Reservoir Avenue \n", "4 7002 4th Avenue \n", ".. ... \n", "59 730 Bryant Avenue \n", "60 2760 Briggs Avenue \n", "61 45-30 36th Street \n", "62 1827 Archer Street \n", "63 1780 Dr. Martin Luther King Jr. Blvd \n", "\n", " Full_Address \n", "0 147-26 25th Drive,NYC,NY,11354 \n", "1 141 Macon Street,NYC,NY,11216 \n", "2 544 7th Avenue,NYC,NY,11215 \n", "3 2780 Reservoir Avenue,NYC,NY,10468 \n", "4 7002 4th Avenue,NYC,NY,11209 \n", ".. ... \n", "59 730 Bryant Avenue,NYC,NY,10474 \n", "60 2760 Briggs Avenue,NYC,NY,10458 \n", "61 45-30 36th Street,NYC,NY,11101 \n", "62 1827 Archer Street,NYC,NY,10460 \n", "63 1780 Dr. Martin Luther King Jr. Blvd,NYC,NY,10453 \n", "\n", "[64 rows x 9 columns]" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "address_df['Full_Address'] = (\n", " address_df['ADDRESS_FIXED'] + ',' +\n", " 'NYC' + ',' +\n", " address_df['STATE']+ ',' +\n", " address_df['ZIP_CODE'].astype(str))\n", "address_df" ] }, { "cell_type": "markdown", "metadata": { "id": "eSNkYvJN6kUh" }, "source": [ "## Geocoding Method 1: Pandas\n", "\n", "This is the preferred method that gives you more control over the geocoding process but involves a few more steps." ] }, { "cell_type": "markdown", "metadata": { "id": "FZN8035SB1ke" }, "source": [ "Now we use the [Nominatim Geocoder](https://geopy.readthedocs.io/en/stable/#nominatim) from GeoPy. Nominatim geocoder is based on OpenStreetMap data.\n", "\n", "The geocoder is free to use but requires rate limiting our queries to 1 query per second. We use the `RateLimiter` provided by `geopy` package to set the delay. Since geocoding many address with the rate limit can take some time - we use a progress bar provided by the `tqdm` package to track the progress.\n", "\n", "> You can also install and run Nominatim on your computer without any limitations and can geocode large number of addresses easily. Here's a good tutorial showing how to [Run Nominatim using Docker](https://nominatim.org/release-docs/latest/admin/Installation/)." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 629, "referenced_widgets": [ "34f255cd9b6241cda2a6e18e01b3d80b", "3354b06c1af5469aace9c295cbbd045d", "0dcfaed53e984d538f11f422daf5c203", "844ef507709b453fb97020220e40bd91", "fc63e525b05d49038117d3326df33b0e", "e6d3a2c6bd794b04afd978889edb5c6f", "4f10e95325a74e17bc69936ba7711f9f", "a1da2a76a66b4ea496af99fa8399963c", "d8fbc4f8e5bd4c56ba8b0582a3cb3fec", "ab392d76d89d46c39371f653dbcc3ae2", "30fe09b03ef24740abd6f83617aef10b" ] }, "id": "ay6uWXM56w1H", "outputId": "104264be-5530-49f4-96fe-854b4eca716d" }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "34f255cd9b6241cda2a6e18e01b3d80b", "version_major": 2, "version_minor": 0 }, "text/plain": [ " 0%| | 0/64 [00:00\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
CITYEC_NameADDRESSZIP_CODEBOROCODESTATEACCESSIBLEADDRESS_FIXEDFull_Addresslocation
0FlushingJ.H.S. 185 - Queens147-26 25 Drive113544NYY147-26 25th Drive147-26 25th Drive,NYC,NY,11354(Junior High School 185, 147-26, 25th Drive, L...
1BrooklynI.S. 258 - Brooklyn141 Macon Street112163NYY141 Macon Street141 Macon Street,NYC,NY,11216(Junior High School 258, 141, Macon Street, Be...
2BrooklynI.S. 88 - Brooklyn544 7 Avenue112153NYY544 7th Avenue544 7th Avenue,NYC,NY,11215(544, 7th Avenue, Greenwood Heights, Brooklyn,...
3BronxWalton HS - X2780 Reservoir Avenue104682NYY2780 Reservoir Avenue2780 Reservoir Avenue,NYC,NY,10468(2780, Reservoir Avenue, Kingsbridge Heights, ...
4BrooklynP.S./I.S.30 Mary White Ovington - Brooklyn7002 4 Avenue112093NYY7002 4th Avenue7002 4th Avenue,NYC,NY,11209(7002, 4th Avenue, Bay Ridge, Brooklyn, Kings ...
.................................
59BronxI.S. 201 - Bronx730 Bryant Avenue104742NYY730 Bryant Avenue730 Bryant Avenue,NYC,NY,10474(730, Bryant Avenue, Hunts Point, The Bronx, B...
60BronxP.S. 46 - Bronx2760 Briggs Avenue104582NYY2760 Briggs Avenue2760 Briggs Avenue,NYC,NY,10458(2760, Briggs Avenue, Bedford Park, The Bronx,...
61Long Island CityAviation HS - Q45-30 36 Street111014NYY45-30 36th Street45-30 36th Street,NYC,NY,11101(45-30, 36th Street, Sunnyside, Queens, Queens...
62BronxP.S. 102 - Bronx1827 Archer Street104602NYY1827 Archer Street1827 Archer Street,NYC,NY,10460(Public School 102, 1827, Archer Street, The B...
63BronxMorris Heights Educational Complex - Bronx1780 Dr. Martin Luther King Jr. Blvd104532NYY1780 Dr. Martin Luther King Jr. Blvd1780 Dr. Martin Luther King Jr. Blvd,NYC,NY,10453(East 125th Street, Manhattan Community Board ...
\n", "

64 rows × 10 columns

\n", "
\n", "
\n", "\n", "
\n", " \n", "\n", " \n", "\n", " \n", "
\n", "\n", "\n", "
\n", " \n", "\n", "\n", "\n", " \n", "
\n", "\n", "
\n", " \n", " \n", " \n", "
\n", "\n", "
\n", " \n" ], "text/plain": [ " CITY EC_Name \\\n", "0 Flushing J.H.S. 185 - Queens \n", "1 Brooklyn I.S. 258 - Brooklyn \n", "2 Brooklyn I.S. 88 - Brooklyn \n", "3 Bronx Walton HS - X \n", "4 Brooklyn P.S./I.S.30 Mary White Ovington - Brooklyn \n", ".. ... ... \n", "59 Bronx I.S. 201 - Bronx \n", "60 Bronx P.S. 46 - Bronx \n", "61 Long Island City Aviation HS - Q \n", "62 Bronx P.S. 102 - Bronx \n", "63 Bronx Morris Heights Educational Complex - Bronx \n", "\n", " ADDRESS ZIP_CODE BOROCODE STATE ACCESSIBLE \\\n", "0 147-26 25 Drive 11354 4 NY Y \n", "1 141 Macon Street 11216 3 NY Y \n", "2 544 7 Avenue 11215 3 NY Y \n", "3 2780 Reservoir Avenue 10468 2 NY Y \n", "4 7002 4 Avenue 11209 3 NY Y \n", ".. ... ... ... ... ... \n", "59 730 Bryant Avenue 10474 2 NY Y \n", "60 2760 Briggs Avenue 10458 2 NY Y \n", "61 45-30 36 Street 11101 4 NY Y \n", "62 1827 Archer Street 10460 2 NY Y \n", "63 1780 Dr. Martin Luther King Jr. Blvd 10453 2 NY Y \n", "\n", " ADDRESS_FIXED \\\n", "0 147-26 25th Drive \n", "1 141 Macon Street \n", "2 544 7th Avenue \n", "3 2780 Reservoir Avenue \n", "4 7002 4th Avenue \n", ".. ... \n", "59 730 Bryant Avenue \n", "60 2760 Briggs Avenue \n", "61 45-30 36th Street \n", "62 1827 Archer Street \n", "63 1780 Dr. Martin Luther King Jr. Blvd \n", "\n", " Full_Address \\\n", "0 147-26 25th Drive,NYC,NY,11354 \n", "1 141 Macon Street,NYC,NY,11216 \n", "2 544 7th Avenue,NYC,NY,11215 \n", "3 2780 Reservoir Avenue,NYC,NY,10468 \n", "4 7002 4th Avenue,NYC,NY,11209 \n", ".. ... \n", "59 730 Bryant Avenue,NYC,NY,10474 \n", "60 2760 Briggs Avenue,NYC,NY,10458 \n", "61 45-30 36th Street,NYC,NY,11101 \n", "62 1827 Archer Street,NYC,NY,10460 \n", "63 1780 Dr. Martin Luther King Jr. Blvd,NYC,NY,10453 \n", "\n", " location \n", "0 (Junior High School 185, 147-26, 25th Drive, L... \n", "1 (Junior High School 258, 141, Macon Street, Be... \n", "2 (544, 7th Avenue, Greenwood Heights, Brooklyn,... \n", "3 (2780, Reservoir Avenue, Kingsbridge Heights, ... \n", "4 (7002, 4th Avenue, Bay Ridge, Brooklyn, Kings ... \n", ".. ... \n", "59 (730, Bryant Avenue, Hunts Point, The Bronx, B... \n", "60 (2760, Briggs Avenue, Bedford Park, The Bronx,... \n", "61 (45-30, 36th Street, Sunnyside, Queens, Queens... \n", "62 (Public School 102, 1827, Archer Street, The B... \n", "63 (East 125th Street, Manhattan Community Board ... \n", "\n", "[64 rows x 10 columns]" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from tqdm.notebook import tqdm\n", "\n", "tqdm.pandas()\n", "\n", "locator = Nominatim(user_agent='spatialthoughts', timeout=10)\n", "geocode = RateLimiter(locator.geocode, min_delay_seconds=1)\n", "\n", "address_df_pd = address_df.copy()\n", "address_df_pd['location'] = address_df_pd['Full_Address'].progress_apply(geocode)\n", "address_df_pd" ] }, { "cell_type": "markdown", "metadata": { "id": "M95b6QZ2Cyz1" }, "source": [ "The response of the geocoding request is a [`geopy.location.Location`](https://geopy.readthedocs.io/en/stable/#geopy.location.Location) object containing the address along with geocoded coordinates as latitude and longitude. We extract them and add them to separate columns." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 597 }, "id": "efUcANoc9_s8", "outputId": "18256541-58ff-4bbd-9070-ba24f4254806" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "summary": "{\n \"name\": \"address_df_pd\",\n \"rows\": 64,\n \"fields\": [\n {\n \"column\": \"CITY\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 16,\n \"samples\": [\n \"Flushing\",\n \"Brooklyn\",\n \"Jamaica\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"EC_Name\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"P.S. 100 - Queens\",\n \"P.S. 268 - Queens\",\n \"J.H.S. 185 - Queens\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ADDRESS\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"111-11 118 Street\",\n \"92-07 175 Street\",\n \"147-26 25 Drive\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ZIP_CODE\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 526,\n \"min\": 10002,\n \"max\": 11433,\n \"num_unique_values\": 56,\n \"samples\": [\n 11354,\n 10022,\n 11358\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"BOROCODE\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 1,\n \"min\": 1,\n \"max\": 5,\n \"num_unique_values\": 5,\n \"samples\": [\n 3,\n 5,\n 2\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"STATE\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"NY\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ACCESSIBLE\",\n \"properties\": {\n \"dtype\": \"category\",\n \"num_unique_values\": 1,\n \"samples\": [\n \"Y\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"ADDRESS_FIXED\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"111-11 118th Street\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Full_Address\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"111-11 118th Street,NYC,NY,11420\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"location\",\n \"properties\": {\n \"dtype\": \"object\",\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"latitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.08068914885752466,\n \"min\": 40.5419561,\n \"max\": 40.88821535,\n \"num_unique_values\": 63,\n \"samples\": [\n 40.83810565\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"longitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.08974170304874299,\n \"min\": -74.21397825256436,\n \"max\": -73.7288703287536,\n \"num_unique_values\": 63,\n \"samples\": [\n -73.86579040869547\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", "type": "dataframe", "variable_name": "address_df_pd" }, "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
CITYEC_NameADDRESSZIP_CODEBOROCODESTATEACCESSIBLEADDRESS_FIXEDFull_Addresslocationlatitudelongitude
0FlushingJ.H.S. 185 - Queens147-26 25 Drive113544NYY147-26 25th Drive147-26 25th Drive,NYC,NY,11354(Junior High School 185, 147-26, 25th Drive, L...40.774912-73.818619
1BrooklynI.S. 258 - Brooklyn141 Macon Street112163NYY141 Macon Street141 Macon Street,NYC,NY,11216(Junior High School 258, 141, Macon Street, Be...40.681918-73.945597
2BrooklynI.S. 88 - Brooklyn544 7 Avenue112153NYY544 7th Avenue544 7th Avenue,NYC,NY,11215(544, 7th Avenue, Greenwood Heights, Brooklyn,...40.660181-73.987930
3BronxWalton HS - X2780 Reservoir Avenue104682NYY2780 Reservoir Avenue2780 Reservoir Avenue,NYC,NY,10468(2780, Reservoir Avenue, Kingsbridge Heights, ...40.870261-73.896569
4BrooklynP.S./I.S.30 Mary White Ovington - Brooklyn7002 4 Avenue112093NYY7002 4th Avenue7002 4th Avenue,NYC,NY,11209(7002, 4th Avenue, Bay Ridge, Brooklyn, Kings ...40.633625-74.024038
.......................................
59BronxI.S. 201 - Bronx730 Bryant Avenue104742NYY730 Bryant Avenue730 Bryant Avenue,NYC,NY,10474(730, Bryant Avenue, Hunts Point, The Bronx, B...40.816040-73.885264
60BronxP.S. 46 - Bronx2760 Briggs Avenue104582NYY2760 Briggs Avenue2760 Briggs Avenue,NYC,NY,10458(2760, Briggs Avenue, Bedford Park, The Bronx,...40.867729-73.890294
61Long Island CityAviation HS - Q45-30 36 Street111014NYY45-30 36th Street45-30 36th Street,NYC,NY,11101(45-30, 36th Street, Sunnyside, Queens, Queens...40.743475-73.929464
62BronxP.S. 102 - Bronx1827 Archer Street104602NYY1827 Archer Street1827 Archer Street,NYC,NY,10460(Public School 102, 1827, Archer Street, The B...40.838106-73.865790
63BronxMorris Heights Educational Complex - Bronx1780 Dr. Martin Luther King Jr. Blvd104532NYY1780 Dr. Martin Luther King Jr. Blvd1780 Dr. Martin Luther King Jr. Blvd,NYC,NY,10453(East 125th Street, Manhattan Community Board ...40.802729-73.933493
\n", "

64 rows × 12 columns

\n", "
\n", "
\n", "\n", "
\n", " \n", "\n", " \n", "\n", " \n", "
\n", "\n", "\n", "
\n", " \n", "\n", "\n", "\n", " \n", "
\n", "\n", "
\n", " \n", " \n", " \n", "
\n", "\n", "
\n", "
\n" ], "text/plain": [ " CITY EC_Name \\\n", "0 Flushing J.H.S. 185 - Queens \n", "1 Brooklyn I.S. 258 - Brooklyn \n", "2 Brooklyn I.S. 88 - Brooklyn \n", "3 Bronx Walton HS - X \n", "4 Brooklyn P.S./I.S.30 Mary White Ovington - Brooklyn \n", ".. ... ... \n", "59 Bronx I.S. 201 - Bronx \n", "60 Bronx P.S. 46 - Bronx \n", "61 Long Island City Aviation HS - Q \n", "62 Bronx P.S. 102 - Bronx \n", "63 Bronx Morris Heights Educational Complex - Bronx \n", "\n", " ADDRESS ZIP_CODE BOROCODE STATE ACCESSIBLE \\\n", "0 147-26 25 Drive 11354 4 NY Y \n", "1 141 Macon Street 11216 3 NY Y \n", "2 544 7 Avenue 11215 3 NY Y \n", "3 2780 Reservoir Avenue 10468 2 NY Y \n", "4 7002 4 Avenue 11209 3 NY Y \n", ".. ... ... ... ... ... \n", "59 730 Bryant Avenue 10474 2 NY Y \n", "60 2760 Briggs Avenue 10458 2 NY Y \n", "61 45-30 36 Street 11101 4 NY Y \n", "62 1827 Archer Street 10460 2 NY Y \n", "63 1780 Dr. Martin Luther King Jr. Blvd 10453 2 NY Y \n", "\n", " ADDRESS_FIXED \\\n", "0 147-26 25th Drive \n", "1 141 Macon Street \n", "2 544 7th Avenue \n", "3 2780 Reservoir Avenue \n", "4 7002 4th Avenue \n", ".. ... \n", "59 730 Bryant Avenue \n", "60 2760 Briggs Avenue \n", "61 45-30 36th Street \n", "62 1827 Archer Street \n", "63 1780 Dr. Martin Luther King Jr. Blvd \n", "\n", " Full_Address \\\n", "0 147-26 25th Drive,NYC,NY,11354 \n", "1 141 Macon Street,NYC,NY,11216 \n", "2 544 7th Avenue,NYC,NY,11215 \n", "3 2780 Reservoir Avenue,NYC,NY,10468 \n", "4 7002 4th Avenue,NYC,NY,11209 \n", ".. ... \n", "59 730 Bryant Avenue,NYC,NY,10474 \n", "60 2760 Briggs Avenue,NYC,NY,10458 \n", "61 45-30 36th Street,NYC,NY,11101 \n", "62 1827 Archer Street,NYC,NY,10460 \n", "63 1780 Dr. Martin Luther King Jr. Blvd,NYC,NY,10453 \n", "\n", " location latitude longitude \n", "0 (Junior High School 185, 147-26, 25th Drive, L... 40.774912 -73.818619 \n", "1 (Junior High School 258, 141, Macon Street, Be... 40.681918 -73.945597 \n", "2 (544, 7th Avenue, Greenwood Heights, Brooklyn,... 40.660181 -73.987930 \n", "3 (2780, Reservoir Avenue, Kingsbridge Heights, ... 40.870261 -73.896569 \n", "4 (7002, 4th Avenue, Bay Ridge, Brooklyn, Kings ... 40.633625 -74.024038 \n", ".. ... ... ... \n", "59 (730, Bryant Avenue, Hunts Point, The Bronx, B... 40.816040 -73.885264 \n", "60 (2760, Briggs Avenue, Bedford Park, The Bronx,... 40.867729 -73.890294 \n", "61 (45-30, 36th Street, Sunnyside, Queens, Queens... 40.743475 -73.929464 \n", "62 (Public School 102, 1827, Archer Street, The B... 40.838106 -73.865790 \n", "63 (East 125th Street, Manhattan Community Board ... 40.802729 -73.933493 \n", "\n", "[64 rows x 12 columns]" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "address_df_pd['latitude'] = address_df_pd['location'].apply(lambda loc: loc.latitude if loc else None)\n", "address_df_pd['longitude'] = address_df_pd['location'].apply(lambda loc: loc.longitude if loc else None)\n", "address_df_pd" ] }, { "cell_type": "markdown", "metadata": { "id": "DSk-tNz0DP8M" }, "source": [ "Some addresses might have failed to geocode. Let's review them." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 89 }, "id": "meUUPnw7QMEO", "outputId": "2b138191-fb70-4a9f-b0f5-2a700c43bd63" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "repr_error": "0", "type": "dataframe", "variable_name": "failed" }, "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
CITYEC_NameADDRESSZIP_CODEBOROCODESTATEACCESSIBLEADDRESS_FIXEDFull_Addresslocationlatitudelongitude
54New YorkP.S. 48 - Manhattan4360-78 Broadway100331NYY4360-78 Broadway4360-78 Broadway,NYC,NY,10033NoneNaNNaN
\n", "
\n", "
\n", "\n", "
\n", " \n", "\n", " \n", "\n", " \n", "
\n", "\n", "\n", "
\n", " \n", " \n", " \n", "
\n", "\n", "
\n", "
\n" ], "text/plain": [ " CITY EC_Name ADDRESS ZIP_CODE BOROCODE STATE \\\n", "54 New York P.S. 48 - Manhattan 4360-78 Broadway 10033 1 NY \n", "\n", " ACCESSIBLE ADDRESS_FIXED Full_Address location \\\n", "54 Y 4360-78 Broadway 4360-78 Broadway,NYC,NY,10033 None \n", "\n", " latitude longitude \n", "54 NaN NaN " ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "failed = address_df_pd[address_df_pd['location'].isna()]\n", "failed" ] }, { "cell_type": "markdown", "metadata": { "id": "5jBoXnu1DpUK" }, "source": [ "We may discard these rows. But sometimes it is useful to manually geocode failed queries. You may use another service manually or obtain the coordinates from another source. The below block assigns the coordinates 40.85339,-73.93350 to the row with index 54." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "GNZvws7HDoub" }, "outputs": [], "source": [ "address_df_pd.loc[54, ['latitude', 'longitude']] = (40.85339,-73.93350)" ] }, { "cell_type": "markdown", "metadata": { "id": "JESmuzQFEYma" }, "source": [ "Select a subset of columns to keep in the final result and rename them." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "o_cl8EGZQW49" }, "outputs": [], "source": [ "address_df_pd = address_df_pd[['EC_Name', 'Full_Address', 'latitude', 'longitude']]\n", "address_df_pd.rename(columns = {'EC_Name': 'Name', 'Full_Address': 'Address'}, inplace=True)" ] }, { "cell_type": "markdown", "metadata": { "id": "6hQL6x41EeAC" }, "source": [ "We create a GeoDataFrame from the latitude and longitude columns." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 424 }, "id": "jrSrfWeB-u3U", "outputId": "0b3f458e-f087-46fa-eee2-a38a0616ff25" }, "outputs": [ { "data": { "application/vnd.google.colaboratory.intrinsic+json": { "summary": "{\n \"name\": \"address_gdf\",\n \"rows\": 64,\n \"fields\": [\n {\n \"column\": \"Name\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"P.S. 100 - Queens\",\n \"P.S. 268 - Queens\",\n \"J.H.S. 185 - Queens\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"Address\",\n \"properties\": {\n \"dtype\": \"string\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"111-11 118th Street,NYC,NY,11420\",\n \"92-07 175th Street,NYC,NY,11433\",\n \"147-26 25th Drive,NYC,NY,11354\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"latitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.08140365243352105,\n \"min\": 40.5419561,\n \"max\": 40.88821535,\n \"num_unique_values\": 64,\n \"samples\": [\n 40.67980995,\n 40.707151198284194,\n 40.7749125\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"longitude\",\n \"properties\": {\n \"dtype\": \"number\",\n \"std\": 0.0890292898907335,\n \"min\": -74.21397825256436,\n \"max\": -73.7288703287536,\n \"num_unique_values\": 64,\n \"samples\": [\n -73.82133694482174,\n -73.7854739585413,\n -73.81861937037871\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n },\n {\n \"column\": \"geometry\",\n \"properties\": {\n \"dtype\": \"geometry\",\n \"num_unique_values\": 64,\n \"samples\": [\n \"POINT (-73.82133694482174 40.67980995)\",\n \"POINT (-73.7854739585413 40.707151198284194)\",\n \"POINT (-73.81861937037871 40.7749125)\"\n ],\n \"semantic_type\": \"\",\n \"description\": \"\"\n }\n }\n ]\n}", "type": "dataframe", "variable_name": "address_gdf" }, "text/html": [ "\n", "
\n", "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
NameAddresslatitudelongitudegeometry
0J.H.S. 185 - Queens147-26 25th Drive,NYC,NY,1135440.774912-73.818619POINT (-73.81862 40.77491)
1I.S. 258 - Brooklyn141 Macon Street,NYC,NY,1121640.681918-73.945597POINT (-73.9456 40.68192)
2I.S. 88 - Brooklyn544 7th Avenue,NYC,NY,1121540.660181-73.987930POINT (-73.98793 40.66018)
3Walton HS - X2780 Reservoir Avenue,NYC,NY,1046840.870261-73.896569POINT (-73.89657 40.87026)
4P.S./I.S.30 Mary White Ovington - Brooklyn7002 4th Avenue,NYC,NY,1120940.633625-74.024038POINT (-74.02404 40.63362)
..................
59I.S. 201 - Bronx730 Bryant Avenue,NYC,NY,1047440.816040-73.885264POINT (-73.88526 40.81604)
60P.S. 46 - Bronx2760 Briggs Avenue,NYC,NY,1045840.867729-73.890294POINT (-73.89029 40.86773)
61Aviation HS - Q45-30 36th Street,NYC,NY,1110140.743475-73.929464POINT (-73.92946 40.74347)
62P.S. 102 - Bronx1827 Archer Street,NYC,NY,1046040.838106-73.865790POINT (-73.86579 40.83811)
63Morris Heights Educational Complex - Bronx1780 Dr. Martin Luther King Jr. Blvd,NYC,NY,1045340.802729-73.933493POINT (-73.93349 40.80273)
\n", "

64 rows × 5 columns

\n", "
\n", "
\n", "\n", "
\n", " \n", "\n", " \n", "\n", " \n", "
\n", "\n", "\n", "
\n", " \n", "\n", "\n", "\n", " \n", "
\n", "\n", "
\n", " \n", " \n", " \n", "
\n", "\n", "
\n", "
\n" ], "text/plain": [ " Name \\\n", "0 J.H.S. 185 - Queens \n", "1 I.S. 258 - Brooklyn \n", "2 I.S. 88 - Brooklyn \n", "3 Walton HS - X \n", "4 P.S./I.S.30 Mary White Ovington - Brooklyn \n", ".. ... \n", "59 I.S. 201 - Bronx \n", "60 P.S. 46 - Bronx \n", "61 Aviation HS - Q \n", "62 P.S. 102 - Bronx \n", "63 Morris Heights Educational Complex - Bronx \n", "\n", " Address latitude longitude \\\n", "0 147-26 25th Drive,NYC,NY,11354 40.774912 -73.818619 \n", "1 141 Macon Street,NYC,NY,11216 40.681918 -73.945597 \n", "2 544 7th Avenue,NYC,NY,11215 40.660181 -73.987930 \n", "3 2780 Reservoir Avenue,NYC,NY,10468 40.870261 -73.896569 \n", "4 7002 4th Avenue,NYC,NY,11209 40.633625 -74.024038 \n", ".. ... ... ... \n", "59 730 Bryant Avenue,NYC,NY,10474 40.816040 -73.885264 \n", "60 2760 Briggs Avenue,NYC,NY,10458 40.867729 -73.890294 \n", "61 45-30 36th Street,NYC,NY,11101 40.743475 -73.929464 \n", "62 1827 Archer Street,NYC,NY,10460 40.838106 -73.865790 \n", "63 1780 Dr. Martin Luther King Jr. Blvd,NYC,NY,10453 40.802729 -73.933493 \n", "\n", " geometry \n", "0 POINT (-73.81862 40.77491) \n", "1 POINT (-73.9456 40.68192) \n", "2 POINT (-73.98793 40.66018) \n", "3 POINT (-73.89657 40.87026) \n", "4 POINT (-74.02404 40.63362) \n", ".. ... \n", "59 POINT (-73.88526 40.81604) \n", "60 POINT (-73.89029 40.86773) \n", "61 POINT (-73.92946 40.74347) \n", "62 POINT (-73.86579 40.83811) \n", "63 POINT (-73.93349 40.80273) \n", "\n", "[64 rows x 5 columns]" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "geometry = gpd.points_from_xy(address_df_pd.longitude, address_df_pd.latitude)\n", "address_gdf = gpd.GeoDataFrame(address_df_pd, crs='EPSG:4326', geometry=geometry)\n", "address_gdf" ] }, { "cell_type": "markdown", "metadata": { "id": "Lz18ejamE0FB" }, "source": [ "Visualize the geocoded addresses on a map." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "colab": { "base_uri": "https://localhost:8080/", "height": 521 }, "id": "1njk3JgK76-3", "outputId": "d9f7008a-1cd1-4a2d-a6f1-3c6ef4086300" }, "outputs": [ { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import folium\n", "m = leafmap.Map(width=800, height=500)\n", "address_gdf.explore(\n", " m=m,\n", " marker_type='marker',\n", " marker_kwds={\n", " 'icon': folium.Icon(color='#fdbb84', icon='hurricane', prefix='fa')\n", " }\n", ")\n", "m.zoom_to_gdf(address_gdf)\n", "m" ] }, { "cell_type": "markdown", "metadata": { "id": "KozdVrvBFDMs" }, "source": [ "Save the results as a shapefile." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "0C5iy_sfPpWi" }, "outputs": [], "source": [ "output_file = 'hurricane_evacuation_centers.shp'\n", "output_path = os.path.join(output_folder, output_file)\n", "\n", "address_gdf.to_file(filename=output_path)" ] }, { "cell_type": "markdown", "metadata": { "id": "gXdGHsAIGXnk" }, "source": [ "We also zip the resulting shapefile along with all sidecar files so it can be downloaded easily." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "5ipKG9LZGclo" }, "outputs": [], "source": [ "output_zip_file = 'hurricane_evacuation_centers.zip'\n", "output_zip_path = os.path.join(output_folder, output_zip_file)\n", "\n", "sidecar_files = [\n", " os.path.join(output_folder, file)\n", " for file in os.listdir(output_folder)\n", " if file.endswith(('shp', 'shx', 'dbf', 'prj'))\n", "]\n", "\n", "with ZipFile(output_zip_path, 'w') as zip_object:\n", " for sidecar in sidecar_files:\n", " zip_object.write(sidecar, os.path.basename(sidecar))\n" ] }, { "cell_type": "markdown", "metadata": { "id": "3aI_0Op_7Lqe" }, "source": [ "## Geocoding Method 2: GeoPandas\n" ] }, { "cell_type": "markdown", "metadata": { "id": "AnHsA5gD9l70" }, "source": [ "GeoPandas also provides a helper tool [geopandas.tools.geocode](https://geopandas.org/en/stable/docs/reference/api/geopandas.tools.geocode.html) for geocoding that allows using a GeoPy geocoder. This is a handy tool that gives you a GeoDataFrame but doesn't offer the flexibility of the previous method. We demonstrate the process here for the sake of completeness." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "UvUOajpB7Nzv" }, "outputs": [], "source": [ "geocoder_options = {\n", " 'user_agent': 'spatialthoughts',\n", " 'timeout': 10,\n", "}\n", "\n", "address_df_gpd = address_df.copy()\n", "\n", "# GeoPandas uses the value of 1-second for rate limiting Nominatim queries\n", "gdf = gpd.tools.geocode(\n", " address_df_gpd['ADDRESS_FIXED'],\n", " provider='Nominatim',\n", " **geocoder_options)\n", "\n", "gdf" ] }, { "cell_type": "markdown", "metadata": { "id": "c3naRS1o9_t9" }, "source": [ "We merge the resulting GeoDataFrame with the original DataFrame to retain some columns from the original dataset." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "G3hw4Zw_8RyR" }, "outputs": [], "source": [ "merged = gdf.join(address_df_gpd)\n", "merged" ] }, { "cell_type": "markdown", "metadata": { "id": "n_l7Y4p0_avG" }, "source": [ "We select the subset of columns and rename them." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "id": "rmHfdBf2_dkd" }, "outputs": [], "source": [ "merged = merged[['EC_Name', 'Full_Address', 'geometry']]\n", "merged.rename(columns = {'EC_Name': 'Name', 'Full_Address': 'Address'}, inplace=True)\n", "merged" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "----\n", "\n", "If you want to give feedback or share your experience with this tutorial, please comment below. (requires GitHub account)\n", "\n", "\n", "" ] } ], "metadata": { "colab": { "provenance": [] }, "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.13.1" }, "widgets": { "application/vnd.jupyter.widget-state+json": { "0dcfaed53e984d538f11f422daf5c203": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "FloatProgressModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "FloatProgressModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "ProgressView", "bar_style": "success", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_a1da2a76a66b4ea496af99fa8399963c", "max": 64, "min": 0, "orientation": "horizontal", "style": "IPY_MODEL_d8fbc4f8e5bd4c56ba8b0582a3cb3fec", "value": 64 } }, "30fe09b03ef24740abd6f83617aef10b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "3354b06c1af5469aace9c295cbbd045d": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_e6d3a2c6bd794b04afd978889edb5c6f", "placeholder": "​", "style": "IPY_MODEL_4f10e95325a74e17bc69936ba7711f9f", "value": "100%" } }, "34f255cd9b6241cda2a6e18e01b3d80b": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HBoxModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HBoxModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HBoxView", "box_style": "", "children": [ "IPY_MODEL_3354b06c1af5469aace9c295cbbd045d", "IPY_MODEL_0dcfaed53e984d538f11f422daf5c203", "IPY_MODEL_844ef507709b453fb97020220e40bd91" ], "layout": "IPY_MODEL_fc63e525b05d49038117d3326df33b0e" } }, "4f10e95325a74e17bc69936ba7711f9f": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "DescriptionStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "DescriptionStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "description_width": "" } }, "844ef507709b453fb97020220e40bd91": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "HTMLModel", "state": { "_dom_classes": [], "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "HTMLModel", "_view_count": null, "_view_module": "@jupyter-widgets/controls", "_view_module_version": "1.5.0", "_view_name": "HTMLView", "description": "", "description_tooltip": null, "layout": "IPY_MODEL_ab392d76d89d46c39371f653dbcc3ae2", "placeholder": "​", "style": "IPY_MODEL_30fe09b03ef24740abd6f83617aef10b", "value": " 64/64 [01:03<00:00,  1.02it/s]" } }, "a1da2a76a66b4ea496af99fa8399963c": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "ab392d76d89d46c39371f653dbcc3ae2": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "d8fbc4f8e5bd4c56ba8b0582a3cb3fec": { "model_module": "@jupyter-widgets/controls", "model_module_version": "1.5.0", "model_name": "ProgressStyleModel", "state": { "_model_module": "@jupyter-widgets/controls", "_model_module_version": "1.5.0", "_model_name": "ProgressStyleModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "StyleView", "bar_color": null, "description_width": "" } }, "e6d3a2c6bd794b04afd978889edb5c6f": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } }, "fc63e525b05d49038117d3326df33b0e": { "model_module": "@jupyter-widgets/base", "model_module_version": "1.2.0", "model_name": "LayoutModel", "state": { "_model_module": "@jupyter-widgets/base", "_model_module_version": "1.2.0", "_model_name": "LayoutModel", "_view_count": null, "_view_module": "@jupyter-widgets/base", "_view_module_version": "1.2.0", "_view_name": "LayoutView", "align_content": null, "align_items": null, "align_self": null, "border": null, "bottom": null, "display": null, "flex": null, "flex_flow": null, "grid_area": null, "grid_auto_columns": null, "grid_auto_flow": null, "grid_auto_rows": null, "grid_column": null, "grid_gap": null, "grid_row": null, "grid_template_areas": null, "grid_template_columns": null, "grid_template_rows": null, "height": null, "justify_content": null, "justify_items": null, "left": null, "margin": null, "max_height": null, "max_width": null, "min_height": null, "min_width": null, "object_fit": null, "object_position": null, "order": null, "overflow": null, "overflow_x": null, "overflow_y": null, "padding": null, "right": null, "top": null, "visibility": null, "width": null } } } } }, "nbformat": 4, "nbformat_minor": 4 }