Downloading Images with XEE

Downloading Images with XEE#

Introduction#

When using data hosted in Google Earth Engine in Python-based workflows - one of the main pain points is the extraction of large datasets. This typically involves a long running Export process which makes subsequent processing of the results challenging. With the introduction of new ee.data.computePixels() API from GEE, we can now directly fetch image data as an array. The XEE package makes this process seamless and extracts the results as an XArray Dataset. We can then use rioxarray to clip and save the extracted XArray DataSet to a GeoTIFF file directly without going through an Export task.

For advanced XEE workflows with Dask, see the Processing Time-Series with XEE and XArray tutorial.

Overview of the Task#

LandScan Population Data Global 1km is one of the highest quality global gridded population datasets and is made easily accessible via the Awesome GEE Community Catalog. In this tutorial, we will download the 2021 population grid for Kenya as a GeoTIFF file directly from the cloud-hosted dataset.

Note: You must have a Google Earth Engine account to complete this section. If you do not have one, follow our guide to sign up.

Input Layers:

ne_10m_admin_0_countries_ind.zip: A shapefile of country boundaries

Output Layers:

population.tif : A GeoTIFF file of gridded population

Data Credit:

Sims, K., Reith, A., Bright, E., McKee, J., & Rose, A. (2022). LandScan Global 2021. Oak Ridge National Laboratory. https://doi.org/10. 48690/1527702
Made with Natural Earth. Free vector and raster map data @ naturalearthdata.com.

Running the Notebook: The preferred way to run this notebook is on Google Colab.

Setup and Data Download#

The following blocks of code will install the required packages and download the datasets to your Colab environment.

%%capture
if 'google.colab' in str(get_ipython()):
    !pip install xee rioxarray

import datetime
import ee
import geopandas as gpd
import matplotlib.pyplot as plt
import numpy as np
import os
import pandas as pd
import rioxarray as rxr
import xarray as xr
from xee import helpers

data_folder = 'data'
output_folder = 'output'

if not os.path.exists(data_folder):
    os.mkdir(data_folder)
if not os.path.exists(output_folder):
    os.mkdir(output_folder)

def download(url):
    filename = os.path.join(data_folder, os.path.basename(url))
    if not os.path.exists(filename):
        from urllib.request import urlretrieve
        local, _ = urlretrieve(url, filename)
        print('Downloaded ' + local)

shapefile = 'ne_10m_admin_0_countries_ind.zip'

data_url = 'https://github.com/spatialthoughts/geopython-tutorials/releases/download/data/'
download('{}/{}'.format(data_url,shapefile))

Downloaded data/ne_10m_admin_0_countries_ind.zip

Initialize EE with the High-Volume EndPoint which is recommended to be used with XEE for workflows that do not use a lot of server side processing and are primarily for extracting data from stored collections.. Replace the value of the cloud_project variable with your own project id that is linked with GEE.

cloud_project = 'spatialthoughts' # replace with your project id

try:
    ee.Initialize(
        project=cloud_project,
        opt_url='https://earthengine-highvolume.googleapis.com')
except:
    ee.Authenticate()
    ee.Initialize(
        project=cloud_project,
        opt_url='https://earthengine-highvolume.googleapis.com')

Downloading Images with XEE

Contents

Downloading Images with XEE#

Introduction#

Overview of the Task#

Setup and Data Download#

Data Preparation#

Downloading Data#