Skip to content
Natural-StatePublic

About

A Python package containing utilities used with Google Earth Engine for environmental remote sensing analysis

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

eetools

Reusable Google Earth Engine utilities for satellite data processing and analysis. eetools wraps multi-sensor imagery behind a consistent API: build cloud- and water-masked image collections, append a harmonized set of spectral indices and image texture, composite over time, reduce over site boundaries, export to Drive or Earth Engine assets, and summarize/visualize the results.

Supported sources: Sentinel-2 SR, Landsat 5/7/8/9 C2 L2 SR, NASA HLS (L30/S30), MODIS (MCD15A3H LAI/FPAR, MOD13Q1 NDVI/EVI, MCD64A1 burned area, MOD17A3HGF NPP, MOD44B vegetation continuous fields), NASA SMAP L4 soil moisture, CHIRPS precipitation, OpenLandMap SM2RAIN monthly precipitation climatology, NASA Black Marble VIIRS (VNP46A2) daily nighttime lights, ESA WorldCover, Sentinel-1 SAR GRD, OPERA DSWx surface water, Dynamic World LULC, Copernicus DEM/terrain, Hansen Global Forest Change, BII (Biodiversity Intactness Index), WDPA protected-area vectors, and Google Satellite Embedding V1 (AlphaEarth Foundations) annual 64-D embeddings. It also includes a LandTrendr temporal-segmentation method subpackage built on the harmonized Landsat builders.

Requirements

  • Python ≥ 3.11
  • A Google Earth Engine account, authenticated on the machine, and a registered Cloud project.

Installation

This project is managed with uv.

# Clone, then create the environment and install (with dev dependencies)
uv sync

To use eetools as a dependency in another project, install it directly:

uv pip install -e /path/to/gee_utils      # editable
# or
uv pip install git+https://github.com/Natural-State/gee_utils.git

Optional dependencies

Extra features live behind optional extras so the base install stays lean:

Extra Adds For
notebook geemap Interactive Earth Engine map visualization in Jupyter notebooks
# Local development — include an extra (or use --all-extras)
uv sync --extra notebook

# As a dependency in another project
uv pip install -e "/path/to/gee_utils[notebook]"
# or
uv pip install "eetools[notebook] @ git+https://github.com/Natural-State/gee_utils.git"

Authentication

Authenticate once per machine, then initialize in code with your project:

uv run earthengine authenticate
uv run earthengine set_project <your-project>   # optional: persists the default
import eetools

eetools.initialize(project="your-project")   # registers the project + ee.Initialize()

initialize() stores the project via eetools.configure(); functions that need it (e.g. asset exports) read it back through eetools.get_project().

Quickstart

import ee
import eetools
from eetools.sensors.sentinel.preprocessing import get_s2_sr_collection
from eetools.io import export_image_to_drive

eetools.initialize(project="your-project")

aoi = ee.Geometry.Rectangle([39.20, -4.30, 39.25, -4.25])
start, end = ee.Date("2021-06-01"), ee.Date("2021-09-01")

# Cloud- and water-masked Sentinel-2 SR with spectral indices appended
collection = get_s2_sr_collection(aoi, start, end)

# Median composite, exported to Google Drive
composite = collection.select("NDVI").median().clip(aoi)
task = export_image_to_drive(
    image=composite,
    aoi=aoi,
    description="ndvi_median",
    folder="eetools_exports",
    file_prefix="ndvi_median",
    scale=10,
)
print(task.status())

By default each collection carries a harmonized core of indices. To choose exactly which indices each image gets, pass indices (by name) and/or domains (whole families: vegetation, water, moisture, soil, burn, urban). Selection is sensor-aware — red-edge indices (MTCI, IRECI, S2REP, BAIS2) compute on Sentinel-2 and are silently skipped on sensors without a red edge.

# Whole domains — here without water masking, since these families omit MNDWI.
veg_burn = get_s2_sr_collection(
    aoi, start, end, domains=["vegetation", "burn"], apply_water_masking=False
)

# Specific indices by name. Water masking is on by default, so the selection must
# include NDVI and MNDWI (both are read by the spectral water mask).
named = get_s2_sr_collection(
    aoi, start, end, indices=["NDVI", "MNDWI", "NDBI", "BAIS2"]
)

Indices come from a central registry (eetools.sensors.indices.INDEX_REGISTRY); the same indices/domains arguments are accepted by calc_indices() directly. Adding a new index is a single registry entry.

Compositing

eetools.compositing reduces a collection over time:

  • build_composite makes a single composite.
  • build_period_composites makes one composite per year or month.
  • build_seasonal_composites makes one composite per year, restricted to a recurring window of months such as a wet or dry season.
from eetools.compositing import build_period_composites, build_seasonal_composites
from eetools.constants import SEASON_WET  # (3, 5) — Mar–May; override per region

# One median composite per year
annual = build_period_composites(
    collection, bands=["NDVI"], start_date="2018-01-01", end_date="2024-01-01",
    temporal_scale="annual",
)

# One median composite per year covering the wet-season months (Mar–May)
wet_season = build_seasonal_composites(
    collection=collection,
    bands=["NDVI"],
    start_year=2018,
    end_year=2023,
    season_months=SEASON_WET,
    season_name="wet",
    composite_stat="median",
)
# Each image carries year, season, season_months, image_count, composite_stat properties

Sensor builders

Each sensor exposes a get_<sensor>_collection(aoi, start_date, end_date, ...) builder that validates the date range, applies scale factors, and appends indices via the sensor's band map. Examples are get_l8_sr_collection, get_landsat_sr_collection, get_hls_merged_collection, get_chirps_collection, get_modis_vi_collection, get_s1_grd_collection and get_dynamic_world_collection.

These sources don't follow the collection-builder shape:

  • Static or single-image: get_copernicus_dem / get_terrain, get_forest_change_image (Hansen), get_bii, get_sm2rain_climatology.
  • Vector: get_wdpa_collection / get_wdpa_in_aoi return an ee.FeatureCollection.

Site boundaries and zonal statistics

eetools.vectors turns local boundary files (GeoPackage, GeoJSON, Shapefile, …) into Earth Engine geometries and features. eetools.zonal reduces images and collections over them. eetools.workflows.run_site_timeseries chains the whole pipeline for a set of sites: build the collection, composite it, reduce per site, and export the table to Drive.

from functools import partial

from eetools.vectors import vector_files_to_feature_collection
from eetools.workflows import run_site_timeseries
from eetools.visualization.tables import stats_fc_to_df

# One feature per file, with caller-supplied site identity
sites = vector_files_to_feature_collection(
    [("site_a.gpkg", "A", "Site A"), ("site_b.geojson", "B", "Site B")]
)

builder = partial(get_s2_sr_collection, indices=["NDVI", "EVI"])
stats = run_site_timeseries(
    sites_fc=sites,
    collection_builder=builder,
    bands=["NDVI", "EVI"],
    start_date="2020-01-01",
    end_date="2024-01-01",
    export_folder="eetools_exports",
    file_prefix="s2_annual",
    scale=10,
)
df = stats_fc_to_df(stats)  # pandas DataFrame, ready for eetools.visualization.plots

run_seasonal_site_timeseries does the same with seasonal composites. The default scale of these reducers is coarse (5566 m) so they run quickly, so set it to the sensor's resolution for real analyses.

Texture

eetools.sensors.texture computes GLCM (Haralick) and first-order texture on any band, typically a vegetation index on a same-season composite. Bands are quantized against a fixed value range (TEXTURE_VALUE_RANGES, overridable per band), because glcmTexture only accepts integers. Output bands are named <band>_<metric>_r<radius>.

from eetools.sensors.texture import calc_texture

s2 = get_s2_sr_collection(aoi.buffer(1000), start, end, indices=["EVI"])  # buffer >= window half-width
evi = s2.select("EVI").median()

# Habitat-heterogeneity covariates: homogeneity + entropy at two ground windows
tex = calc_texture(evi, ["EVI"], window_m=[50, 250], scale_m=10, mask=habitat_mask)
# -> EVI_idm_r2, EVI_ent_r2, EVI_idm_r12, EVI_ent_r12

# Extra classification features
feats = stack.addBands(
    calc_texture(evi, ["EVI"], window_m=50, scale_m=10,
                 glcm_metrics=("idm", "contrast"), first_order=("sd",))
)
tex = tex.clip(aoi)  # clip AFTER texturing

The window is radius pixels of the requested scale, so export or sample at scale_m. To view or sample at another scale, pass pin_scale=True, crs=.... This reprojects the output so the window keeps its ground size, but costs about 55% more compute. Texture within radius px of a masked area is computed from fewer pixels, so treat it with care.

Project layout

src/eetools/
├── _config.py        # global project-id state (initialize / configure / get_project)
├── constants.py      # collection IDs, band maps, scale factors, mask thresholds, texture defaults
├── compositing.py    # single, per-period (annual/monthly) and seasonal composites
├── io.py             # exports to Drive / EE assets, asset uploads, task status helpers
├── utils.py          # date windows/validation, clipping, min/max, joins, temporal reducers, resampling
├── vectors.py        # local vector files -> ee.Geometry / ee.Feature / site FeatureCollections
├── zonal.py          # per-region reductions, timeseries, sampling and histograms
├── workflows.py      # end-to-end site timeseries pipelines (annual/monthly and seasonal)
├── sensors/          # per-sensor masking.py + preprocessing.py, plus indices.py and texture.py
├── landtrendr/       # LandTrendr temporal-segmentation method (collection/segmentation/outputs)
└── visualization/    # plots.py, tables.py, vis_params.py

Development

uv run ruff check src/ tests/      # lint
uv run black --check src/ tests/   # format check
uv run mypy src/                   # type check
uv run pytest                      # full test suite

Testing

Tests are split by pytest marker:

Command What runs
uv run pytest -m "not ee" Pure/mock tests only — fast, no credentials needed (this is what CI runs)
uv run pytest -m "not slow" Adds synthetic-image Earth Engine tests; skips real-dataset network tests
uv run pytest Everything

Earth-Engine-marked tests need an authenticated session. They read the project from EE_PROJECT (or GOOGLE_CLOUD_PROJECT) and skip cleanly when no session is available, so the pure tests still run anywhere:

EE_PROJECT=your-project uv run pytest          # bash
$env:EE_PROJECT="your-project"; uv run pytest  # PowerShell

Continuous integration (.github/workflows/ci.yml) runs ruff, black --check, mypy, and pytest -m "not ee" on Python 3.11 and 3.12.

About

A Python package containing utilities used with Google Earth Engine for environmental remote sensing analysis

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages