Skip to content

Fail a video that none of the given geotag sources can read - #868

Merged
caglarpir merged 6 commits into
mapillary:mainfrom
caglarpir:report-videos-no-source-can-read
Oct 22, 2026
Merged

caglarpir merged 6 commits into
mapillary:mainfrom
caglarpir:report-videos-no-source-can-read

Conversation

@caglarpir

Copy link
Copy Markdown
Contributor

Stacked on #867; review the last commit.

Problem

Sources such as exif and the legacy gopro, blackvue and camm sources geotag images only. A video given only such sources was left out of the description file and process exited 0:

mapillary_tools process DIR --geotag_source exif   # reports the images of DIR, nothing about its videos

Fix

Report such a video as MapillaryVideoGPSNotFoundError, naming the sources that cannot geotag videos ("The geotag source exif cannot geotag videos"). A video that another source geotags, or fails to, is reported as before. The legacy sources still geotag images from a video as before.

Verification

  • A folder with a GoPro JPG and three videos with --geotag_source exif reports all four files (was the JPG only).
  • Default process output is unchanged on sample folders and videos.
  • New unit tests and a doctest. Unit and integration tests, doctests, mypy --python-version 3.9, ruff check, ruff format --check and usort diff are clean.

Note: the README lists video, camm, gopro and blackvue as --video_geotag_source values, but no video reader is built for the last three (and video is no source type). They used to give nothing; they now get this error. Making them work is a separate change.

A misspelled source type, as in --geotag_source foo or gpx,foo, failed
with a ValueError traceback ("'foo' is not a valid SourceType"), and a
JSON source with a key the schema does not know, such as

    {"source": "gpx", "path": "track.gpx"}

failed with a jsonschema ValidationError traceback. Both exited 1, as for
a crash.

They now raise MapillaryBadParameterError (exit code 2), naming the
supported source types and their aliases, or what the schema rejected
("'path' was unexpected"). This covers --geotag_source and
--video_geotag_source in process and video_process, which parse them the
same way.
…r image

A truncated GPX file, or a file that is no GPX at all, given as the GPX
to geotag images from failed the whole run with a RuntimeError traceback
("Error parsing GPX ...: GPXXMLSyntaxException"). A video geotagged from
the same file gets a MapillaryInvalidGPXError in its description. A GPX
path that does not exist failed with a RuntimeError too, and a GPX file
with no timestamped point with an AssertionError. An NMEA file failed
with a traceback in each of these cases, and when a sentence had no time
or date (TypeError), such as the void ones a receiver writes before its
first fix, or when the file had no RMC sentence to give the date
(UnboundLocalError).

Every image now gets a MapillaryInvalidGPXError in the description,
saying what is wrong with the file, and the run exits as for any other
file that fails to process. The file was given to geotag the images
from, so a mistyped path fails them, as it did, rather than leaving
them to the next geotag source without a word.
…is given

The README says that --video_geotag_source applies to videos and
--geotag_source to images. But the default sources were used only when
neither option was given, so with --video_geotag_source alone, images
had no source at all, and went missing from the description file
without a word, as if they had not been there. That includes the frames
video_process samples: `video_process VIDEO --video_geotag_source native`
sampled the frames, then wrote an empty description file and exited 0.

Geotag images by the default sources when only --video_geotag_source is
given, as when neither option is. The videos are still geotagged by
--video_geotag_source alone.
GeotagImagesFromVideo geotagged every sample by the track of every
video it was given, since the samples moved from its constructor to
to_description(). With the samples of two videos, each sample came out
twice, the second time outside the other video's track, and the check
that it returns no more images than it was given failed:
`process SAMPLES --geotag_source gopro_videos --geotag_source_path DIR`
crashed with an AssertionError for a DIR of two videos.

Geotag the samples of each video by its own track, as before.
…rames

A pattern names the XML of a video after the video:

    video_process VIDEO OUT --geotag_source '{"source": "exiftool_xml", "pattern": "%g.xml"}'

sampled VIDEO by its XML, then failed every frame with "Cannot find the
image in the ExifTool XML". To geotag the frames, video_process gives
every source the video import path, which overrides a pattern, so the
video itself was read as the XML. Without it, the pattern would have
been resolved against each frame, which has no XML of its own.

Resolve the pattern against the videos that the frames were taken from,
found in the video import path, for the videos whose XML it names. The
other frames, and images, are geotagged from the ExifTool XML as before.
Sources such as exif and the legacy gopro, blackvue and camm sources
geotag images only. A video given only such sources was left out of the
description file, as if it had not been there, and process exited 0:
`process DIR --geotag_source exif` reported the images of DIR, and
nothing about its videos.

Report such a video as MapillaryVideoGPSNotFoundError, naming the sources
that cannot geotag videos. A video that another source geotags, or fails
to, is reported as before.
@meta-cla meta-cla Bot added the cla signed label Oct 22, 2026
@caglarpir
caglarpir merged commit 50660bf into mapillary:main Oct 22, 2026
16 checks passed
caglarpir added a commit that referenced this pull request Oct 23, 2026
Two ways a file given to process could be left out of the results, as
if it had not been there, with exit 0:

#875 left a video out when --filetypes named types of video (gopro,
blackvue, camm) and none of the geotag sources given could read videos:
the type of a video is known only once it is read, so it might not be of
any of them. That brought back the silent drop that #868 fixed. On a
GoPro video, `process VIDEO --filetypes gopro --geotag_source exif`
exited 0 with an empty description.

A file could also be given to no source at all, when the filetypes of
every source leave it out. `--geotag_source '{"source": "native",
"filetypes": ["image"]}'` gave the videos of a folder to no source, and a
source limited to GoPro videos its images. --filetypes limits the
filetypes of every source further: `--filetypes gopro
--video_geotag_source '{"source": "native", "filetypes": ["blackvue"]}'`
gave a GoPro video to no source.

Fail such a file, as #868 fails a video that no source can read: "No
geotag source is given for this video: the geotag source native is
limited to other file types", or "--filetypes gopro leaves no geotag
source for this video: the geotag sources native, exiftool_runtime are
limited to other file types". A file of a type that --filetypes leaves
out is still left out. For a video, which --filetypes names by the type
of its GPS, that is told as the native reader does while geotagging:
given the types asked for, it recognizes one of them in the video, or
none. A video of none of them is left out, as #875 leaves out the videos
that a source found to be of other types, and any other is failed. No
video is read for it without --filetypes or with --filetypes video.

Checked process on a folder of a GoPro, a CAMM, a BlackVue and an
unrecognized video and an image, and on the GoPro video alone: with no
--filetypes, and with gopro, camm, blackvue, video, image, image,gopro,
gopro,camm,blackvue and camm,video; each with the default sources, exif,
native, gpx, --video_geotag_source native, exif and --video_geotag_source
native, --video_geotag_source gpx, and exif and a native source limited
to camm (144 runs). Only the 13 runs in which exif was the only source
given videos change: the videos of the types asked for are failed, and
the others are still left out. Nor do the flows whose sources are
limited by design change: process with the default sources, with exif,
with --video_geotag_source native, gpx or exiftool_runtime alone, with
exiftool_runtime, exiftool_xml, gpx or NMEA, with the image-only gopro,
camm and blackvue sources on sampled frames, alone or beside their
videos, and video_process with the default sources, exif,
--video_geotag_source native, gpx, exiftool_xml and the gopro source.
video_process with only a gpx source limited to videos, whose frames no
source geotagged, now fails them rather than exit 0 with an empty
description.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant