Repository navigation
Read each ExifTool GPS record of a video on its own - #876
Merged
Merged
Conversation
The README lists NMEA among the sources that geotag videos, but no video was read from an NMEA file: process and video_process gave the video no description at all, and exited 0, so it was silently left out. Sampling fell back to the GPS in the video, which a video recorded with an external logger does not have. Read the NMEA file into a track and place it on the video exactly as a GPX track is: synced by the GPS clock of the video, or by --video_start_time, or assumed to start with the video, with the interpolation options, the coverage checks and the clipping of process. GPXVideoExtractor now reads its file through _parse_tracks(), which NMEAVideoExtractor overrides, and names the format of the file in its messages. A file that is not valid NMEA is reported as an invalid GPX file is. Interval sampling checks an NMEA track started with the video by --interpolation_use_gpx_start_time as it checks a GPX track. On an action camera video with a GPX and the same track as NMEA, process gives the same track, and video_process samples the same frames at the same positions.
Images geotagged from an NMEA file failed with a traceback when the file had an RMC sentence without a date, such as the last one of a log cut short (TypeError), or GGA sentences and no RMC sentence at all (UnboundLocalError), and a video was failed as if the file were not NMEA. A void RMC sentence, which receivers log before they acquire a fix, gave the date of their unset clock, and a GGA sentence without a fix added a point at 0, 0. Only $GP sentences were read, so a log of a receiver of several systems, which writes $GN sentences, gave no points at all. Skip RMC sentences that are void or have no date or time, and GGA sentences without a time or a fix, and read the sentences of any talker, keeping the first fix of each time, so that a receiver that logs a fix from two talkers does not give two points at the same time. GGA sentences without the date of any RMC sentence give no points, with a warning. The points read from valid files are the same.
process searches a directory for the images and videos it can process, and skips the other files. It skipped a file named explicitly the same way, without a word: `process VID.insv` processed 0 files and exited 0, with nothing on stderr, so a user could not tell why the video was not there. Warn of such a file. It is still skipped, and the run does not fail: callers that pass the files a user selected rely on that, and find the skipped files in the process report (--process_report_path). A zip file is not warned of, as process_and_upload uploads it.
video_process samples each video into a folder of its own in the import path, and then processes the import path with the same options. With --skip_subfolders, the processing step skipped those folders: it found no samples, wrote an empty description file and exited 0, and video_process_and_upload uploaded nothing. Process, and upload, the sample folders of the videos as import paths of their own. --skip_subfolders still keeps the other subfolders of the import path out, and the description file is written to the import path, as without the option.
For a video without a GPS clock, process places a GPX track at the start of the video: it assumes the track starts with it, or --interpolation_use_gpx_start_time says so. --interpolation_offset_time then moves the track back, before the start of the video. The upload drops the points before video time 0 without putting one at 0, so the first frames had no position: with an offset of 2.5 seconds and a point every second, the first half second of the video. Clip such a track at the start of the video, with a point interpolated at video time 0, as a track that a clock places is clipped at both ends. Its end is left as it is, as the track of a timelapse without a GPS clock runs on past the end of the video. Sampling, which reads its track only within the video, is unchanged. The warning of a track that runs on for more than twice the video said, with an offset too, that the video was assumed to start with the track. With an offset, it now says where along the track the offset places the start of the video, and lists checking the offset next to trimming the GPX and video_process --video_start_time. Without one, it is unchanged.
A GPX file can hold several trips, with the GPS logger switched off in between. A video or an image recorded during such a gap was given a position interpolated between the last point before it and the first after it, which can be hours and kilometres apart. A dashcam clip placed by --video_start_time in a 14-hour gap of a six-day GPX was sampled and geotagged as if the track covered it: the coverage checks compared the video or image only with the ends of the track. Consecutive points more than an hour apart now leave a gap that counts as outside the track, like the time before its start or after its end. ExifTool's geotagger likewise treats a time in a gap of more than GeoMaxIntSecs (30 minutes by default) as outside the track. - An image (or a sampled frame geotagged from the GPX) in a gap fails with MapillaryOutsideGPXTrackError, so a later geotag source can take it, as it does an image past either end of the track. - A video that falls in a gap misses the track, and raises or warns as one that misses it before or after does. One that falls in a gap only in part warns that part of it is outside the track. Both errors name the gap: its length, and the time and position of the last point before it and the first after it. For a timelapse the gap is measured in GPS time, as the other checks are. The video's own GPS track, which the legacy video sources geotag images from, is not checked. The limit can be raised with MAPILLARY_TOOLS_MAX_GPX_GAP_SECONDS (3600 by default), like the other limits in constants.py.
--filetypes gopro, blackvue or camm limits process to the videos of those types. The type of a video is known only once it is read, so the native reader tried only the readers of those types, and reported a video none of them recognized as having no GPS. ExifTool, which the default sources fall back to, then read it anyway: a GoPro video with --filetypes blackvue was processed as a GoPro video and exited 0, a BlackVue video with --filetypes gopro as a generic video, and in a folder of mixed cameras, the videos of the other cameras were reported as errors. Leave a video out of the results once a source finds out that it is of none of the types --filetypes asks for, as the files of other kinds are: a source read it as another type, or the native reader, given all of them, recognized none of them in it. No later source reads it. A video read as a generic video is of a type that is not known, and is kept, so a GPX or ExifTool still geotags a video of a type asked for. A source limited further by its own filetypes leaves the video to the next one, as before. A video that the native reader recognizes, but whose GPS is empty or noisy, is reported as before, while one that no source can read is left out: it may not be of any of the types asked for.
ExifToolReadVideo read the QuickTime and Insta360 GPS tags of a video as one flat list per tag, and zipped them into points, padding a short list at its end. A record without a tag shifted every later value of that tag onto the point before it. A BlackVue writes no GPSTrack while it stands still, nor a GPSAltitude in its last record, so the headings and altitudes of the points after them were those of other points. A BlackVue video also holds a copy of its GPS log in the low resolution recording embedded in it, which ExifTool lists as well. The shifted headings kept the two copies of a point from matching, so the track had every point twice. Group the tags by record instead: a record starts at its SampleTime, at its GPSDateTime, which ExifTool lists first in a record, or at a tag that the record so far already has. Each point takes its fields from its own record, and the copies match and merge.
caglarpir
force-pushed
the
exiftool-gps-per-record
branch
from
October 22, 2026 20:18
609d050 to
50ddbea
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #875; review the last commit.
Problem
The ExifTool video reader read the QuickTime and Insta360 GPS tags as one flat list per tag, and zipped them into points, padding a short list at its end. A record without a tag shifted every later value of that tag onto the point before it. A BlackVue writes no GPSTrack while it stands still, nor a GPSAltitude in its last record, so
gave most points another point's heading and altitude. The BlackVue also embeds a copy of its GPS log in its low-resolution recording, which ExifTool lists too; with the shifted headings the copies did not match, and the track had every point twice (120 instead of 60).
Fix
Group the tags by record: a record starts at its SampleTime, at its GPSDateTime (which ExifTool lists first in a record), or at a tag the record already has. Each point takes its fields from its own record, so the two copies match and merge as before.
Verification
exiftool_runtimeandexiftool_xml: 60 points instead of 120, each with its own heading and altitude; positions match the native reader.processis unchanged.mypy --python-version 3.9,ruff check,ruff format --checkandusort diffare clean.