Sources

Using Garmin devices to record my outdoor activities, all tracks are send to Garmin-cloud via the app GarminConnect automatically. This is very convenient and cloud-to-cloud connections forward the tracks to Strava and Komoot. Long before the Bluetooth-Cloud-Automagic-era I already used a Garmin Edge 710 with USB connectivity only. As a Linux user I read out the TCX files from the mass storage, stored it locally and sent it to Garmin years later for completeness.

Archiving at home

But I cannot accept Garmin as a long-term archive for all data of highly personal value. My track archive dates back until 2008 and I do not wont to loose it some day just because a company goes out of business or the like. So digging in the web I found a script to download activities from Garmin as GPX files. For many years this helped me to just download and sync to my own Nextcloud. The smart watches also can record health data and the Body Battery is one of the highlights for me. So the next logical step was to also archive these data at home. The library python-garminconnect offers a complete API to Garmin cloud and to store time series data, InfluxDB seems to be a good choice. So a few lines of glue code later, a new script was born to

  • Download health data and send to my local InfluxDB,
  • Download activities, save as local GPX files and let Nextcloud client sync to my self-hosted Nextcloud.
  • A decent backup strategy (3-2-1) shall help to not loose data.

Visualizing #1

Nextcloud’s plugin Gpxpod can scan directories and visualize all GPX/TCX files on a map. Because all geo data was already synced to Nextcloud, this was a no-brainer and worked very well for many years. But when the plugin architecture changes, it seems that all geo-filtering was done in the frontend (the browser) and in my setup this totally crashed. Selecting a directory with only hundreds of tracks results in too many of track points. Sending these to the browser need time and bandwidth, filtering for the current viewport of the map in the browser needs memory and CPU. So my browser tab gets stuck and then crashed.

Visualizing #2

Recently I stumbled upon Dawarich (German for “I was there”). May that solve my geo-visualization problem? 20 Minutes later I had the Postgres, Dawarich and the Sidekick container up and running (Redis was already available) and imported the first tracks. The visualization and filters are really great. But with a million points it ran into the same problem, maybe in the backend but with the result of being unusable.

Data filtering and simplification

So it seems I have to work on the data. First approach was, to separate files in directories based on their area. Reason: There are far more files from my home region than from other regions e.g. vacation trips. But the later are more interesting. So I added a geo-filter to my script from above:

  • taking the first track point of each file and compare to predefined geo-bounding boxes,
  • put all tracks from “home” into a directory,
  • put all tracks from “fomer home” into a second directory,
  • Put all other tracks in a third directory.

For Nextcloud / Gpxpod this helped to focus on other regions, but rendering was still very slow. My home region was still to large in number of files / track points.

Next step: Sub-sampling. The original tracks have a sampling rate of 1 to a few seconds, resulting in very precise recordings. But for coarse visualization this is not required. Fiddling with gpsbabel on the commandline I decided that filtering according to a cross track error of 10 meters ("-x simplify, crosstrack,error=0.01k") reduces the number of track points to 5 % and only very sharp turns get distorted. The visual quality in Dawarich and Gpxpod is totally sufficient.

To add this step to my sync script I chose the lazy way: Just calling for gpsbabel as an external command.

Summary

Getting all data from Garmin semi-automatically (start the script), filtering tracks geographically by bounding boxes and simplifying to 5 % of data is required to visualize >2000 tracks in Gpxpod and Dawarich. But the result is great: I can see all my activities, even those in my home region interactively on a single map!