20 Dec 2020

Strava: Export all activities to JSON file

In my continued playing around with the Strava API, I wanted to write a script to download all of my Strava activities to a JSON file.

As I mentioned in a previous blog post, the approach to authenticating requests has changed in the last two years, so we first need to generate an access token via the OAuth endpoint. Luckily Odd Eirik Igland shared a script showing how to solve most of the problem, and I’ve adapted it to do what I want.

Pre-requisites

First we need to install the following libraries:

pip install stravalib fastapi uvicorn jsonlines

And the script that we’re going to execute next relies on the CLIENT_ID and CLIENT_SECRET environment variables. We can get those values from the Strava API application page, and set them by running the following commands:

export CLIENT_ID="client_id"
export CLIENT_SECRET="client_secret"

Generate access token

Now we’re ready to generate a Strava access token, using the following script:

authenticate.py

import os
import pickle

from fastapi import FastAPI
from fastapi.responses import RedirectResponse
from stravalib.client import Client

CLIENT_ID = os.getenv("CLIENT_ID")
CLIENT_SECRET = os.getenv("CLIENT_SECRET")
REDIRECT_URL = 'http://localhost:8000/authorized'

app = FastAPI()
client = Client()


def save_object(obj, filename):
    with open(filename, 'wb') as output:  # Overwrites any existing file.
        pickle.dump(obj, output, pickle.HIGHEST_PROTOCOL)


@app.get("/")
def read_root():
    authorize_url = client.authorization_url(client_id=CLIENT_ID, redirect_uri=REDIRECT_URL)
    return RedirectResponse(authorize_url)


@app.get("/authorized/")
def get_code(state=None, code=None, scope=None):
    token_response = client.exchange_code_for_token(client_id=CLIENT_ID, client_secret=CLIENT_SECRET, code=code)
    access_token = token_response['access_token']
    refresh_token = token_response['refresh_token']
    expires_at = token_response['expires_at']
    client.access_token = access_token
    client.refresh_token = refresh_token
    client.token_expires_at = expires_at
    save_object(client, 'auth/client.pkl')
    return {"state": state, "code": code, "scope": scope}

If we run the following command, it will launch a web server on port 8000:

uvicorn authenticate:app --reload

Output

INFO:     Uvicorn running on http://127.0.0.1:8000 (Press CTRL+C to quit)
INFO:     Started reloader process [850840] using statreload
INFO:     Started server process [850842]
INFO:     Waiting for application startup.
INFO:     Application startup complete.

Next we need to navigate to http://localhost:8000, from where we’ll be redirected to the Strava login page:

Figure 1. Strava Authorisation

Once we click on 'Authorize', the web browser will redirect to http://localhost:8000/authorized, and we’ll see something like the following:

{
"state": "",
"code": "<our-code>",
"scope": "read,activity:read"
}

The script will also save a Strava client containing an access token to auth/client.pkl.

Download Strava activities

We’re now ready to download our Strava activites, which we’ll do using the following script:

download_activities.py

import os
import pickle
import time

import jsonlines

CLIENT_ID = os.getenv("CLIENT_ID")
CLIENT_SECRET = os.getenv("CLIENT_SECRET")


def check_token():
    if time.time() > client.token_expires_at:
        refresh_response = client.refresh_access_token(client_id=CLIENT_ID, client_secret=CLIENT_SECRET,
                                                       refresh_token=client.refresh_token)
        access_token = refresh_response['access_token']
        refresh_token = refresh_response['refresh_token']
        expires_at = refresh_response['expires_at']
        client.access_token = access_token
        client.refresh_token = refresh_token
        client.token_expires_at = expires_at


def load_object(filename):
    with open(filename, 'rb') as input:
        loaded_object = pickle.load(input)
        return loaded_object


try:
    client = load_object('auth/client.pkl')
    check_token()
    athlete = client.get_athlete()
    print("For {id}, I now have an access token {token}".format(id=athlete.id, token=client.access_token))

    with jsonlines.open('data/activities-all.json', mode='w') as writer:
        for activity in client.get_activities():
            writer.write({
                "id": activity.id,
                "distance": activity.distance.get_num(),
                "moving_time": activity.moving_time.seconds,
                "elapsed_time": activity.elapsed_time.seconds,
                "total_elevation_gain": activity.total_elevation_gain.get_num(),
                "elev_high": activity.elev_high,
                "elev_low": activity.elev_low,
                "average_speed": activity.average_speed.get_num(),
                "max_speed": activity.max_speed.get_num(),
                "average_heartrate": activity.average_heartrate,
                "max_heartrate": activity.max_heartrate,
                "start_date": str(activity.start_date)
            })

except FileNotFoundError:
    print(
        "No access token stored yet, run uvicorn authenticate:app --reload and visit http://localhost:8000/ to get it")
    print("After visiting that url, a pickle file is stored, run this file again to download your activities")

This script does the following:

Loads the Strava client object from auth/client.pkl
Uses the refresh token to generate a new access token if the old one has expired
Iterates through all of our activities and writes them to data/activities-all.json

We can run the script like so:

python download_activities.py

After it’s finished running, data/activities-all.json has the following contents:

head -n3 data/activities-all.json

Output

{"id": 4489569395, "distance": 13751.9, "moving_time": 4150, "elapsed_time": 4155, "total_elevation_gain": 96.9, "elev_high": 79.3, "elev_low": 25.0, "average_speed": 3.314, "max_speed": 4.6, "average_heartrate": 149.8, "max_heartrate": 176, "start_date": "2020-12-19 07:05:51+00:00"}
{"id": 4485841116, "distance": 12881.6, "moving_time": 3890, "elapsed_time": 3946, "total_elevation_gain": 90.7, "elev_high": 74.3, "elev_low": 22.9, "average_speed": 3.311, "max_speed": 4.4, "average_heartrate": 155.0, "max_heartrate": 191, "start_date": "2020-12-18 05:35:08+00:00"}
{"id": 4478032915, "distance": 13148.8, "moving_time": 3998, "elapsed_time": 4039, "total_elevation_gain": 100.4, "elev_high": 69.9, "elev_low": 22.9, "average_speed": 3.289, "max_speed": 5.3, "average_heartrate": 156.6, "max_heartrate": 183, "start_date": "2020-12-16 05:35:09+00:00"}

About the author

I'm currently working on short form content at ClickHouse. I publish short 5 minute videos showing how to solve data problems on YouTube @LearnDataWithMark. I previously worked on graph analytics at Neo4j, where I also co-authored the O'Reilly Graph Algorithms Book with Amy Hodler.