Skip to content

Strava webhook — automatic activity import

Context

Today Strava import is entirely manual: Settings → Account → Import Strava. After every ride that syncs from Garmin to Strava, the user has to open the app and click a button. The goal is for activities to land in Wattlog on their own.

The complication: EC2 is shut down 23:00–06:00 (terraform/scheduler.tf), while a Strava webhook assumes an endpoint that is always reachable — events sent during the downtime window are lost. Hence the buffer: an always-on receiver on API Gateway + Lambda pushes events into SQS, and the app on EC2 drains the queue on startup and throughout the day.

Choosing a webhook over the simpler sync-on-app-open is deliberate: the ingest → buffer → drain skeleton is meant to be reused for future integrations (Garmin, Wahoo).

Key design decision: the event is a signal, not data. An SQS message only says "user X has something new". The drain then runs the existing cursor-based import_recent_activities — it never parses activity data out of the payload. Consequences:

  • zero new import code; the production path is reused in full,
  • idempotency and collapsing: five events for one user become a single sync,
  • security: Strava does not sign webhook payloads, so the POST endpoint is unauthenticated by design. A forged event can at most trigger a redundant sync for a user who is already connected. No payload data ever reaches the database.

Decisions already made with the user

  • Address: webhooks.wattlog.pro (dedicated subdomain, with future integrations in mind)
  • Event scope: create + deauthorize; activity update/delete ignored

Architecture

Strava ──HTTPS──> API Gateway (webhooks.wattlog.pro) ──> Lambda ──> SQS
                          [always on]                            [buffer, 7-day retention]
                                                                     │
                                              EC2 06:00–23:00 ───────┘
                                              drain (long poll) → import_recent_activities

1. Terraform — new file terraform/webhooks.tf

Repo conventions to follow: names as "${var.project}-<thing>", inline IAM policies (aws_iam_role_policy, one resource per concern — there are no managed-policy attachments anywhere in this project), per-resource tags = { Name = ... }, one file per concern.

Resources:

  • aws_sqs_queue.strava_webhook — 7-day retention (covers the nightly gap and outages with room to spare), visibility_timeout_seconds ~60s. Plus a DLQ and redrive_policy (maxReceiveCount = 5) so a poison message cannot loop forever.
  • aws_lambda_function.strava_webhook — runtime python3.12, no VPC configuration (it only touches SQS, which is a public AWS API). This is deliberate: a VPC would force a NAT Gateway at roughly $32/month, about 600× the cost of everything else here. Environment: QUEUE_URL, VERIFY_TOKEN.
  • data.archive_file zipping terraform/lambda/strava_webhook/ → filename + source_code_hash. Requires adding the archive provider to terraform/main.tf:4-17. There is precedent for keeping code inside Terraform — aws_cloudfront_function uses code = file(...) at landing.tf:45 and tiles.tf:154.
  • aws_iam_role.lambda + aws_iam_role_policy — sqs:SendMessage on the queue plus CloudWatch Logs permissions.
  • aws_apigatewayv2_api (HTTP protocol), AWS_PROXY integration with the Lambda, routes GET /strava and POST /strava, aws_apigatewayv2_stage with auto_deploy, and aws_lambda_permission for API Gateway.
  • Custom domain: aws_acm_certificate for webhooks.${var.domain} + DNS validation + aws_apigatewayv2_domain_name + aws_apigatewayv2_api_mapping. Note: a separate certificate on purpose, rather than appending a SAN to aws_acm_certificate.main (cdn.tf:3-14) — that one is attached to the CloudFront distribution serving production app.wattlog.pro, and replacing it forces a distribution update. A new certificate keeps the blast radius contained. The default region is already us-east-1, so the regional certificate for API Gateway is created without a provider alias (unlike provider = aws.us_east_1 used for CloudFront).
  • aws_route53_record.webhooks in terraform/dns.tf — an A alias to the API Gateway domain (pattern: dns.tf:138-148).
  • EC2 permission: a new aws_iam_role_policy "sqs_consume" on aws_iam_role.ec2 (ec2.tf:3-14) — sqs:ReceiveMessage, sqs:DeleteMessage, sqs:GetQueueAttributes, sqs:GetQueueUrl, sqs:ChangeMessageVisibility.
  • Queue URL output in terraform/outputs.tf.

VERIFY_TOKEN goes into terraform/terraform.tfvars (confirmed untracked by git) as a new variable declared in variables.tf.

2. Lambda code — terraform/lambda/strava_webhook/handler.py

A single small handler with no external dependencies (just boto3, present in the Lambda runtime). It must respond within 2 seconds — hence no VPC and minimal work:

  • GET — subscription handshake: compare hub.verify_token against VERIFY_TOKEN; on match return {"hub.challenge": <value>}, otherwise 403.
  • POST — push the raw body to SQS, return 200. No business validation: event filtering happens in the drain so the handler stays as short as possible.

3. Drain in the app — src/cycling_trainer_hub/services/strava_webhook_drain.py

Structurally mirrors stale_session_watchdog.py, including the split into two functions (a single separately testable sweep plus the loop):

  • drain_once() -> int — one cycle. boto3 is synchronous, so SQS calls go through asyncio.to_thread. Long polling with WaitTimeSeconds=20, MaxNumberOfMessages=10.
  • run_strava_webhook_drain(stop_event) — while not stop_event.is_set() loop, body wrapped in a try/except that logs and continues, interruptible sleep via asyncio.wait_for(stop_event.wait(), timeout=...).

Long polling gives near-instant import during operating hours at effectively no cost (~4,300 receives/day against the 1M permanent SQS free tier).

Sweep logic:

  1. Receive a batch, parse messages, group by owner_id — multiple events for one user collapse into a single sync.
  2. For object_type=activity, aspect_type=create: owner_id → StravaConnection.strava_athlete_id (indexed column, models/auth.py:39) → user_id → AthleteService(session).get_default(user_id) (services/athlete_service.py:73-79; already falls back to the first athlete, returning None only when the user has none) → import_recent_activities(...) → for each imported training await _compute_metrics_for(training_id, athlete_id). _compute_metrics_for (api/routers/strava.py:195-219) is already free of HTTP request context — it builds its own session from get_engine(). Call it directly, unchanged.
  3. For object_type=athlete with updates.authorized == "false" (deauthorize): do not delete immediately. The POST endpoint is unauthenticated, so a forged event could wipe someone's connection. Verify first — make a Strava request with the stored token and delete StravaConnection only if Strava rejects it (401). One extra request closes the hole.
  4. Any other event: delete the message without acting.

Message-deletion error handling:

  • no StravaConnection for owner_id → delete (nothing to sync)
  • no athlete → log and delete
  • httpx.HTTPStatusError from token refresh, i.e. a revoked token (services/strava_client.py:124-152) → delete, unrecoverable
  • rate limit (429) or transient error → do not delete; the message returns after the visibility timeout, and after 5 failed attempts lands in the DLQ

4. Wiring in api/app.py

Follows run_stale_session_watchdog exactly (app.py:120-131 for startup, 135-146 for shutdown): its own asyncio.Event, asyncio.create_task, both objects on app.state, and on shutdown set the event then await asyncio.wait_for(task, timeout=5) inside try/except asyncio.TimeoutError that warns and calls task.cancel().

The task only starts when STRAVA_WEBHOOK_QUEUE_URL is set, so local dev and staging (which has no Strava configuration) are unaffected.

5. Configuration

  • docker-compose.prod.yml (alongside the existing STRAVA_* entries at lines 80-85): STRAVA_WEBHOOK_QUEUE_URL from the Terraform output.
  • No Secrets Manager entries needed — the Lambda receives VERIFY_TOKEN via env, and the app never validates the handshake.

No database migration is required — the signal-based design needs no debounce column, and the last_import_at cursor already exists (models/auth.py:47).

6. Subscription registration — src/cycling_trainer_hub/cli/register_strava_webhook.py

A one-off (and idempotent) CLI tool modelled on cli/purge_deleted_users.py: POST /api/v3/push_subscriptions with client_id, client_secret, callback_url, verify_token. Subcommands --list and --delete for management. Strava allows one subscription per application; staging has no Strava configuration (confirmed — docker-compose.staging.yml contains no STRAVA_* variables), so there is no conflict.

Ordering matters: registration immediately triggers the GET handshake, so API Gateway must already be up.

7. Tests

Following the repo convention (tests/test_strava_promote.py): monkeypatch.setattr(strava_client, "request", _fake_request) with a hand-rolled _Resp (it must expose raise_for_status(), since strava_import_service.py:113 calls it). The SQS client is stubbed the same way.

  • message parsing and event routing — pure functions, no fixtures (pattern: tests/test_strava_import_geometry.py)
  • grouping by owner_id collapses multiple events into one sync
  • deauthorize does not delete the connection while the token still works
  • deauthorize deletes the connection on a 401 response
  • a transient error leaves the message in the queue

Testing note: drain_once opening its own session via async_session_factory would bypass the rollback in the session fixture (tests/conftest.py:84-93). So drain_once takes an optional session parameter — the loop injects its own, tests pass the fixture.

Execution order and verification

  1. Terraform → cd terraform && terraform plan (review carefully: state is local, main.tf:19-27, the S3 backend is commented out) → terraform apply. Verify: curl https://webhooks.wattlog.pro/strava?hub.mode=subscribe&hub.challenge=test&hub.verify_token=<token> returns {"hub.challenge":"test"}; a wrong token returns 403.
  2. Drain code + tests → pytest tests/test_strava_webhook_drain.py (needs a running postgres container — docker compose up -d postgres).
  3. Deploy → ./deploy-to-ec2.sh (rsync + build on the box; terraform/ is excluded from rsync, deploy-to-ec2.sh:127).
  4. Register the subscription → python -m cycling_trainer_hub.cli.register_strava_webhook, then --list to confirm.
  5. End-to-end test → record a short Strava activity (or upload a file) and check: the message appears in SQS (aws sqs get-queue-attributes), drain logs show the sync, and the training is in Wattlog without clicking anything.
  6. Overnight test → upload an activity after 23:00 and confirm the next morning that it imported itself once EC2 came up.

Out of scope

  • Activity update/delete — would need an "update an existing training" path that does not exist today (there is only dedup by strava_activity_id)
  • Generalising the receiver for Garmin/Wahoo — the skeleton is built for it, but the generalisation itself waits for the second integration
  • GPS geometry backfill for earlier imports — separate, already-written work in cli/backfill_strava_geometry.py