Grand Challenge Status

This page shows the status of components of grand-challenge.org.

Incident history

Grand Challenge cannot start evaluations, algorithms, or new viewer sessions

Started 05 August 2026 15:55
  • Viewers
  • Algorithm jobs
  • Challenge evaluations
  • Image handling

Incident Summary

Grand Challenge was not able to start new processing jobs of any kind due to a problem in our job queue caused by a temporary database connection issue. A bug fix was applied and the issue is mitigated.

Timeline

05 August 2026 15:55Issue started, no new AWS Lambda tasks can start.
05 August 2026 17:30Issue was detected
05 August 2026 17:55Manual intervention
05 August 2026 19:03Task queues recovered; bugfix applied.
Resolved 05 August 2026 19:08

Grand Challenge unreachable due to CloudFront VPC Origins outage

Started 16 July 2026 09:51
  • Api

Incident Summary

Grand Challenge was completely unreachable for approximately 51 minutes. The outage began at the same time as a routine deployment, which initially led us to suspect our own changes. Investigation ruled out our infrastructure, and we mitigated the issue by switching the main API from a CloudFront VPC origin to a global CloudFront origin. It later turned out to be an unrelated, unannounced AWS outage affecting CloudFront VPC Origins connectivity.

Timeline

16 July 2026 09:51Deployment applied; 5xx error rate for the API began increasing
16 July 2026 09:57Loss of access to Grand Challenge noticed and investigation started
16 July 2026 10:35Main API switched from CloudFront VPC origin to a global CloudFront origin
16 July 2026 10:42Grand Challenge back online
Resolved 16 July 2026 10:42

Reader studies do not load

Started 20 June 2025 16:00
  • Viewers

Incident Summary

A problem with the grand-challenge backend prevents reader study cases to load in common scenarios for some users.

Friday evening: After an initial analysis of the issue it seems that it is contained to only a few users of the platform. However, for those users it seems to be a persistent problem which needs further investigation. Right now, as evident from the application log files, only developers seem to have observed the problem. Therefore, we lowered the severity from major to minor because it does not seem that users are directly affected at this moment.

Monday: We have fully tracked down the issue to a code change that added an alternative way of counting cases in a reader study. The issue affected users for grand-challenge who:

  1. are editors of a reader study,
  2. are readers of that same reader study,
  3. had saved annotations,
  4. and were using regular, non-admin accounts.

We reverted the change which resolved the issue. Reader studies are fully operational again.

Timeline

19 June 2025 14:52Incident started
20 June 2025 16:00Issue identified
20 June 2025 18:00Severity lowered to minor
23 June 2025 15:15Issue resolved
Resolved 23 June 2025 15:15

Broken Viewer Sessions

Started 03 June 2025 20:10
  • Viewers

Incident Summary

We experienced a viewer session outage affecting all viewing of items on the platform. This included reader studies, archive items, algorithm-job results, and others.

Timeline

03 June 2025 20:10Incident started
04 June 2025 09:00Issue identified
04 June 2025 11:20Issue resolved
Resolved 04 June 2025 11:20

Broken Viewer Sessions

Started 30 November 2024 20:00
  • Viewers

Incident Summary

We experienced a viewer session outage affecting all viewing of items on the platform. This included reader studies, archive items, algorithm-job results, and others.

Timeline

02 December 2024 08:45Incident started
02 December 2024 09:30Issue identified
02 December 2024 10:05Issue Resolved
Resolved 02 December 2024 10:05