Incident Summary

A problem with the grand-challenge backend prevents reader study cases to load in common scenarios for some users.

Friday evening: After an initial analysis of the issue it seems that it is contained to only a few users of the platform. However, for those users it seems to be a persistent problem which needs further investigation. Right now, as evident from the application log files, only developers seem to have observed the problem. Therefore, we lowered the severity from major to minor because it does not seem that users are directly affected at this moment.

Monday: We have fully tracked down the issue to a code change that added an alternative way of counting cases in a reader study. The issue affected users for grand-challenge who:

1. are editors of a reader study,
2. are readers of that same reader study,
3. had saved annotations,
4. and were using regular, non-admin accounts.

We reverted the change which resolved the issue. Reader studies are fully operational again.

Timeline

Incident Summary

We experienced a viewer session outage affecting all viewing of items on the platform. This included reader studies, archive items, algorithm-job results, and others.

Timeline

Incident Summary

We experienced a viewer session outage affecting all viewing of items on the platform. This included reader studies, archive items, algorithm-job results, and others.

Timeline

Incident Summary

We are experiencing issues with our queues. As a result, algorithms currently cannot be queued for execution and emails can't be sent. We are working on fixing the issue.

Timeline

Incident Summary

Image and container uploads currently get stuck in "initializing" state due to an issue in our uploads backend. We are working on resolving the issue.

Timeline