February 2017

High load incident
We experienced high load due to a corrupt booking which downed 2 servers in our clusters.
Feb 2, 09:48 UTC

January 2017

DNS name servers for bokun.is are down
The bokun.is domain has now been successfully moved and is now on Amazon Route53 DNS servers.
Jan 24, 22:42-23:24 UTC
Expired SSL certificate
SSL certificates have now been upgraded on all environments, and all systems are fully operational.
Jan 15, 21:20-22:35 UTC
Service disruption payment service not responding
Outage resolved by Github
Jan 13, 16:31-16:45 UTC
[Scheduled] System update
The scheduled maintenance has been completed.
Jan 5, 15:46 - Jan 6, 10:47 UTC

December 2016

Degraded performance due to service outage at Mixpanel
This incident has been resolved.
Dec 15, 10:01-10:14 UTC

November 2016

Search engine outage
A search engine server outage caused the extranet to go down from 16:18 to 16:23. Causes are being investigated further.
Nov 24, 17:29 UTC

October 2016

[Scheduled] System upgrade
The scheduled maintenance has been completed.
Oct 28, 10:00-13:39 UTC
Database server issues
Some connections were getting write errors from 10:52 to 11:12. Database server rebooted, service restored.
Oct 24, 11:08-11:19 UTC
Disk failure on monitoring cluster
Disk on monitoring cluster has been replaced. Issue resolved.
Oct 18, 14:30-14:31 UTC

September 2016

[Scheduled] System update
The scheduled maintenance has been completed.
Sep 27, 17:45-18:15 UTC

August 2016

[Scheduled] System update
The scheduled maintenance has been completed.
Aug 8, 10:30-11:30 UTC

July 2016

Heavy traffic causing degraded performance
Performance is back to normal.
Jul 5, 13:11-14:52 UTC

June 2016

Booking problems
Problem has been resolved, system is back to being fully functional.
Jun 10, 17:32-17:37 UTC
[Scheduled] Major system upgrade
All done!
Jun 9, 12:06-19:15 UTC

May 2016

Corrupt cache component
We are seeing sporadic errors for users in the last 30 minutes or so. We had a problem with a corrupt cache component which was affecting all users. This problem has now been resolved.
May 2, 12:15 UTC

April 2016

Extranet outage
Corrupt terraform upgrade script caused a TCP port to be closed, blocking all SSL traffic.
Apr 25