Apr 11, 09:08 UTC
All affected cloud servers have been brought back online. We sincerely apologise for the long recovery period and impact this has had on our users. Please reach out to support if you have any further questions.
We will be updating users with a post mortem on the incident.
Apr 11, 03:42 UTC
We have brought back 95% of the affected cloud servers in FI-HEL1 and continuing work on the remaining, do reach out to support if you have any questions.
Apr 11, 02:35 UTC
API is fully operational now.
We are continuing to roll out fixes for FI-HEL1, do reach out to support if you have any questions.
Apr 11, 01:01 UTC
We brought back into operational status almost all of the affected cloud servers in US-SJO1. In total, we have recovered almost 40% of affected cloud servers in both locations. We understand the significant impact this has for our users and are doing our utmost to bring back all cloud servers as soon as possible.
If cloud servers which are operational (green status) in the control panel continue to face issues, we advice our users to log in through VNC or console access to resolve any potential OS-level problems due to the sudden error state received.
We will continue to communicate the progress here.
Apr 10, 21:09 UTC
We are progressing with the recovery efforts. Many of the affected cloud servers have already been restarted. We continue relentlessly with the work and will keep you updated here. You are always welcome to reach out to our support in case you require assistance.
Apr 10, 19:37 UTC
The root cause has been identified and we are working to get the affected cloud servers online.
To make sure we get all affected cloud servers back online as soon as possible, we are required to restart the affected host machines. This might in some cases also affect operational cloud servers to speed up recovery of the infrastructure.
Apr 10, 18:49 UTC
We are still working on the issue. Affected customer servers in FI-HEL1 and US-SJO1 will be restarted and returned to running state.
Apr 10, 17:37 UTC
The incident has caused some VMs to stop temporarily. We are working to restore these in working condition.
Apr 10, 16:54 UTC
We are aware of and currently working on fixing issues with API and deployments in all zones. We will update as soon as resolved.
Apr 10, 16:42 UTC