An emergency maintenance will be carried out Thursday, October 16th 2014 at 4:30PM CEST on the Gandimail platform.
This maintenance will likely cause some email mailboxes to be unavailable during this time. Please accept our apologies for any inconvenience this emergency maintenance may cause.
5:55 CEST : End of the gandimail emergency maintenance
There is currently a problem with BaseKit, which GandiSite is based on.
Consequently, websites using GandiSite are offline. (SiteMaker sites are not affected.)
Our teams are in the process of investigating the problem. Updates will be provided here.
Update Wed 15 Oct 20:00 UTC (1pm PDT): Everything should be back to normal. Please let us know if you encounter any further issues.
We just suffered a major incident at one of our facilities. A faulty processor caused the shutdown of a storage unit.
As communications to the disk were interrupted, all operations (reboots, changes, etc) were suspended.
We restarted the unit, and all the services have begun recovering. Operations were queed and are being executed once again. No data was lost. Everything should be returning to normal.
This incident started at
16:19 CEST (07:19 Pacific time). The system was recovered at 16:57, and all queed operations were fully resolved at 17:25 CEST (08:25 Pacific time).
We do apologise for this interruption in service.
As a reminder, you can see the status of our services here:
You can also follow our twiiter feed from the Gandi Noc at @gandinoc.
This news feed is available at: https://www.gandi.net/news
Following an incident on a storage unit, it has been necessary to reboot it in order to complete an update necessary for fixing the problem.
All operations will be paused until the unit is running normally again.
In the meantime, please do NOT launch any operation on your server(s). The situation will return to normal shortly.
20:00 CEST, 11:00 Pacific: Incident officially resolved, all operations back to normal.
An incident has occurred on one of our storage units in the Parisian datacenter. Our technical team is working to resolve the issue as quickly as possible.
Please do not perform any operations on your virtual machines in the meantime. Services should be restored automatically once the issue has been corrected.
We will update this post as new information arises.
Update Tue Oct 7 19:28:19 UTC: Some faulty hardware has been identified; we're in the process of swapping it out.
Update Tue Oct 7 22:33:14 UTC: Our technical team is still trying to fix the issue.
Update Tue Oct 7 23:35:44 UTC: A ZIL disk has failed, and its failover also failed. We're currently performing a manual switchover, and are proceeding very carefully to minimize the risk of data loss.
Most importantly: we understand how disruptive this is for you and we're working as hard as we can to fix it. We will do our best to make it right.
Update WedOur technical team is bringing the storage unit back up. The incident is nearly resolved and services are already beginning to come back online. Oct 8 00:39:21 UTC:
Update 02:31:10 UTC: We're now seeing high loads on the problematic filer. The investigation continues!
Update 04:05:54 UTC: After working all night, our technical team in Paris has resolved the problem. Services should now be back to normal.
A postmortem and compensation details, as described in our IaaS Hosting Contract (section 2.2) will be provided in the days to come.
Update Thu Oct 9 17:31:34 UTC: A postmortem about this incident is available here.
We will be proceeding with some maintenance on a Gandi Mail storage unit. The window for this maintenance will be from Tuesday October 7th 2014 from 11:30 PM to midnight CET (Paris time). There will be several thousand Gandimail mailboxes that will be inaccessible for several minutes during that time. No mails will be lost during this time, they will be held awaiting delivery.
[EDIT] The maintenance is postponed to the 8th of October 2014 from 11:30 PM to midnight CET (Paris time).
We will reboot a storage unit on the Paris/FR datacenter tonight.
The maintenance window will start
3 October at midnight and end at 1am CEST (3-4pm PDT, 22:00-23:00 UTC) . Update: the maintenance window has been extended by 30 minutes and is expected to end at 1:30am CEST (4:30pm PDT, 23:30 UTC)
You will not need to reboot your server (IaaS) or instance (PaaS) during this maintenance.
Sorry for the inconvenience.
Update : end of the maintenance at 2AM CEST, sorry for the delay.
Following a network incident that took place in our offices based in Luxembourg, our international team was unable for a couple of hours to address customer requests as usual.
We have re-distributed these requests to the Paris, Taiwan and San Francisco teams. The surplus load is now being handled and we are confident that the answer time will get back to normal within the next days. Please accept our apologies for this temporary delay.
We will operate an emergency maintenance today, 25 July, at 3pm PDT (22:00 UTC).
This maintenance will have a limited impact on the website for a few minutes.
Sorry for the inconvenience.
We will need to reboot a storage unit as an emergency.
Indeed, a bug is at the source of this emergency maintenance.
This maintenance will not impact the data hosted on this storage unit.
The disks will take their I/O back where they stopped.
Thank you not to do any operation on your VM in the meantime.
The hosting operations will be stopped during the maintenance.
Sorry for the inconvenience this emergency maintenance may cause to you.