We will need to reboot a storage unit as an emergency.

Indeed, a bug is at the source of this emergency maintenance.

This maintenance will not impact the data hosted on this storage unit.

The disks will take their I/O back where they stopped.

Thank you not to do any operation on your VM in the meantime.

The hosting operations will be stopped during the maintenance.

Sorry for the inconvenience this emergency maintenance may cause to you.

 


We will proceed to the migration of the hosting emergency console.

This maintenance will occur on Monday, the 23rd of June 2014 CEST (Paris timezone).

It should take a few minutes, the impact will be the connection of the IaaS (VM) emergency console, the PaaS/Simplehosting instances console, and the IaaS statistics on the web control panel.

Sorry for the inconvenience this maintenance may cause to you.


Our operations team will perform an emergency maintenance on storage units today, Monday 9 June 2014.

This maintenance may impact the following services :
  • www.gandi.net: access to the control panel, orders
  • mail.gandi.net: IMAP, POP, SMTP, webmail
  • Web redirections
  • GandiSite (Basekit) / Sitemaker
  • DNS zone provisioning
  • Gandi Groups / Gandi wiki
  • Emergency console / DNS cache for hosting datacenter in Paris
  • SimpleHosting (PaaS) virtualhost provisioning

The maintenance window will be four hours, and will occur between 2 and 4pm PDT (11pm to 3am CEST/Paris).

We apologize for the inconvenience this maintenance may cause.

 

Updates:

2:00pm PDT [11:00 PM CEST, 21:00 UTC]: start of the maintenance

2:47pm PDT: end of the maintenance


We are currently experiencing a routing issue causing a part of our network to be unreachable from certain locations.

Our technical team is analyzing the problem in order to correct it as quickly as possible.

We will provide updates as the situation evolves.

 

UPDATE 05h15 : The release of a new network configuration disturbed our network. Sorry for the inconvenience this issue may have caused to you.


Following the discovery of an intermittent but serious network issue, Gandi teams have determined that a rolling maintenance will be necessary. 

We regret that, while most of the affected systems will simply require migration and no interruption in service, some will almost certainly require restarting. We will endevor to make these interruptions as short as possible, and only perform them when absolutely necessary. 

We are starting with dc0 (Paris) this week. We will proceed on to dc2 (Luxembourg) on Monday, June 9. 

The issue is not detected on dc1 (Baltimore) at the moment but, if necessary, we will proceed to fix it there.  We apologize for any inconvenience this may cause.  


Here's the incident history:

 

* 08:25 UTC 12 hosting nodes are made inaccessible due to a switch failure. ~200 Virtual machines (VMs) are made unreachable. 

* 08:40 UTC Switches are recovered and VMs are once again accessible. Investigation does not reveal cause of incident. 

* 12:01 UTC A second incident occurs, affecting 8 nodes and ~180 VMs. 

* 12:09 UTC Switches are recovered, VMs are made available again. Additional data collection measures are put in place to help determine cause. 

* 14:56 UTC A third incident occurs, affecting 10 nodes and 321 VMs. 

* 15:10 UTC Nodes and VMs are available again. This time extensive forensic data is made available, and we expect to find the root cause and execute a permanent fix, which will be implemented as soon as possible. 

We do apologise for the inconvenience this issue may have caused. 



Some physical machines hosting IaaS VMs are unreachable due to an issue we are currently analyzing.

We are fixing the issues and restarting the VMs on other physical machines as soon as possible.

Thank you not to do any operation on your VM until the emergency maintenance is ongoing. and not finished.

At the end of the maintenance, if your server does not respond well, please contact our hosting support team by mail using the 'blocked server' option.

 

Sorry for the inconvenience this issue may cause to you.

 

UPDATE 12h20 : the situation is back to normal, we are still analyzing the metrics and the logs.


A maintenance will be performed on our hosting infrastructure at Baltimore (USA) and Bissen (Luxembourg).

The following services will be interrupted for a few minutes starting at 2014-04-29 05h00 UTC (2014-04-29 10pm PST):

  • New SFTP/GIT sessions
  • Reverse resolvers for customer IPs
  • Operations on servers and Simple Hosting

Please accept our apologies for the inconvenience.


An emergency maintenance will be done to improve our storage on Paris datacenter Wednesday 22 April 2014 between 3pm and 9pm PST (2014-04-22 22h00 and 2014-04-23 4h00 UTC).

There may be network interruptions lasting a few seconds per storage unit. The I/O will come back without any operation needed on your side. It is not necessary to reboot your virtual server.

Please accept our apologies for any inconvience caused by this maintenance. This post will be updated when maintenance has been completed.

 

This maintenance has been completed. Some additional issues were uncovered, and the entire maintenance took a little over 2 hours.

We do apologise for the inconvenience.  

 

EDIT Tue Apr 29 23:30:00 CEST 2014 : The maintenance will be completed from 00:00 to 01:00 tonight for the storage units that have not been upgraded last time.


Page 1 2 36 7 8
Change the news ticker size