A critical security issue in the virtualization software Xen will become public July 26 and the Xen team has already informed Gandi of the necessary patches. 

Since this announcement, we have already preemptively deployed the patches required to correct the issue. We have been monitoring the particular security flaw and have determined we will need to stop/start certain Xen VMs in order to assure that no further possible attack vector will remain.

We will be contacting the affected customers directly in order to allow them to sufficiently prepare for this stop/start and those of you who have not received any message from us are therefore not affected.

In order to minimize downtime and also to help minimize the impact in general, we would advise all affected to schedule a stop/start of their platforms yourselves sometime between now and the cutoff date of July 26, 2016.

Any affected VMs that you have not yet stopped and started again by 12:00 AM PDT July 26, 2016 (07:00 UTC), we will stop/start at some point between then and July 28 at 9:00 AM PDT (16:00 UTC). Please expect around 30 minutes of downtime per stop/start.

As always, if you have any questions or have any difficulties, please do not hesitate to contact our Customer care team.

Edit 7/21/16: Previously we used the term "reboot" instead of "stop/start." Rebooting isn't sufficient to apply the security patch. Your VM(s) need to be stopped and then started again in order for the patch to take effect.


ImageMagick announced a security vulnerability, registered as CVE-2016-3714, that allows malicious users to craft filenames to execute code remotely.

We have applied the appropriate fixes on Simple Hosting to protect customer applications using ImageMagick libraries.

If you're using ImageMagick in your application, make sure you restart your instance after 18:00 UTC (11:00 AM PDT) on May 4, 2016 in order to apply the patch.

You can restart your instance from the website or from the terminal with Gandi CLI:

$ gandi paas restart {instance_name}

Please don't hesitate to contact Customer Care if you experience any issues or have any questions related to this topic.


Following this announcement we applied the necessary patches, thus reinforcing the existing security measures we had previously implemented. Over the past week, we have continued to study the vulnerability. As a preventative measure, we have decided that a reboot of KVM based VMs is required in order to ensure that all possible attack vectors have been mitigated.

We will contact affected customers directly via email to provide instructions on performing the reboot on their own. This preventive reboot will not affect customers we do not contact.

We will reboot the VMs of affected customers (who have not rebooted on their own) on November 19th. An outage of 30 minutes maximum is expected for each impacted VM.

If you have questions or encounter any problems regarding this issue, our support team is available to assist you.

A new critical security vulnerability will be publicly announced Thursday, October 29. The Xen team has already communicated fixes to Gandi. This flaw is found in the Xen virtualization software.

Following this announcement we applied the necessary patches, thus reinforcing the existing security measures we had previously implemented. Over the past week, we have continued to study the vulnerability. As a preventive measure, we have decided that a reboot of Xen-based VMs is required in order to ensure that all possible attack vectors have been mitigated.

We will contact affected customers directly via email to provide instructions on performing the reboot on their own. This preventive reboot will not affect customers we did not contact
We strongly recommend that customers concerned by this to restart their VMs themselves, in order for them to verify that all of their services have been correctly restarted.

We will reboot the VMs of affected customers (those which were not rebooted by their owner) from Thursday, October 22 until Wednesday, October 28. An outage of 30 minutes maximum is expected for each impacted VM.

Maintenance status page: http://status.gandi.net/timeline/events/226


We have updated Linux kernel 3.12 (now 3.12.45) and published a new version (3.18) on our HVM platform. These new versions no longer support AUFS and might force some clients to take corrective measures for their services.

Starting today, every server that is created or rebooted on our HVM platform will automatically use version 3.12.45 of the Linux kernel, unless configured to use version 3.18 or a custom kernel.

Please note that these kernel versions do not include AUFS support. Docker users should take special notice, because AUFS has been the default storage driver for quite some time.

To continue to use Docker with this new kernel version, users must upgrade their docker client and images to use a different storage driver, such as btrfs or overlayfs (available for kernel version 3.18 only).

To use version 3.18, you can execute following Gandi CLI [4] command:

$ gandi disk update  --kernel "3.18-x86_64 (hvm)"

You can also change the kernel from the web interface by following these instructions [3].

After the operation is completed, make sure you reboot your server and update your software packages and kernel modules [1].

Clients wishing to use a custom kernel can access more information on our Wiki page [2]. You can also access more information
about kernel update history on our Changelog [5]

[1] http://wiki.gandi.net/iaas/references/server/kernel_modules

[2] https://wiki.gandi.net/fr/iaas/references/server/hvm

[3] http://wiki.gandi.net/en/iaas/references/disk/advanced-boot

[4] http://cli.gandi.net

[5] https://wiki.gandi.net/fr/iaas/references/server/kernel_changelog?&#section312


A new security vulnerability, CVE-2015-3456, was announced last week. The flaw is found in the QEMU virtualization software, and permits an attacker to gain access to a vulnerable host from a virtual machine located on that host.

Immediately following this announcement, we applied the necessary patches, thus reinforcing the existing security measures we had previously implemented. Over the past week, we have continued to study the vulnerability. As a preventative measure, we have decided that a reboot of certain VMs is required in order to ensure that all possible attack vectors have been mitigated.

This preventive reboot will only affect a small proportion of our customers. We will contact affected customers directly via email to provide instructions on performing the reboot on their own.

We will reboot the VMs of affected customers (who have not rebooted on their own) on Monday, May 25 at 11:59 p.m. PDT (that is: Tuesday, May 26, 2015 at 07:59 UTC).

For more information, see the following resources:

If you have questions or encounter any problems regarding this issue, our support team is available to assist you.


Earlier this month we started the process of unifying the SFTP service on Simple Hosting across our three data centers by merging their SFTP keys. Over the next few weeks, we're taking the next step towards a more unified infrastructure with migrations of the SFTP endpoints.

While most customers will not notice any disruption in service, we want to keep you informed of our operations so you can avoid any possible issues.
Here's a schedule:

Datacenter Endpoint Date of migration
Baltimore sftp.dc1.gpaas.net December 30, 2014
Luxembourg sftp.dc2.gpaas.net January 5, 2015
Paris sftp.dc0.gpaas.net January 6, 2015

What are the possible issues?

Loss of connectivity

It's possible (though unlikely) that a running SFTP connection will lose connectivity during migration. This will happen very rarely, and will not have major consequences. Recovery will consist of simply reconnecting.

DNS / Firewall Issues

Since the IP address of each endpoint will change, DNS propagation problems may arise for some customers.

If you are having trouble connecting to the service immediately following the migration window, this may be the cause. Simply waiting for propagation should resolve it.

Also, if your firewalls or other security systems limit SFTP outbound to specific IP addresses, it will be necessary to adjust the rules on these systems to allow the new addresses (and disallow the old).

Feel free to contact Gandi Support if you encounter any issues.


An incident has occurred on one of our storage units in the Parisian datacenter. Our technical team is working to resolve the issue as quickly as possible.

UPDATE - 2:57 AM (UTC) : the situation should be back to normal. Feel free to contact our support if you notice anything wrong.


We just suffered a major incident at one of our facilities. A faulty processor caused the shutdown of a storage unit. 

 

As communications to the disk were interrupted, all operations (reboots, changes, etc) were suspended. 

 

We restarted the unit, and all the services have begun recovering. Operations were queed and are being executed once again. No data was lost. Everything should be returning to normal.

This incident started at 16:19 CEST (07:19 Pacific time). The system was recovered at 16:57, and all queed operations were fully resolved at 17:25 CEST (08:25 Pacific time). 


We do apologise for this interruption in service. 

 

As a reminder, you can see the status of our services here:

https://www.gandi.net/servstat 

You can also follow our twiiter feed from the Gandi Noc at @gandinoc. 

This news feed is available at: https://www.gandi.net/news 


On Tuesday, 7 October, we experienced a series of serious incidents affecting some of the storage units in our Parisian datacenter. These incidents caused two interruptions in service for some of our customers, affecting both Simple Hosting instances and IaaS servers.

The combined effect of these interruptions represents the most serious hosting outage we've had in three years.

First and foremost, we want to apologize. We understand how disruptive this was for many of you, and we want to make it right.

In accordance with our Service Level Agreement, we will be issuing compensation to those whose services were unavailable.

Here's what happened:

On Tuesday, October 7, shortly before 8:00 p.m. Paris time (11:00 a.m. PDT), a storage unit in our Parisian datacenter housing a part of the disks of our IaaS servers and Simple Hosting instances became unresponsive.

At 8:00 p.m., after ruling out the most likely causes, we made the decision to switch to the backup equipment.

At 9:00 p.m., after one hour of importing data, the operation was interrupted, leading to a lengthy investigation that resulted in eventually falling back to the original storage unit. Our team, having determined the culprit to be the caching equipment, proceeded to change the disk of the write journal.

At 2:00 a.m., the storage unit whose disk had been replaced was rebooted.

Between 3:00 and 5:30 a.m., the recovery from a 6-hour outage caused a heavy overload, both on the network level and on the storage unit itself. The storage unit became unresponsive, and we were forced to restart the VMs in waves.

At 8:30 a.m., all the VMs and instances were once again functional, with a few exceptions which were handled manually.

We inspected our other storage units that were using the same model of disk, replacing one of them as a precaution.

At 12:30 p.m., we began investigating some slight misbehavior exhibited by the storage unit whose drive we had replaced as a precaution.

At 3:50 p.m., three virtual disks and a dozen VMs became unresponsive. We investigated and identified the cause, and proceeded to update the storage unit while our engineers worked on the fix.

Unfortunately, this update caused an unexpected automatic reboot, causing another interruption for the other Simple Hosting instances and IaaS servers on that storage unit.

By 4:15 p.m., all Simple Hosting instances were functional again, but there were problems remounting IaaS disks. By 5:30 p.m., 80% of the disks were accessible again, with the rest following by 5:45 p.m.

This latter incident lasted about two hours (4:00 to 6:00 p.m.). During this time, all hosting operations (creating, starting, or stopping servers) were queued.

Due to the large number of queued operations, it took until 7:30 p.m. for all of them to complete.

These incidents have seriously impacted the quality of our service, and for this we are truly sorry. We have already begun taking steps to minimize the consequences of such incidents in the future, and are working on tools to more accurately predict the risk of such hardware failures.

We are also working on a customer-facing tool for incident tracking which will be announced in the coming days. 

Thank you for using Gandi, and please accept our sincere apologies. If you have any questions, please do not hesitate to contact us.

The Gandi team


Page 1 2 37 8 9
Change the news ticker size