UPDATE 13:30 - Groupdata is back online
UPDATE 02:25 - Astrogate and Windata are back online, except groupdata
UPDATE 22:20 - Home server is back online and email working again

In order to upgrade the operating system on serveral servers, we schedule a maintenance downtime on

Sunday, 4th January 2015, starting at 22:00.

Schedule:

22:15 start working on the home server (mail services disabled, incoming mail will be queued)

22:20 start working on the group share servers (windata & astrogate)

~ 22:45 home directories and mail services should work again

~ 00:00 group shares will incrementally come back during the night

During the downtime you can access readonly backups of your data of the night before, take a look at our readme.

We apologize in advance for any inconvenience this service interruption might cause.

The ETH Zurich will be officially closed between Wednesday, 24th of December 2014 and Sunday, 4th of January 2015. During this time, we can only provide limited support. Please follow these rules to save us from superfluous work:

• Switch off printers
• Switch off your personal workstation and notebook except for the following:
• Do not switch off our managed Linux workstations.

We will try to follow our e-mail, but you may also have luck and meet some of us in our IRC channel.

This post is meant to give you a short overview of what has been accomplished in D-PHYS IT by ISG this year. We’ve been hard at work to further improve and extend our services for you, our customers. Some highlights of 2014:

• eXile: in order to be able to keep Windows XP machines that cannot be upgraded connected to the network, we have created the exile system of dedicated virtual firewalls. Currently there are 57 computers safely hidden in this network.
• Security flaws: 2014 saw the disclosure of three rather severe and widespread security problems in quick succession: Heartbleed, Shellshock and Poodle. We patched all affected systems within hours of the announcements and also scanned the network for hosts that had been overlooked. If you're managing any networked machines (not just servers!) yourself, please make sure those are not vulnerable.
• Outages: we had a major incident on August 27 due to a failure of the server room cooling system. Fortunately we were able to repair the damage within hours. Other than that, our systems have been very stable in 2014 and we only had minor issues.
• Storage: in 2014 the disk space occupied by data and backup grew from 535 TiB to 685 TiB, further increasing the yearly growth rate. Another 120 TiB are already in the pipeline.
• Printing: in cooperation with Informatikdienste we prepared and introduced the new ETH printing system in D-PHYS. Several groups have migrated already, the rest of D-PHYS will follow in 2015.
• IPv6: during the last 12 months we prepared the D-PHYS network for dual stack (IPv4 + IPv6) operation. The biggest step towards a working IPv6 infrastructure was the deployment of an IPv6-ready DHCP server. Beginning next January we will incrementally hand out IPv6 addresses in the D-PHYS network. Later on, we'll make our services IPv6-ready.
• Brain drain: two ISG group members decided to take on new challenges this year. In November, Thomas Berchtold left us after 3 successful years to become the new Head of IT of D-BAUG, and Elmar Heeb, the founding father of ISG D-PHYS, will start his new job in Informatikdienste in February. We thank both Thomas and Elmar for their dedicated work and contribution to the team and hope to stay in regular contact with them in the future. Christian Ringger will replace Thomas in January, while Elmar's succession is still work in progress.

Happy Holidays and see you in 2015!

Together with our colleagues at Informatikdienste we have adopted the new ETH printing system in our department. Read about the many advantages of the project in our documentation. On October 15 we have migrated the first Institute (ITP) where people now can benefit from various features like pull printing and automatic toner supply.

So who's next to join us? We have printers in stock, so if your group is interested, we can accommodate you on very short notice. Just get in contact.

UPDATE Thu 09:30 - all systems should be back to normal. Please let us know if you still encounter problems. Thanks to Axel and Paddy for their commitment and the incredible Dalco service for fixing it within 6h (at 8am, mind you).

UPDATE Thu 00:50 - a broken valve blocked the cooling water in the HIT D 13 server room and all 14 water cooled racks severely overheated (not just D-PHYS). We managed to revive almost all services with the exception of the GGL file shares (this server is dead). We'll post updates later today when we have more information.

complete loss of cooling in the server room. We have yet to assess the damage.

Update Thursday 07:15 - all systems back to normal. whew.

Update Wednesday 11:00 - first rack back to normal. The picture shows how it looked during the remodeling.

Update Wednesday 07:00 - the most difficult part is over and the first rack is being retrofitted as we speak

All of D-PHYS's important servers (and services: mail, homes, SAN, web) reside in two water-cooled racks in HIT D 13. On Wednesday, August 20 those racks will have to be retrofitted by our colleagues of Informatikdienste since certain spare parts are no longer available. We have an elaborate plan how to externally power the servers while the racks are offline that schedules a 5-minute downtime that most of you won't even notice. However, there is a small chance that this external power supply does not work as expected which would lead to a longer interruption. Unfortunately we have no influence on the date, time and procedure of this modification and can only try our best to minimize potential consequences. So if something should go wrong next Wednesday, please don't panic, we'll be hard at work to fix it ASAP.
Thank you for your cooperation.

We have scheduled a software maintenance of the D-PHYS mail server for tomorrow, Wednesday, the 18th of June 2014, starting in the late afternoon around 5pm. A downtime of all D-PHYS mail services during the evening will be part of the maintenance. The downtime is expected to take approximately 15 to 30 minutes.

During the downtime sending and receiving e-mails will not be possible and the web mail service will be not available. Incoming mails during the downtime will be delayed.

Additionally there will be a downtime of our "BackupPC" backup service for laptops and lab PCs due to server relocation on Thursday (19th of June 2014) starting around 9am.

Microsoft provided a final bunch of patches for Windows XP in April 2014. Since then no more security and stability fixes are going to be released. This means that still running Windows XP machines conflict with the ETH Bot (Acceptable Use Policy for Telematics) which requires that every computer connected to the ETH network must be fully updated and secured.

The central IT security group of ETHZ continuously inspects the network streams for signatures of XP computers. In the D-PHYS public networks they still detect around 15 Windows XP based computers. If you have a running XP machine connected to the public network, please migrate the operating system to a newer version i.e Windows 7.

In case you are forced to keep Windows XP up and running, you can migrate the machine to our eXile network. Simply send the required information to isg@phys.ethz.ch after you've read and understood the eXile Terms-of-Use, so we can prepare the machine for the eXile network.

If you have any questions or need help please do not hesitate to contact the ISG D-PHYS Helpdesk

On Monday, May 5, there will be a scheduled power outage in the HPT building, between 13:00 and 22:00. This will also affect ISG's offices, but none of the servers. Our services will run as usual, but we'll have to move the helpdesk to a temporary location during the outage. So please be patient when calling and wait for your call to be redirected to our pager or write an email instead. We hope to be back to normal by Tuesday morning.

On Monday the public was made aware of a severe bug in OpenSSL, a cryptography library which is used as the core of many cryptographically secured IT services. Since the bug was in the Heartbeat extension it has been named "Heartbleed".

This bug allowed attackers to stealthily access parts of the memory used for cryptographic actions, i.e. it may include digital keys in use on servers or passwords transferred over encrypted connections.

If you used any password-protected D-PHYS web services or the D-PHYS mail server between 12th of December 2013 (or used the BackupPC web-interface since end of 2012) and Tuesday, the 8th of April 2014, there is a very small chance that your D-PHYS password and possibly other transmitted data may have been leaked to an attacker. We currently have no indication that this has actually happened on our servers.

To be safe, you might want to change the password of your D-PHYS account and any other account where the same password is used. See this Heise article for a discussion (in German) about whether you should change your password or not.

Read the rest of this entry »