Update - Power has been restored, but nodes will remain down until we can reaccess power tomorrow.
Aug 26, 2026 - 19:13 PDT
Aug 26, 2026 - 19:13 PDT
Identified - One of the power breakers that powers parts of both Hive and Farm tripped. Data Center operators are calling facilities.
Aug 26, 2026 - 17:23 PDT
Aug 26, 2026 - 17:23 PDT
Cluster: Hive
Degraded Performance
Cluster: Farm
Operational
Cluster: Franklin
Operational
Login Node
Operational
Compute Nodes
Partial Outage
GPU Nodes
Partial Outage
Network
Operational
Storage
Operational
Quobyte Parallel File System
Operational
Home Directories
Operational
Legacy Storage
Operational
Module System and Software
Operational
Hippo User Portal
Operational
OnDemand
Operational
Scheduler (Slurm)
Operational
Operational
Degraded Performance
Partial Outage
Major Outage
Maintenance
Major outage
Partial outage
No downtime recorded on this day.
No data exists for this day.
had a major outage.
had a partial outage.
Scheduled Maintenance
Emergency Slurm update September 2nd Sep 2, 2026 08:00-18:00 PDT
Update -
We will be undergoing scheduled maintenance during this time.
Aug 24, 2026 - 15:38 PDT
Aug 24, 2026 - 15:38 PDT
Scheduled -
SchedMD, the company that develops the Slurm job scheduler, has privately announced several security vulnerabilities. These vulnerabilities have not been disclosed at this time pending remediation, but have been reported to allow escalated privileges within the clusters.
SchedMD is scheduled to publish a fix on September 2, 2026. As per our commitment to follow UCOP security guidelines, we will be upgrading Slurm once the fix has been released. As a result, we will be implementing a maintenance reservation that will prevent jobs from starting if they overlap the September 2 restart. Currently running jobs that are scheduled to run past September 2 may be subject to cancellation depending on our assessed severity of the vulnerabilities once they are fully disclosed.
Aug 24, 2026 - 15:32 PDT
SchedMD is scheduled to publish a fix on September 2, 2026. As per our commitment to follow UCOP security guidelines, we will be upgrading Slurm once the fix has been released. As a result, we will be implementing a maintenance reservation that will prevent jobs from starting if they overlap the September 2 restart. Currently running jobs that are scheduled to run past September 2 may be subject to cancellation depending on our assessed severity of the vulnerabilities once they are fully disclosed.
Aug 24, 2026 - 15:32 PDT