Incident Details

Resolved [15/02/2022 23:09]

Issue resolved.

Update [14/02/2022 16:42]

Reason For Outage

A dark fibre between THN and LD8 dropped at around 04:40AM on Friday morning due to what was supposed to be a non-service impacting audit in one of the splice cases near Telehouse. That itself wasn't service affecting as all the traffic immediately failed over to the fibre between THW and LD8. Whilst the two fibres are geographically diverse, including how they enter the building at Harbour Exchange, they route to the same Zayo rack.

The third party contractor that was sent out to investigate and repair the fibre between THN and LD8 on Friday afternoon was specifically told to only work on the fibre to THN but unplugged the fibre to THW briefly which isolated LD8 from the rest of the network at 12:10. This resulted a short blip in connectivity for leased lines at LD8, but it was unplugged long enough for all the broadband subscribers' PPP sessions to time out which caused them to disconnect. It took around 15-20 minutes for them to all reconnect.

The contractor found a bend in the THN to LD8 fibre and repaired it by 12:52 on Saturday afternoon.

We apologise for the inconvenience caused by this disruption. We are actively looking to introduce further alternative routes into our network to ensure that this cannot happen again.

Update [11/02/2022 12:44]

The issue was isolated to our LD8 POP. Disruption should have been limited to around a minute for leased lines linked to this POP. Broadband sessions routing to LD8 would have dropped and it may be necessary to reboot the router to regain a session. We are now seeing the vast majority of broadband sessions as restored. We apologise for the disruption this may have caused and will supply further details once fully understood.

Investigating [11/02/2022 12:27]

We are aware of an incident which has caused issues for broadband and leased line circuits. We will provide further details once full information is understood.