[RESOLVED] Routing problems within BT connected networks

We are currently seeing routing issues to BT Internet connected locations and 3rd parties who receive service from BT related services.

Upstream engineers are monitoring the major service outage on the BT network but at this time only external/routed connections to the BT networks are affected.

This post will be updated as and when more information is received.

Update @ 11:20 – The problem with BT’s network is starting to hit the major technical news sites.  More useful information can be found at ArsTechnica: http://arstechnica.co.uk/business/2016/07/bt-isps-telehouse-north-major-outage/
and The Register: http://www.theregister.co.uk/2016/07/20/telecity_power_outage_bt_offline/

[RESOLVED] Network Connectivity

Earlier this morning a routing issue was identified in one of our upstream network providers which caused connectivity problems when attempting to access our network.

A work around has been put in place which has restored access and we are continuing to monitor the situation.

We will update this post when we have more details.

Update 10:05: The earlier partial connectivity problems were due to our upstream provider not providing us with a full BGP routing table.  This resulted in only part of the Internet being fully reachable from our network.  A workaround was put in place which restored connectivity whilst engineers investigated further.  As of 10:00 a full BGP routing table is now being received and the earlier workaround has been removed.  We will continue to monitor connectivity and will update this post if required.

[DELAYED] Planned network maintenance

On Tuesday 9th February starting at approximately 21:00 we will be performing essential maintenance on our core network

The maintenance will be performed in such as way as to minimise any possibility of interruption to your services. However
due to the nature of the work, network services will be classed “at risk” as resilience will be impacted.

It is estimated that the work will take less than 1 hour to complete.

Additional updates will be posted to our service status blog at http://www.zebrastatus.net

Update 18/02/2016 – 09:00

Due to the hardware problems we encountered during the original maintenance procedure, we’re waiting for additional spare backup parts to arrive before we’re prepared to perform more maintenance.

As soon as these parts arrive a new maintenance period will be arranged.


Update 12/02/2016 – 16:30

“Anubis”, the affected core router has now been replaced and full network resiliency has been restored.

The maintenance due to be performed on the second core router is now expected to be rescheduled for the week commencing 15/02/2016.  A further update will be made once a firm plan is in place.


Update 12/02/2016 – 12:30

The replacement hardware has arrived on site and is being prepared for installation.


Update 10/02/2016 – 10:00

Replacement hardware has been ordered and we are now awaiting delivery.  Whilst our network resilience is reduced until the replacement can be installed, customers should not encounter any problems.  Engineers will update this post as and when further information is available.


Update: 09/02/2016 – 22:20

An issue was encountered during the maintenance of one of our core edge routers. As a result, this router is currently out of service. All further maintenance tasks have been suspended.

[RESOLVED] Shared Hosting Server Degraded Performance

One of our shared hosting servers, Linsvr2, is currently experiencing higher than usual load.

This is causing websites & email accounts for customers hosted on that server to either run slower than usual or timeout with an error.

Engineers are currently investigating and will update this post as soon as possible.


It appears a customers WordPress website has been compromised and as a result a rogue webscript had been installed.  This script contained programming errors that caused it to run in an “infinite loop”, using all available processing resources.

The compromised website has been shut down until our customer can confirm they have resolved the problem.

[INFORMATION] Christmas & New Year Business Hours

Over the festive period, Zebra’s office will be closed on Christmas Day, Boxing Day and New Year’s Day.

We will be closing at 13:00 on Thursday 24 December and Thursday 31 December. Our usual business hours will apply on all other days.

Outside these times please submit a support request from within your Domaincheck control panel or email [email protected].

Customers who use our datacentre services can continue to use the emergency out-of-hours support line for urgent support issues.

We would like to thank you all for your business this year and we wish you all a Merry Christmas and a Happy New Year!

[RESOLVED] Email Sending Issues

Over the weekend one of our customers email addresses was compromised and used to send out a large amount of SPAM.  This had caused the server to be temporarily listed on a few external email blacklists.

The compromised account has been suspended and the server is no longer blacklisted, we are now waiting for other email providers to update their blacklists.

In the meantime customers hosted on “Linsvr1” might experience difficulties sending to some email destinations but not to others.

Updated: 17:20 

[COMPLETED] Important Network Maintenance

Our upstream network provider, Onyx, have recently made a significant investment in upgrading their core network to a resilient 10Gb capacity architecture to vastly improve the level of service and flexibility of connectivity offered to customers.

The upgrades are now ready to be put into place and this requires Onyx to undertake out of hours migration work during November. Onyx are planning to ensure minimum impact to customers as well as offering additional support to us before, during and after the migration.

Please be advised that a maintenance window from 23:00 on 12th November 2015 until 04:00 on 13th November 2013 has been allocated to this work.

Whilst every effort will be made to minimise impact, services we provide such as Internet connectivity, Website Hosting, Email and DNS and should be considered at risk during this period and we will be sending updates before, during and after the work.

If after the maintenance period has closed you are still encountering issues, please contact us via your usual support channels.

Update: 13/11/2015 04:00 – The network maintenance period has been closed.  No impact on our services was observed whilst the work was taking place.  If any customers should be encountering issues, please contact us via your usual support channels.

[RESOLVED] Shared hosting server failure

Yesterday evening a hardware problem occurred on one of our shared hosting servers. This caused a number of the services offered by this server to fail. As a result, customers hosted on this server would have had problems with sending/receiving emails and/or problems accessing their website.

Users trying to access their control panel may have seen a “Server under maintenance” message.

Engineers worked as quickly as possible to restore this server and all services were up and running again at approximately 08:20 this morning.

Plans were put in motion a number of months ago to replace the server hardware with new high-performance servers and migrate customers to the new platform. The new hardware is already installed in our datacentre and has been undergoing stress-testing before being brought into production. The migration plan we have in place for moving customers over to this new server will be accelerated and we will let the affected customers know as soon as possible when we plan to migrate customers.

[RESOLVED] Temporary routing issue

Earlier this evening at approximately 18:25, we lost connectivity with one of the edge routers in our upstream provider’s network.

Connectivity was still available via alternate paths but routing would have taken a few minutes to stabilize.  Connectivity with the router was re-established approximately 5 to 10 minutes later but during this time connectivity would have been affected by routing converging on the alternate path.

Connectivity was affected for approximately 10 minutes during this time.  We are continuing to monitor the situation.


Update 19:15: Connectivity has stabilized and traffic levels have returned to normal.  We are continuing to monitor the situation and will provide any updates as and when we have further information.

Update 18:50: Connectivity is still being affected via various paths, we’re investigating and will update this post as soon as we have more information.

Update 18:45: We have received an update from our provider advising us of a faulty circuit on one of their primary links causing connectivity issues.  This link has been removed from service whilst investigations are taking place.

[Resolved] Zebra Office Phone Line

At approximately 14:20 on 29.6.15 a fault developed on our office land line interrupting both inbound and outbound calls. Some calls are connecting successfully then cut off during the call.

This fault has been raised to our telecom provider and we are waiting for inbound calls to be redirected to an alternative line.

In the meantime we are still available via email, [email protected] for general enquiries and [email protected] for technical support.

Sorry for any inconvenience caused while trying to contact Zebra Internet Services, we will update this post as soon as we know more.

UPDATE: 17:00
A fault has been detected in our telephone exchange and engineers are investigating.

UPDATE: 15:00
We have been informed that a faulty component has been identified and replaced  at the exchange.
We can once again be contacted on 0191 261 2252.

Service Status Information