Guest comment by Alan Stewart-Brown
One of the downsides of running a business in the digital age is that no provider can offer a 100 per cent guarantee that their network connection to the rest of the world will always be available. Service level agreements often compensate for downtime against the price of the connection, but this can pale in comparison to the cost of the business of having no network – both directly in terms of lost sales and productivity as well as reputational damage.
The largest outage recently in the UK was in April this year, when Virgin Media’s AS5089 network went down – and critical digital services provided by businesses throughout the country came to a halt. But national and regional disruption is a relatively common occurrence, with six serious outages in the US over the past 12 months alone.
Network outages at a more granular level are often caused by misconfigurations on premises or within the data centre – with telcos blameless, even though the business network can’t be reached.
A 2022 survey conducted by Opengear canvassing the views of 500 network engineers and 500 CIOs, showed that 50 per cent of CIOs ranked financial loss among the main impacts on their business due to network outages over the past two years.
It showed the proportion of outages costing over $100,000 has soared in recent times. Over 60% of failures result in at least $100,000 in total losses, according to the report, up from 39 per cent in 2019. The share of outages costing upwards of $1 million increased from 11 to 15 per cent over that same period.
The ongoing expansion of networks to embrace edge computing has led to increased compute being pushed to the edge and more complex equipment being put in place in remote locations, where there are no IT staff
In terms of monetary cost to businesses, 20 per cent of CIOs polled stated that network outages have cost their business over $1 million over the past two years and for 3 per cent of this group the costs escalated to more than $25 million.
Unfortunately, 52 per cent of network engineers say their organisation has seen an increase in the number of network outages experienced over the past two years. Naturally, with a trend of rising costs and increased business disruption, network downtime is a concern for the CFO as well as the CIO.
Why are these outages occurring?
But outages themselves can often be difficult to avoid. After all, they have a wide range of root causes. Cable interconnects, power supplies, switches, dense compute chassis, storage arrays, and even air conditioning are all potential sources of problems. The climate crisis has even been linked to a greater risk of outages.
And network devices and services are continually increasing in complexity. New software stacks are created to achieve more within the enterprise, but as a company’s digital footprint increases, risks from bugs, exploits and cyber attacks also increase.
Additionally, the ongoing expansion of networks to embrace edge computing has led to increased compute being pushed to the edge and more complex equipment being put in place in remote locations, where there are no IT staff, and where network redundancy is not feasible.
Perhaps the underlying reason for more outages is simply because more services are being offered by IT now than ever before. Digital transformationmeans we can work from wherever we like, assuming we have a fast internet connection, while the security, software, communications, and connection management is undertaken in a data centre or on the company’s premises. Putting it simply, the more things there are, the more that can break, and the more business disruption there will be.
It’s clear that network outages and the resulting downtime remain a serious issue for many businesses operating today. However, the approaches taken by organisations to rectify these problems often fall short. Too many businesses still rely on manual ways of working, sending engineers out to site and relying on manual methods of documentation.
Every company needs to know without question that when trouble strikes for whatever reason – whether it’s a hurricane or a cyber-attack, a local power outage or a global pandemic, their network will be ready to deal with it.
Since the Out of Band network separates management and user traffic, businesses can lock down, restrict access, and fully secure the management plane
One priority must be ensuring businesses have visibility and the agility to pivot as problems occur. Many are not proactively notified if something goes offline. Even when they are aware, it may be difficult to understand which piece of equipment at which location has a problem. That’s where Smart Out of Band Management using an alternative path into the network really comes into its own.
OOB gives organisations an alternate way to connect to their remote equipment such as routers, switches, and servers through the management plane, without directly accessing the device’s production IP address in the data plane and independent of the primary ISP connection the company uses.
This Out of Band path is separate from the production network and allows administrators to securely monitor, access, and manage all devices without interfering with normal operations, and even more importantly, without having to allow the data plane level access to the management plane.
Since the Out of Band network separates management and user traffic, businesses can lock down, restrict access, and fully secure the management plane. Also, they can configure, manage, and troubleshoot their devices even when the data plane is down. An OOB solution offers organisations a secondary connection, often through 4G LTE, that lets network technicians solve problems from anywhere, and most importantly, saves the company time and money.
When network outages occur, companies have to act quickly. By assuming that a network will at some point go down, no matter how resilient it is designed, companies can prepare for the eventuality and build in ways to mitigate the impact. It’s this ability to think forward and consider the bigger costs – lost sales, operational and reputational impact – that motivates decisions to build resilience, such as Out of Band (OOB), into the heart of IT operations. By doing this, companies ensure that technical emergencies don’t escalate into financial disasters.
Alan Stewart-Brown is Vice President EMEA, Opengear