Liberal, Conservative MPs to speak at Oct. 7 march to Parliament Hill
A Liberal MP and a Conservative MP will be part of a team delivering speeches at an event in Ottawa commemorating the one year anniversary of the attacks on Oct. 7.
An independent report into the 2022 Rogers outage says the company lacked several protections and redundancies that could have either prevented the outage or ended it sooner.
The report delivered to the Canadian Radio-Television and Telecommunications Commission says that since the outage, the telecom company has implemented the changes needed to address the cause of the outage and improve network resiliency and reliability.
In a separate letter posted to its website Thursday, the CRTC confirmed that Rogers has also implemented all the report's additional recommendations.
"We said we would fix this – we completed a full review of our networks, strengthened our network resiliency, implemented all the report recommendations, and today our networks are recognized as the most reliable by global benchmarking leaders," said Rogers spokeswoman Sarah Schmidt in a statement.
The outage in the early morning of July 8 two years ago lasted more than 24 hours and affected more than 12 million customers.
A configuration error during the network upgrade caused a flood of data to the core network routers, which crashed, according to the executive summary of the report by Xona Partners Inc. posted online Thursday.
The network failure could have been prevented if the core network routers had been configured with an overload limit, the report said.
Once the outage occurred, the report says it was prolonged by several factors.
The Rogers network operation centre and other critical remote infrastructure sites did not have redundant connectivity from other service providers, the report said, limiting access to critical equipment during the outage. Staff had to be physically dispatched to remote sites in order to access the affected routers, delaying recovery efforts.
In addition, Rogers staff also didn’t have backup connectivity from alternative service providers, and so they couldn’t communicate with each other until the company sent SIM cards from other service providers to its remote sites.
The report said that staff also didn't initially have access to information like the routers' error logs and were unable to pinpoint the root cause of the outage for around 14 hours. There had also been multiple configuration changes made that day. These two factors contributed to the root cause being initially misdiagnosed, the report said.
The measures taken by Rogers since the outage include addressing the critical deficiencies exposed by the outage, separating the IP core for its wireless and wireline networks, and improving the processes for change management and incident management, the report said.
The report made seven recommendations of additional measures Rogers could take to improve its network resiliency.
Among the recommendations, which have since been taken by Rogers, are that the company test emergency roaming with other mobile network operators, develop a detailed root cause analysis for future outages, and expand the scope of incident management drills.
Rogers sent a letter to the CRTC on Jan. 17 outlining how it responded to the report's recommendations of additional measures.
In the CRTC's letter confirming those additional measures were implemented, the commission said that by July 4 next year, Rogers must report on whether the measures continue to address reliability issues, and on progress made in separating wireline and wireless core networks.
Rogers is partnering with Cisco in its work to split and build a new dedicated IP core, separating the two networks, said Schmidt. The company has also introduced new change controls that will limit the effects of "customer-impacting events," she said, as well as "AI-based predictive simulation capabilities to strengthen our testing and monitoring."
The report also included recommendations for all telecom network operators based on the "important lessons learned" from the outage. These include implementing router overload protection in the IP core and distribution networks; providing backup connectivity for the network operation centre, critical remote sites and critical staff; and simulating network failure and outage scenarios to uncover deficiencies.
This report by The Canadian Press was first published July 5, 2024.
A Liberal MP and a Conservative MP will be part of a team delivering speeches at an event in Ottawa commemorating the one year anniversary of the attacks on Oct. 7.
Travelling on a budget can be stressful, but there are ways you can ensure you're getting the best deal on flights as the holiday season approaches.
A Toronto police officer has been arrested after allegedly stealing three bottles of alcohol from a store.
Police on Vancouver Island have made an arrest in the case of a 78-year-old woman found dead in her home almost two years ago.
People across Florida were given notice Sunday that Milton, for now just a tropical storm off the coast of Mexico, could intensify rapidly into a major hurricane before slamming midweek into the storm-ravaged Gulf Coast.
Donald Trump's fourth scheduled stop in eight days in Wisconsin is a sign of his increased attention as Republicans fret about the former president's ability to match the Democrats' enthusiasm and turnout machine.
Police released the identities of the mother and daughter who were killed after a fire tore through a 160-year-old building in Old Montreal on Friday.
According to a survey on recent alcohol consumption, younger Canadians are more likely to have not had a drink in the past week. For those interested in slowing or stopping their alcohol consumption, there's a phrase: 'sober curious.'
Three people were taken to hospital after a crash between a pickup truck and a car in a rural part of Ottawa's south end on Saturday afternoon.
Chantal Kreviazuk is set to return to Winnipeg to mark a major milestone in her illustrious musical career.
From the beaches of Cannes to the bustling streets of New York City, a new film by a trio of Manitoba directors has toured the international film festival circuit to much pomp and circumstance.
A husband and wife have been on the road trip of a lifetime and have decided to stop in Saskatchewan for the winter.
The grave of a previously unknown Canadian soldier has been identified as a man from Hayfield, Man. who fought in the First World War.
A group of classic car enthusiasts donated hundreds of blankets to nursing homes in Nova Scotia.
Moving into the second week of October, the eastern half of Canada can expect some brisker fall air to break down from the north
What does New Westminster's təməsew̓txʷ Aquatic and Community Centre have in common with a historic 68,000-seat stadium in Beijing, an NFL stadium and the aquatics venue for the Paris Olympics? They've all been named among the world's most beautiful sports venues for 2024.
The last living member of the legendary Vancouver Asahi baseball team, Kaye Kaminishi, died on Saturday, Sept. 28, surrounded by family. He was 102 years old.
New data from Greater Vancouver and the Fraser Valley shows a surge in supply and drop in demand in the region's historically hot real estate market.