Facebook is one of the world’s tech giants. The company is worth almost a trillion dollars. Nearly half of the people on the planet, 3.5 billion, use one of Facebook’s apps, which include Instagram and WhatsApp, regularly. They are at the forefront of the tech revolution and have been a driving force behind many of the major changes in our society, both good and bad. But for five hours on Monday, their entire global operation was frozen, anyone who typed facebook.com into their web browser was automatically told that “this site can’t be reached.”
Many suspected that the company was hacked. The outage came only hours after a 60 Minute segment aired which featured a whistleblower who claimed that the company knew their platform was causing disinformation and hate speech but did little to curtail it. But an official statement later said that the problem was due to a faulty configuration change in the company’s Border Gateway Protocol (BGP) records. The BGP is how internet service providers share routing information so the error basically deleted the map telling computers and mobile phones where to find the pictures of food, cat memes, and news (both real and fake) that the company is known for. Several domain registrations sites even listed the domain facebook.com as up for sale when their automated web crawlers found that the site had seemingly expired.
It wasn’t just users that were locked out of Facebook. Employees reported that they were not able to use even get into the building because their badges no longer worked. This was likely one of the reasons why the site was down for so long.
So, how did this happen? “This is what happens when systems are not properly segmented,” said Charles Meyers. He worked as a Chief Technical Architect for Wells Fargo for nearly twenty years and his work integrating their workplace technology led him to help build the Real Estate Cyber Consortium which he began leading this year. “I felt like Chicken Little at the time,” he said, “I kept telling our team that the technology in our buildings was vulnerable but most people didn’t see the threat.”
It wasn’t until he walked through a new office with the company’s head of IT that he finally got his message heard, “I asked him how he planned on managing the shadow networks in our buildings and he kept saying ‘there are no networks in our buildings that I don’t know about.’ Then I walked him up to a light switch and told him that it was running a Zigbee wireless protocol by default that was activated by the installer without our knowledge. That was when he began to understand the threat.”
The cyber security struggles of Facebook when it comes to their physical office is not unique. There is often a disconnect between a company’s IT departments and the building’s operational technology (OT) personnel that can lead to a “not my problem” mentality. It can also be due to how building systems are outsourced. “The team that installs and configures building systems is usually not the one that maintains it,” Meyers said. “The installers usually have an admin account but then create an operator account once they are done so the admin never gets updated.” This can lead to laughable and disastrous oversights and easily guessable user names like ‘admin’ and predictable passwords like ‘password.’
Part of the solution, Meyers thinks, is to have an industry-wide partnership to help standardize protocols and best practices. “If enough users and vendors get together and demand a certain level of security then anyone else creating or maintaining building systems will have to conform,” he explained.
Increased scrutiny of building system cyber security might also affect the relationship between the landlord and the tenant. Savvy occupiers might spend as much time analyzing and vetting a building’s digital systems as they do its physical properties. Building engineers will need to become well versed in their tenants’ IT practices and IT departments will need to have a better understanding of how operation technology fits into their system architecture.
When Facebook went down the internet erupted with people exclaiming that the world was better without it. Whether the company is a net positive for society is still up for debate but what is clear is that Facebook’s high-profile tech troubles have taught the world about the interconnectivity of companies’ internal technology and their offices. Facebook’s pains were self-inflicted but as embarrassing as that might be, it could have been much worse. How long before we see another one of the world’s tech giants brought down by a thermostat or light switch with a ‘temporary’ password?
