When the Internet fights back
By Laura Haight
Originally published as The Digital Maven in Upstate Business Journal
It's been a rough couple of weeks for Internet users. And an eye-opening one for IT engineers and security analysts.
Both Cloudflare and the Amazon web services crash were self-inflicted wounds. Add to that, the Internet of Things (IoT), which promises to change the way we do just about everything, is already delivering on that promise. But not in the way we hoped. Instead even child's toys - like CloudPets - can be a Trojan Horse. CloudPets lets parents and kids communicate through a cuddly stuffed animal. But those audio files have been living on a completely unsecured database, used frequently by app developers because it is flexible and free. Unfortunately, the hidden cost of "free" is the lack of security and user passwords as well as audio files themselves were freely accessible on the Internet. That database was exposed because it was insecure, not hacked
SC Media, a cybersecurity news service, reported that Paul Calatayud, chief technology officer (CTO) at FireMon, a network security firm, calls “IoT the IoMT as in the Internet of Malicious Things.” The cuddly teddy bear exposure, he says, illuminates two big problems: The growing use of open source databases that lack security, and putting devices on the internet.
For most of us, CloudFlare may have seemed a non-story, one of those techy stories. But only because we didn't really understand how deeply it might affect us. As a journalist, I was taught - and in turn trained others - to always be sure to answer one question in every story: "What does this mean to me."
CloudFlare is a company that provides enhanced security to nearly 5 million websites. Those websites have tens of millions of users and you are probably one of them.
Techies now call the CloudFlare incident #cloudbleed to tie it to the disastrous #heartbleed attack in April 2014. Both incidents are leaks, not hacks. And they are the result of a bug in the code. Millions, maybe billions of lines of code knit together in the systems software. Finding one small error is the perennial "needle in a haystack." If you were looking for it, you would probably never find it. But Google employee Tavis Ormandy stumbled across it and found “private messages from major dating sites, full messages from a well-known chat service, online password manager data, frames from adult video sites, hotel bookings” in the data saved by search engines.
The exposure was a slow leak and the bad code exposed date only about .00003 percent of the time. But when you are dealing with traffic on 5 million sites with an untold number of users for a 6 month period. Well, that's a lot of hits. And a lot of errors.
And then there's Amazon's web services. Home to more than 150,000 websites and cloud services, the world notices when something goes wrong at Amazon. The four-hour outage last Thursday was the most recent, but certainly not the only hit to the midsection Amazon has had to endure. The company has had a significant outage just about every year. But this one was an unforced error.
An engineer trying to upload a fix to correct a problem in a small number of servers made a typo that took out a large swath of Amazon's servers across the country. If your website was not running on an Amazon server, you might still have had a problem. Much of the data – photos, videos, logos, databases – used to populate websites were on the affected servers.
In some way, all of us were affected. But mostly, the outage was annoying. Some cloud services slowed down, some were unavailable. Other sites were functional but painfully slow. But big business took the biggest hit. The Wall Street Journal reported that the outage "cost companies in the S&P $150 million, according to Cyence Inc., a startup that specializes in estimating cyber-risks. Apica Inc., a website-monitoring company, said 54 of the internet's top 100 retailers saw website performance slow by 20% or more."
Technology and cybersecurity blogs are working overtime trying to find the meaningful takeaway from these events. Some fall into "we've-created-a-monster" camp; others are in the "stuff-happens-don't-sweat-it" group.
There's a middle ground, however. Yes, we've become overdependent on something we don't understand and sometimes it will rear up and bite us just to show us it can.
That was last week. We got bit.
As an IT executive, I have experienced my share of emergencies and self-created problems. My post-event focus was to classify what happened first and then determine if it was or could be within our capabilities to stop it. If not, what did we need to work on to develop workarounds and redundancies to ameliorate the problems the next incident would cause.
When the mass of inter-connected devices, servers, toasters, furry toys and Fitbits that we call the Internet gets us back, there is little we can do while we're in the thick of it. But when the dust clears, we can resolve to be smarter and, in turn, ask better questions, and demand better answers from ISPs, online services, and third-party vendors. How are you protecting my data? Have you ever been hacked? What did you change after the hack? Have you tested a total failure to see how long it would take to come back up?
The post-mortem on Amazon's server event is focusing on the lack of redundancy. Only the largest companies have enough money or foresight to distribute their data across multiple companies and servers so they can never lose access to everything. Fewer still maintain high availability (HA) with have redundant servers in geographically diverse locations (such as east and west coasts) that back each other up fully in real time. Those are expensive options.
But smaller regional and local business can still ask online cloud providers about redundancy. Most providers promise that they are doing daily backups. You might ask where the backups are located. They will be far less use to you if your backups are In the same building; even sometimes on the same server. It's also important to ask how you can quickly switch over to a West Coast backup server if the East Coast goes down. Still, it will be bigger businesses that consider these solutions.
Most of us will muddle along, swearing under our breath, when once or twice a year, the Internet rears back and throws a sucker punch our way.
