The Washington PostDemocracy Dies in Darkness

How the cloud has opened new doors for hackers

Easy remote access has helped transform many companies’ computer systems from electronic fortresses to entities more like coffee shops with people and services streaming to and fro.

March 2, 2020 at 7:00 a.m. EST
(iStock)

When Wyze Labs announced late last year that data on 2.4 million users of its smart home security camera had been stolen, the hack was the result of an employee essentially leaving the door to its database unlocked.

The Seattle-based company, started by two Chinese nationals three years ago, has mushroomed in popularity since it launched a cheaper alternative to consumer-grade security cameras sold by Ring, Nest and others. To help manage that growth, co-founder Dongsheng Song wrote on the company’s website, Wyze put customer data into a new database. Protections for this data were mistakenly removed by a Wyze employee in China, allowing the hack.

The incident, which put the start-up in a hall of shame with other companies that have fallen victim to big cybersecurity breaches, illustrates the dangers of corporations moving operations to the cloud, massive commercial collections of powerful data centers scattered around the world and accessed via the Internet. That easy remote access has helped transform many companies’ computer systems from electronic fortresses into something more like coffee shops, with a steady stream of people and services moving in and out, researchers and cybersecurity experts say.

Capital One looked to the cloud for security. But its own firewall couldn’t stop a hacker.

Cloud providers such as Amazon Web Services, Microsoft’s Azure and Google Cloud have their own security features, but they typically manage security only for the underlying infrastructure. Customers are responsible for securing the applications and databases that they put on top of that infrastructure. Software that powers smart thermostats, smart speakers, online shopping, online games — nearly everything anyone does online these days — runs through applications and databases in the cloud.

The market research firm Gartner estimates that the global cloud market was worth more than $226 billion last year and is likely to reach $263 billion in 2020, a growth of 16 percent. Amazon Web Services, the first large public cloud, was launched in mid-2007 and is today a $40 billion business. (Amazon CEO Jeff Bezos owns The Washington Post.)

Previously, computers with sensitive information were housed in secure rooms at individual companies’ facilities and that information was accessed only by company employees, or they were kept at local data centers, which securely housed computers for companies. But cheaper, easier and physically more secure solutions offered by Amazon, Google, Microsoft and others allow companies to store their data off-site and run a variety of applications — for example, conducting complicated analytics on proprietary financial data.

Capital One says data breach affected 100 million credit card applications

“When managing their own physical servers, companies have to maintain the equipment, manage everything in a secure facility and supervise all personnel with access to the equipment,” said Manav Mital, a cybersecurity expert. Cloud companies take care of these tasks, protecting servers in high-security facilities with layers of backup in case of a failure and managing network security. They also apply economies of scale — running 50,000 physical servers or more in a single location — bringing everyone’s costs down.

Though the cloud is physically more secure, the ease of use has led to a boom in new applications and databases and increasingly complex configurations that are difficult to manage and monitor, said Mital, who co-founded the cloud-security start-up Cyral.

And while companies still wall off their private information from unauthorized personnel, using firewalls or software that protects access to a network or to applications or databases within that network, more people and programs now need access to the information, making it easier for hostile actors to find potential holes.

“The cloud has made expectations of fast delivery a reality, and so the temptation is enormous for engineers to pull down the firewall when they’re on the hook to deliver,” said Dan Ehrlich, a Texas-based computer security consultant who discovered the Wyze breach. Sometimes the engineers fail to lock up again.

Wyze declined to comment further.

It’s not just Wyze. The Choice Hotels chain, whose brands include Quality Inn and Cambria Hotels; the global technology company CenturyLink; the multimedia software company Adobe; and the cannabis sales system maker THSuite all lost control of sensitive customer data. That resulted in the exposure of names, email addresses and sometimes credit card numbers. In THSuite’s case, the breach even exposed the quantity and frequency of individual customers’ cannabis purchases.

Last fall, Capital One was breached, exposing tens of millions of credit card applications, including 120,000 Social Security numbers and nearly 80,000 bank account numbers — a hack enabled in part by a misconfigured firewall.

The vulnerability goes beyond public hacks, too. Certain cybersecurity websites scan the public Internet and flag threats and exposures. For example, BinaryEdge.io, a cybersecurity data firm, recently listed 35,516 unsecured databases worldwide, most of them in China and the United States, and the majority in the cloud. Ehrlich spotted the unsecured Wyze database while browsing exposed databases with the BinaryEdge service.

U.S. Customs and Border Protection says photos of travelers were taken in a data breach

The cybersecurity firm Risk Based Security estimates that unauthorized access to sensitive information, including cloud exposures, increased by 54 percent in the first half of 2019 compared with the same period the previous year.

Though cloud infrastructure is secure, the explosion of increasingly complex services the cloud has enabled makes it more difficult to monitor access and easier to make mistakes. Think of multiple control boards with arrays of switches that lock and unlock doors in multiple huge buildings. Each switch has to be flipped up or down depending on the desired flow of traffic into, out of and through the buildings. It’s easy for one switch to be flipped the wrong way, leaving a door open. In the pre-cloud days, there were simply fewer doors to be left unlocked and more people involved in setting up applications or databases.

Many popular modern developer tools are designed to be initiated without any access restrictions or even passwords in place. This allows developers to quickly try out these tools, and enables maximum agility for teams building services. However, it places on the developers the duty of ensuring that the appropriate access restrictions are applied each time the tool is used.

“People don’t know how to configure these databases in the cloud,” said Chris Morales, the head of security analytics at Vectra, which helps companies respond to breaches. He said human error in setting up systems in the cloud are responsible for most of the breaches, rather than criminals gaining access by stealing passwords or by other means. “Misconfiguration has driven most of these exposures,” he said.

Many of the publicized incidents have involved Amazon Simple Storage Service, known as Amazon S3, and the software company Elastic’s Elasticsearch, caused largely by the services’ popularity. Both data storage services can be set up quickly and cheaply.

“Developers look for functionality first, then performance, and security last,” said Jack Kudale, who founded the cyber-insurance firm Cowbell Cyber to protect small and medium-size companies from the potentially devastating cost of breaches.

Amazon S3 is secure by default, meaning that access is locked down to just the account owner and administrator if a customer uses the standard configuration. However, developers sometimes change these configurations in ways that expose data to a wider-than-intended audience. To allow an analytics program access to data, for example, they may temporarily open public access to a database with a toggle on a dashboard but then forget to close access when finished.

Judge’s order halting JEDI work stops move to the cloud

More than a terabyte of internal data from Attunity, a data management company, was exposed by a misconfigured Amazon S3 bucket last year, including emails mentioning customers Ford Motor and TD Bank. While no sensitive customer information was leaked, Qlik, which has since acquired Attunity, now applies stricter security standards to Attunity environments, including round-the-clock monitoring, according to Qlik spokesman Derek Lyons.

An Amazon S3 error at THSuite, which makes software for retail cannabis sales, exposed customer information collected by at least three U.S. dispensaries, including names, birth dates, phone numbers, addresses and the kinds of cannabis and quantities customers bought and when. THSuite did not respond to requests for comment.

In Capital One’s case, the breach was carried out via a sophisticated exploitation of an Amazon virtual server that allowed a former Amazon employee to access the S3 data.

Amazon spokesman Grant Milne said the company has continuously added free features and protections intended to help customers avoid misconfigurations. As recently as November, it launched a feature for security teams to check that the policies governing access are functioning as intended.

California-based Elastic’s Elasticsearch is popular because it is fast and free, but its security features are disabled by default when the software is downloaded from the Internet. Developers in a hurry or without sufficient training can inadvertently leave the database unsecured.

Steve Kearns, Elastic’s vice president for product management, said the company recognizes that data security can sometimes feel like an extra step that slows developers when they are being asked to work quickly. He noted that Elastic’s paid software as a service is secure by default and that the company includes free security features for clients who download the free software — but those features need to be configured.

Choice Hotels exposed 700,000 customer records on Elasticsearch, blaming a vendor that has since been dropped. Michelle Peters, Choice Hotels’ director of external communications, said the company has “put additional controls in place to prevent any future” leaks. The Wyze breach also involved Elasticsearch.

Even if the data has been secured, breaches can occur when an application or other component with access to the data is misconfigured. The massive breach at Capital One was not the result of a simple unsecured database but rather the work of a former Amazon employee who understood the infrastructure well enough to use a vulnerability in another component, called an identity and access management module, to allow access to Capital One’s data. Similarly, Equifax’s 2017 breach, attributed to Chinese government hackers, was also done through a vulnerable component — in this case an open source tool used to build applications.

Each hack is costly. The Ponemon Institute, in a report sponsored by IBM, says breaches cost companies an average of $3.92 million each, with some costing far more. Equifax agreed to pay $700 million to settle a class-action lawsuit arising from a 2017 breach, and Capital One said its breach could end up costing the company at least $100 million. Marriott faces a potential $130.4 million fine in Europe alone for its cloud breach.

“This has touched every part of the consumer’s life,” said Kudale, the Cowbell Cyber founder. “Whether it’s staying at a hotel, or getting bloodwork done, or taking out a mortgage, or setting up a Facebook profile or using a credit card, your information can be exposed at any time.” Even if your data hasn’t been exposed, you’re paying more because breaches are proliferating.

Craig S. Smith is a former correspondent for the New York Times. He is the host of the podcast Eye on AI.