I have long considered encryption to be the primary method of protecting cloud data. Now, at a time when a convergence of factors is putting more strain on companies in safeguarding their data, a promising new technology, microsharding, has come along that should cause companies to examine whether there might be a better path forward.
Most companies are well along the journey of migrating their data and applications to the public cloud. The COVID-19 pandemic has prompted many organizations to accelerate their cloud usage, given the convenience of cloud access for their increasingly remote workforces.
This is all unfolding at a time when data protection and data privacy have never been more important, and privacy regulations such as GDPR and the CCPA have further elevated their prominence. The recent EU court ruling striking down Privacy Shield makes a challenging data protection and compliance environment even trickier.
As a longtime veteran of the industry, it’s no surprise that data protection has emerged as a hot area, but even so, I continue to be amazed at the extent to which professionals and organizations are recognizing its importance, and acting accordingly.
I serve on the board of directors for global non-profit technology association ISACA, and the new technical privacy certification that ISACA released this year, Certified Data Privacy Solutions Engineer (CDPSE), has generated a huge response, reinforcing the growing emphasis that is being placed in this area. For enterprises around the world, increasingly, a central business imperative is determining how to best protect their organization’s data while also ensuring their data that is stored in the cloud is compliant with data protection and privacy regulations.
This is where microsharding comes into play. Many security practitioners are familiar with sharding, which has long been used by storage and database companies like Oracle, Altibase and MongoDB, and tools like ElasticSearch and MySQL, to improve performance by splitting files into multiple pieces and storing them in different locations so that input/output (I/O) can be done in parallel to make the process faster. Normally these pieces are a few thousand to a few million bytes in size.
Microsharding, a newer approach used by companies such as ShardSecure, also splits a file up into multiple pieces, but the pieces are extremely small. This could be as tiny as a single byte, but practically speaking, each microshard tends to be a few bytes. Each of these microshards are stored in different locations – they could be dispersed across multiple cloud providers or even be stored in on-premise locations.
The benefits of microsharding are multi-faceted – not only in meeting the critical need to effectively protect the data of customers and employees, but also from a regulatory standpoint. As a data privacy and security professional, I like the idea of reducing the attack surface – a key advantage with microsharding.
Microsharding, in my view, also reduces what is in scope for data sensitivity; if the data has been shredded to the extent that a bad actor can’t even extract a credit card number or Social Security number, is it still sensitive data? I would contend the answer is no, which can dramatically reduce companies’ data protection burden and the cost of compliance.
With microsharding, if the cloud storage admin is hacked – a common concern in complying with regulations such as GDPR – the data is not in jeopardy. In a bigger picture sense, in almost all cases with adding privacy and security, there is a performance cost – doing so tends to slow things down. Microsharding represents a rare case in which layering on additional data protection also improves overall performance.
There are also distinct benefits of microsharding compared to utilizing encryption. While encryption generally provides good protection, microsharding allows companies to avoid concerning themselves with the key management concerns that accompany encryption.
Additionally, certain encryption algorithms might not be quantum-resistant – an increasingly important consideration with quantum computing looming as a probable game-changer for the industry. Imagine an organization with terabytes or petabytes of data that has to go through the process of decrypting and re-encrypting, at tremendous cost, due to broken encryption.
In my view, any organization that is dealing with large amounts of unstructured data – for example, using Microsoft 365 applications that include research and sensitive data – should strongly consider microsharding, which can be particularly effective in sectors such as healthcare and for law firms.
Financial institutions also would stand to benefit, but many of them are so tied in with their current encryption technologies that, given their risk-averse nature, they are less likely to be early-adopters for their core business practices.
So, how soon will microsharding catch fire on a large scale? We’ve seen enough case studies in emerging technology to anticipate that it won’t be immediate – with any new technology, it takes time to gain widespread acceptance beyond the early adopters. That said, the timing is right for microsharding to become a promising alternative or supplement to encryption.
Even if the pandemic subsides in the coming months – as we all hope it will – the trends toward more remote work and growing reliance on cloud data storage are not going away. Microsharding can be a significant piece of the equation as companies continue acclimating to this new era.