The explosion of data is predicted to reach 175 zettabytes by 2025 and is increasingly stored across disparate, hard-to-access, silos. Visibility in the current ecosystem is poor.
This growth of data has been fueled by market forces that look to capitalise on the value that can be extracted from the valuable resource. This is mirrored by the dramatic shift to the cloud and edge.
An estimated 90% of this data is unstructured information, like text, video, audio, web server logs, social media and more. And, all this data can’t be moved to a central data store or processed in its entirety.
Now, unstructured data management, for data-heavy organisations, is an enterprise IT priority. They need to identify, index, tag and monetise this information.
Komprise, an intelligent data management and mobility company, says it’s offering can solve this challenge: “We dramatically save our customers money by tiering cold data to the cloud, in a transparent, native AI/ML ready solution that doesn’t sit in front of the hot data,” said Kumar Goswami, CEO and co-founder.
“Data management doesn’t work if you sit in front of the data, as you don’t want to compromise access speed and reliability,” he added.
Hot and cold
The difference between hot and cold data is defined by the company – what data they still use to drive value and what data they don’t.
“The boundary between hot and cold data is based on what can you index and search, and what the customer defines. The customers can set different policies for different data sets,” explained Krishna Subramanian, COO and co-founder.
Intelligent data management
The first problem Komprise solves is providing insight into their customers’ data – where it is and the level of importance.
The hot and cold data is then defined, indexed and tagged, across disparate data silos; and this can be moved between on-premise and cloud via smart migration.
The tiering of this data leads to dramatic costs savings, up to 60% for IT infrastructure and data storage, through data tiering, data replication, data migration and capacity planning.
An additional feature is the onboarding of this unstructured data to a single view platform, allowing organisations to extract value through deep analytics. What they call, the Global File Index, for true cloud transformation that also enables legal compliance, governance and security.
Unstructured data management is transforming as a category, moving away from storage
Global File Index
The Global File Index allows for the continuous search (find image files), execution (copy the right files) and enrichment (tagging to extract value) of data.
The speed at creating new applications for unstructured data is also accelerated to days, as opposed to months with the custom workflows on the Global File Index.
Deep analytics action use cases
One of Komprise’s customers, Matt Madill Senior Storage Admin at Duquesne University, said: “We see a lot of use cases for Deep Analytics Actions at the University. For instance, different research groups have unique requirements which users can support with tagging, so that those data sets can not only be discovered easily but they can apply the appropriate data management policies to them for long-term storage.”
In action, the results of this use case were:
Accelerate unstructured data analysis pipelines
- Find 2TB across 10PB of data across multi-vendor NAS and cloud that belongs to specific experiments by specific researchers
- Find just the data related to self-driving cars by traffic lights across multiple data stores
Delete Obsolete Data
- Delete emails by ex-employees that have not been read in 3 years
Comply with Regulations
- Identify just the data that needs to be retained and move it to an object-locked cloud bucket
- Find original raw data files spread across multiple systems to avoid “Quality of Concern” during regulatory inspections
Enable “User-Driven Data Management”
- Users can identify and tag specific data sets they want moved to the cloud for a new study
See also: Unstructured data will be key to analytics in 2022 – Kumar Goswami, CEO and co-founder of Komprise, provides his predictions for data management trends that will emerge in 2022
