Data analytics mountain

Climbing the Analytic Maturity Mountain

This is the first follow-up article in my series about trends in data analytics. The concept of analytic maturity follows a simple pattern we see reproduced across businesses and industries; as a given technology or business process becomes common, companies seek competitive advantage by extracting value from more challenging sources. With analytics, the progression of source data looks like this:

Structured Data → Semi-Structured → Unstructured → Unified

Since the dawn of the computing age, structured data has been a primary focal point. You would expect this to be the easiest type of data to analyze, because the well-defined schemas describe the intended use. However, just ask any database administrator: data quality and consistency issues quickly creep into even the most carefully maintained databases. As a result, increasingly sophisticated approaches have been developed to maximize the value of the data. For example, the field of business intelligence has historically been based almost exclusively on structured data, and has brought big gains in understanding and organizational efficiency. Similarly, predictive analytics for fraud detection and decision trees rely primarily on structured data.

Semi-structured data, such as log files or other forms of machine or sensor-created data, is more difficult to work with. It often doesn’t have a single format, and may not even have clearly defined fields. The primary new challenge here is to parse this content and create a structured representation of the data. Where structured data had some quality issues, semi-structured data is typically significantly less clean, consistent, and complete. Despite the challenges, this is one of the fastest growing areas of analytics. A Google search for Log Analytics shows more than 115 million results.

Unstructured data typically refers to images, videos and textual content, which may or may not have useful metadata associated with it. This near-complete lack of structure makes this the most difficult type of data to effectively analyze. There are a number of sophisticated approaches which attempt to identify or extract key pieces of metadata. Companies are using these advanced techniques to identify sentiment in social media, track stock-related news to power automated trading systems, and provide high quality search, exploration and navigation capabilities. The capabilities in this area are still developing quickly, and I’m excited to see where it goes next.

The future, however, is almost certainly unified information access, which ties together data of multiple types, across silos to provide a single picture of an organization and how it interacts with customers, suppliers and employees. The concept of bringing data from different systems together isn’t new – in the structured data world, this was why data warehouses were created. However, by spanning silos and bringing different types of data together, businesses can answer more complex questions than ever before, and capture the insights that matter. This allows you to put your content in context, and leverage the mountains of data your organization is collecting.

In short, analyzing the data is difficult, but once you master the data and begin extracting actionable insights, the fun really begins.

So what do you think? Where are you seeing the value of analytics in your business?

  Like This

Steve Kearns

Steve is the Director of Product Management for DataGravity, focused on defining and delivering Data Intelligence. He has spoken at conferences around the world about the power of search and analytics and has worked with many of the worlds most successful companies and government agencies implementing these technologies.