WHERE NEXT FOR BIG DATA
The rise of big data collection, aggregation, analysis, and consumption has seen a rapid growth in organisations seeking understanding of what happens in their business. While there have been an increase in technology solutions to assist these organisations in that regard, with a key focus on extracting and presenting data in a more consumable way, organisations may still struggle to make sense of the information.
Many of those solutions can slice and dice data leading to numerous combinations and permutations of dependent variables, often leading to confusion rather than clarity. That has led to the growth of data analyst roles and data scientists whose job is to make sense of all the data by extracting knowledge or insights that will enable organisations to gain efficiencies, refine service delivery processes, and improve competitiveness.
However, these data scientists are generally not involved in daily business operations mimicking a trend seen in IT departments from the dotcom boom over 15 years ago. The struggle lies with them understanding the business well, rather than just their domain, in order to gain insights that are genuinely that _ insights, than merely analysis and reports of co-ordinated or even connected facts.
To configure a report or dashboard surfacing big data, there are four key attributes that need to be present: 1) Understanding what data is available, 2) knowing where it is, 3) identifying what data points to present, and 4) knowing how you want it presented. The underlying premise of this process however, effectively limits the real insights to be gained because the attributes are determined via pre-conceived notions of what you need and what may be valuable. This is the missing ingredient of almost all discussions around big data: how do you get to know what you don’t know?
Some people feel Artificial Intelligence (AI) can address this problem while others, not wanting to make such a quantum leap in complexity and cost, are pinning their futures on variable dashboards and data visualisation. In all these solutions, the search for insight remains elusive with many sweeping assumptions needing to be made from the data in order to apply corrective or enhancing changes in the way they are doing business. The assumptions and attribution can then only be validated after having made the changes and re-assessing the new data. What remains therefore, is not much better than what most organisations did before without the big data!
Such is the case in digital marketing where the nirvana of a single view of the customer across all systems and interactions is seen as the answer to successful marketing. Such a solution of integrating all systems containing customer data remains complex and costly, and that is just to expose the data in a single view. It still does not tell you why and how customers took the actions they did. So while your data may be connected, the data points are not reflected as relationships in that there is no meaning inherent in the connection i.e. lack of understanding between cause and effect.
A good but simple example of this is a customer name, an address, and a purchased product such as a walking stick. These data points can and are easily connected. However, a relationship link which indicates that the customer is too old to drive, lives 2km from the nearest public transport, and has also bought orthopaedic shoes, suddenly provides an insight that could easily lead to much more personalised and targeted marketing. The key point here is, while there is information available in Business Intelligence (BI) and reporting solutions to display the data, the data points required to further provide insights would in all probability never be configured to report such insights in the first place. That is because you don’t know what you don’t know.
The answer lies somewhere between BI and AI in the realm of semantic processing. Semantic processing of linked data allows data that is just a string to become a useable entity. For example, my name “Stephen” means nothing to a reporting or BI solution: it just lists my name as configured to do. However, when my name is processed semantically and linked to other data, the sentence “Stephen wears a watch” immediately identifies Stephen as a person. That then provides a basis upon which other links can be established such as “where does Stephen live”, “is Stephen married”, or “does Stephen have a job” because those all make up properties of a person’s profile.
In this manner, data can be semantically processed and linked to provide relationship insights that are otherwise unknown to the user. Solutions which provide this capability, including Latize Ulysses, are not yet predominant in the marketplace but the very real gap in gainful insight is driving the search for such, and other answers from big data.
