"It is useful to watch new disciplines emerge which realise the potential of big data, recognising that ideas and techniques may need to move rapidly from one industry to another."
PA business intelligence EXPERT
As organisations accumulate increasing volumes and diverse types of information – known in the IT industry as ‘big data’ – they are being presented with a major new commercial opportunity to gain insight into their customers and identify the patterns that will help them predict future consumer behaviour.
Data volumes are exploding. Just over a decade ago, only a small number of organisations had data warehouses that reached or exceeded a terabyte in size. Today, media companies like Netflix and Facebook – as well as some banks and telecommunications companies – have announced that their data warehouses have already exceeded a petabyte (1000 terabytes) of data and are continuing to grow fast.
The type of data being stored has also increased in complexity, from the simple transactional data of the 1990s held in a structured way within organisations’ own systems, to far more diverse data sets such as video, text, voice and email that rests within, outside and between the organisation and public domain. For example, the wealth of information available from research firms and marketplace transactions is becoming integral to businesses’ strategic decision-making, and as a result, this data finds its place in enterprise data warehouses. As this data increases in scale, so does the opportunity to use it for commercial advantage.
So what are the main issues that companies need to face down to gain command of the opportunities available with big data?
Using data to get the most from limited assets. Earlier this century, most organisations’ data analysis would have been limited to the simple historical reporting of what was sold, by whom, how often and where. There is now a growing need for organisations to perform far greater data analytics on their data sets. The goal is to not only gain insight into how people use their products but, more importantly, to start to predict what may happen next.
Over the next five to ten years as tools and techniques develop we will see a progression from the current position where organisations see questions which are too hard to answer, to organisations seeing ‘solvable questions’ – moving from a state of chaos in big data to a state of order.
However, to realise commercial advantage, organisations need to be able to use big data to get the most of their limited assets. This will vary by organisation. For example, a telecoms operator’s key asset will be its network capacity. Therefore, big data can be used to understand how to balance the load across a network to respond to peaks and troughs in demand for key events such as sporting occasions or to better understand the investments required for the advent of 4G technology. Retailers are analysing detailed transactions to fully comprehend customer shopping patterns, forecast demand and optimise merchandising. This can help them maximise the return on investment from marketing spend. Other organisations which are benefitting from big data are internet providers. They are documenting how consumers surf their websites so they can enhance visitor experiences and improve the efficiency of targeted advertising.
Organisations need to invest and develop their understanding of the potential impact and opportunities this new technology creates. It is useful to watch new disciplines emerge which realise the potential of big data, recognising that ideas and techniques may need to move rapidly from one industry to another. Google is building a service to allow the analysis of large amounts of data in the cloud. The service, which is called BigQuery, would help organisations analyse their data without the need for building infrastructure.
Coping with an ever-diminishing pool of skilled resource. While technology and data volumes are expanding at a substantial rate, research has highlighted the shortage of workers with deep analytical skills. Finding the right people who can undertake analysis within the commercial context of an organisation – especially at the right price – could prove the biggest barrier to organisations benefiting from their big data.
This is a challenge due to both the jump in demand for these skills but also because the skill set to exploit big data has several components that need to be applied in a balanced way. Firstly, individuals require the technical skills to find patterns and draw conclusions from the data. Secondly, they need to use the conclusions to draw a set of hypotheses that are linked to the commercial drivers, and lastly they need to be able to present back to senior executives in a compelling and realistic way. Executives should be thinking about their critical hires now to help jump-start or improve the success of their planned big-data initiatives. They also need to recognise that the individuals will require investment, nurture – and some patience – to get results.
Consuming big data using mobile business intelligence tools. In line with the shift in the type of data, we are currently in the middle of a shift in the way we consume business intelligence (BI), from static paper reports and desk-based analytics to mobile devices. The development of interactive dashboards, that provide the ability to drill down and access more detailed transactions, via a mobile device will become more mainstream in future.
Research by the Aberdeen Group has shown that organisations that use mobile BI believe it gives them a competitive advantage. Executives were also found to value tablets and smart phones over PCs; making BI available through these devices has helped make BI more visible to top management and has driven its adoption. For example, CFOs could have critical financial metrics or sales executives delivered to their iPad, with access to supporting interactive charts, graphs and maps.
Dresner Advisory Services’ Mobile Business Intelligence Study November 2011 noted that BI is ranked in the top three high demand mobile applications, after email and personal-information management (using calendars and scheduling). The research also demonstrated that this is a phenomenon taking place in organisations of all sizes, across all geographies, industries and functions.
Some of the fastest growth markets for mobile BI are in the health and retail sectors. Clothing retailer Guess uses mobile BI technology to allow their buyers to drill into their data to see forecasts, goals and historical trends while they are on the road. This is because key decision makers are typically not desk bound but still need for up-to-date data and analysis. Delivery of this through a mobile device allows them to make quicker and more critical decisions.
The techniques and approaches are set to rapidly develop over the next ten years. Organisations should look for first mover advantage and be prepared to invest in pushing the boundaries. Organisations with the correct understating of what is possible, aligned to the correct strategy and supported by skilled resource, will gain competitive advantage over their peers.