• Phone
  • Contact us
  • Locations
  • Search
  • Menu


  • Add this article to your LinkedIn page
  • Add this article to your Twitter feed
  • Email this article
View or print a PDF of this page

Breaking convention to mine real value from big data

The volume and diversity of data available to organisations, and the speed at which it can be processed, is increasing rapidly. Partly driven by hype, such as the New York Times heralding the ‘crossover year for big data’, many businesses feel pressured by vendors to upgrade their technology with a promise of improved analytical capabilities, better insight and crucial competitive advantage. 

Technology alone, however, is unlikely to deliver on the big data promise while businesses continue to use a traditional supply-driven model for business intelligence and analytics (BI). Typically, in this approach, all data is first loaded into industrial-strength structures, such as data warehouses, before insight can be generated. Although this offers high levels of quality control, security and maintainability, it comes at the expense of agility. As a result, many businesses fail to get the answers they want quickly enough from their big data investments. 

To become agile enough to deliver real advantage from big data, organisations need to break the conventions of supply-driven BI and adopt an approach that is responsive to business demand. Our approach draws inspiration from the way the oil industry explores for new resources, only investing in infrastructure once it knows it has struck a well.

Focusing investment on viable propositions

A low-risk exploration phase – enabled by smart use of technology, employee skills and agile processes – can identify early on the most likely sources of value. This is an effective way to contain the costs associated with exploring and exploiting big data. Similarly, because the industrialisation of data extraction only begins once the business has articulated its needs and prioritised the areas it wants to develop, investment in infrastructure and change is focused and is therefore less expensive and more effective. For a leading Dutch retail bank, we used this approach to spearhead the development of new multi-source reports to improve IT service effectiveness and efficiency. Doing this ahead of the development of a data warehouse led to a reduction in risk and delivery times.

Gathering insight for the business more quickly

The wider remit afforded by the exploration phase gives the business scope to focus on new data sources and ask incidental, relatively unstructured questions, such as: ‘How do we match what makes our customers tick with what we offer them?’ Because the ‘right’ discussions are taking place – the ones that focus on the issues the business really wants to address – greater, more-valuable insight emerges, more quickly. The report prototypes we delivered to the Dutch bank allowed for the early realisation of cost savings up to 15% as well as improved customer satisfaction levels before the launch of the final product.

Sustaining a demand-driven model

As the influence and uptake of new technology grows in every area of society, so too will the volume and complexity of data available to organisations. Those using a traditional BI model – loading all data into industrialised structures before beginning analysis – will struggle to keep pace. Selectively loading data to explore specific business questions before industrialising the loading process represents a more sustainable option.

We have used this demand-driven approach to help clients in a number of sectors get the answers they want from business data. For a leading European insurance broker we explored existing but un-mined data to give our client the insight they needed to define customer strategy based on value. As a result of our work, cross-selling, up-selling and retention all improved, which justified implementation of a more efficient and maintainable solution.

Our wider experience helping organisations understand and make the most of big data includes working with the UK Met Office to build the cloud-based Weather Observations Website (WOW) that has so far collected 20 million weather observations from six continents for a tiny infrastructure cost.

To find out more about using big data to answer your organisation's key commercial questions, contact us now.

Tom McEwan
IT consulting
contact us now

By using this website, you accept the use of cookies. For more information on how to manage cookies, please read our privacy policy.