Due to several more recent developments, Big Data is experiencing a surge in popularity and is receiving investigation and investment. In fact, as many enterprises embark on a digital transformation of their organisations, it has become clear that factoring in the capabilities to safely, accurately and productively store, process, manage, analyse and leverage data is a crucial element.
Nine out of ten C-suite executives today consider data to be the fourth factor of production, as important as the traditional three pillars of land, labour and capital.
It, therefore, comes as little surprise that global investment in Big Data is expected to grow by 17 per cent CAGR to reach $76bn by the end of 2020. Organisations are turning to the masses of information available to them to improve the customer experience, minimise risk, optimise operations, enhance business models and, more accurately, track market trends.
But where has all this underlying data come from? The answer is largely unsurprising. More people than ever before have access to the internet, or own a mobile or computing device, and many have several devices transmitting data all at once. At the same time, data storage options offer ever-greater capacity at lower cost, assisted by the advent of advances such as data lakes and the Cloud. New regulation has also had a role to play. In the financial sector, for example, being able to identify, capture and leverage data is vital if reporting requirements are to be met.
In all industries, customer expectations for tailored and personalised services have soared, alongside heightened demand for visibility over their accounts, transactions, delivery of goods and convenience of use.
This means companies need to store and quickly access vast amounts of data around these services, effectively in real-time. At the same time, banks and corporates may find themselves up against the limits of their legacy systems - requiring further technological investment.
These drivers and challenges are contributing significantly to corporates' and banks' intensified focus on Big Data, but so, too, is the potential for reward. For example, Big Data has the potential to improve business performance by an average of 41 per cent.
With such a clear incentive, adoption rates are higher than ever, with adoption typically broken down into four stages: education - focusing on knowledge gathering and market observations; exploration - developing strategy and roadmaps based on business needs and challenges; engagement - piloting big data initiatives to validate value and requirements; and execution - deploying two or more big data initiatives, and continuing to apply advanced analytics.
When surveyed, 47 per cent of respondents felt that they were in stage two, (exploring and strategising), with only six per cent positioned to execute Big Data initiatives.
However, the coming years should see most corporates attempting to understand and utilise Big Data to improve their businesses. In the Middle East in particular, as it cements its position as an international financial hub, Big Data capabilities should prove to be a key differentiator of trade performance and financial services in years to come, from tackling cybercrime and money laundering to improving KPIs.
In addition, the Middle East is a gateway to Africa in terms of both trade and banking, where many developing nations - free of legacy systems - will be in a position to leapfrog developed countries' technologies thanks to quickly-implemented access to internet and mobile devices.
Putting theory into practice
In the banking industry, Big Data has already seen significant application resulting in better service, lower costs and safer operations. The retail sector has seen improvements to the loan industry - still one of its most vital roles - specifically in the process of credit scoring. For example, audience insights company VisualDNA has developed a way of credit scoring loan applicants via a 10-15 minute psychometric test that helps profile the borrower.
By establishing the applicant's attitude to risk via this data-driven method, lenders have improved loan default rates by up to 50 per cent.
Big Data has also been used to break down banks' internal siloes, by collecting and aggregating customer information across internal boundaries - from checking accounts and mortgages to wealth management relationships. Banks can now centralise customer data, putting themselves in a better position to read that data and then improve customer service with more targeted solutions.
In both retail and corporate scenarios, Big Data can enhance risk mitigation and cost savings. The insights gleaned from data can reduce credit card fraud risk, in turn generating lower overhead costs, and allowing banks to offer lower interests rates. In fact, through high-speed (almost real-time) data analytics, large banks can determine at the point of sale whether a purchase is legitimate.
Equally, algorithm-based scanning allows for early detection of risky behaviour, for example that of rogue traders. And through the assessment of global credit card information and other transactional data, alongside economic statistics, banks can gain up-to-date overviews of consumer trends - which can in turn be offered to their corporate clients.
Such information can identify new trading patterns, the financial or trading hubs of the future, and the ebb and flow of demand through various trade corridors at any given time. By generating and offering such insights to corporate clients, banks can advise them on where to invest in new plants, subsidiaries or outlets.
Convinced of its transformative potential, Deutsche Bank has made significant investments into Big Data capabilities across all areas of the bank from incubator labs to fully-fledged products.
The bank currently leverages these capabilities to further develop its risks platforms - for instance, analysing P&L and market risk and Volcker Key Performance Indicators. It uses data analytics to "teach" machines what is normal and abnormal in order to more quickly highlight errors and minimise false positives, bolstering anti-money laundering efforts.
Big Data also helps generate automated reports straight from the data source, bypassing a need for dedicated reporting systems, and it can offer more accurate and transparent pricing options through increased and up-to-the minute visibility.
Already, the Middle East has seen forums, events, conferences and the like around Big Data and its various uses and applications; but the majority of developments in this area have yet to come and it remains largely in the conceptual phase.
Beware the pitfalls
While Big Data is still in its early days, there have already been some hurdles to overcome. Chief among these has been the occasional fire it has come under for trespassing against personal privacy. While de-identification has always been an aim, it has been proven that traditional methods can fall short of this goal.
Organisations have used several methods to attempt to protect consumer data - such as anonymisation, pseudonymisation, encryption, key coding and data-sharding - to allow analysis to continue without infringing on customers' rights.
However, in the US for example, just three key pieces of information - namely birthday, gender and zipcode - are enough to re-identify a specific individual in 87 per cent of cases. So there is still a long way to go.
Social media has caused a significant shift in public attitude towards the sharing of information and protection of privacy - but attitudes towards personal privacy in the web context still diverge greatly from expectations in the banking space.
Corporates and banks that continue to leverage the power of Big Data going forward must do so with privacy in mind - not only to comply legally, as regulations will likely change, but also to behave ethically and in a manner that safeguards their reputation and public trust.
This mindset must come to bear on the security of storage methods, on the methods of de-identification, and on the ways in which the data is combined, analysed and leveraged.
In addition to the question of ethics, Big Data can hit other snags - again, often complexities that may be ironed out as the technology matures. There are, for example, the technical challenges of data management, from collecting to sorting and analysing data sets. Among these obstacles are how to process or utilise the majority of data, which is still being captured in unstructured form. Similarly, sheer volume will count for nothing where the data is not of sufficient quality and methods must be developed to separate data of value from the mountains of information stored every day.
Often corporates can put great faith in the accuracy and detail of their knowledge and control through Big Data, when in actuality their data is from the wrong sector, geography or even time-period.
Furthermore, most data sets will contain systematic biases, depending on how the data was gathered and from what section of people. Even if the right demographic is represented, there still remains a skew towards the measurable information that can be collected. Both the data and the interpretation can lead to blind spots and errors of comprehension, which can in turn be caused partly by a lack of training and expertise in this arena. With rising demand on human resources for this sector, we may be approaching a dearth of skilled experts.
That said, these are all short-term considerations. In the long-tem, Big Data will undeniably be a powerful force for improving business performance and gaining an improved understanding of the marketplace.
Corporates and banks that wish to remain competitive and implement a successful digitisation strategy must come to terms with the Big Data phenomenon and explore how best to grapple with its technologies and implications.
Big Data should continue to evolve rapidly - and those who learn to leverage it successfully will grow alongside it.