What it is all about?
Real time analytics is about acting on data in the moment of business need. In reality it may happen right at the time of an event or close to it but this is becoming a pressing requirement for the data teams within enterprises to deliver in order to enable digital transformation and helping business deliver high personalized product and services in real time to stay ahead in competition. The data required for making a business decision or action in the moment either by human or AI can come from variety of fast evolving data sources such as cameras, sensors, IoT devices, social media and so on.
Let us think through examples?
Let’s quickly think through some examples, when a bank requires to detect a fraudulent transaction at the time of approving your credit card payment at a restaurant etc for your own safety they need to rely on data availability and accuracy in real time to make such a decision. Such a decision if made correctly can save them from otherwise millions spend on tracking, cancelling and rectifying such a transaction. Similarly, Facebook may would like to moderate and block a inappropriate content either presented to a user or public preferable before it is done or as soon as it is done to protect its users and its own legal compliance requirement. No moving the example needle away from tech to a unique real scenario would help you expand your understanding around the value of real time date and actionable AI. The wildlife insights program uses machine learning to analyze video data taken in South Africa Kruger’s national park to automatically trigger a suspicious activity that can lead to poaching.
Emerging era of continual intelligence
Globally close to 3 quintillion bytes of data is generated every day, and more most enterprise storing the old data is becoming cost inefficient. Given shelf life of data is shrinking the new modal is for businesses to instead invest in keeping data fresh and relevant for making a decision in real time so that business can stay close and responsive to customer and act in time thus deepening their high personalized customer relationships. Now picture the same philosophy extended for employees, and vendors and so on. Thus, real time data can and will continue to help enterprise position their competitiveness to thrive in digital economy.
Increasing use of real time AI by business businesses to enable process and up their customer experience demands for a well thought and calibrated long-term real-time data strategy for your enterprise or business. Let us quickly scan what’s happening around us AI would soon be driving cars, even fly aircrafts, create personalized conversations, delivery real time adaptive and hyper-personalized customer experiences. Take an example of ChatGPT and Stable Diffusion which have created a ripple effect in the world of businesses on how AI can be used and applied for business advantages.
Real time data need to become a foundational strategy within your big picture enterprise data strategy. As more and more devices and applications would become a part of integral of your business data ecosystem to be able to drive business decisions, customer experience and stay competitive within your industry. Let us take an inventor of some very easy to understand examples
We see 24/7 devices all around and across industries, such as doctors using these devices to monitor patients, security systems using these to monitor potential theft or threats, manufacturing plants are using these to prevent near misses and hazards from happening, and more. Thus edge computing is increasingly becoming important for your business to explore and capture value, foundation of which resides in real time data provisioning.
Chat bots are becoming popular and you see them pinging you whether you be on a website or just browsing internet. These chat bots are trying to talk to you, engage you in the very moment you may be looking for something either for your research, purchase or networking.
How many of us are getting more and more involved with games, and gamification around our day to day lives. As an example, a huge number of apps are mushrooming to offer a gamified learnings around wide array of learning topics, including language, or deep scientific subject. Your business need a right gamification strategy to engage your users, customers and employees which is empowered by AI and which ultimately relies on real time data.
Cloud native apps, mobile apps and right microservices help your business innovate its approach to pricing, partnerships, customer experience and more. The rise of connected world is demanding for connected experiences for your customers, which requires data to flow real time across systems within and beyond your enterprise.
Do not be left behind
Many enterprises are already making an effort to lead forward in building their real time data and AI capabilities. As an example, Walmart is building world’s biggest private cloud capable of processing 2.5 petabytes of data every hour. Building real time intelligence capabilities can help enterprises capture higher customer value that its competitors while saving tremendous amount of money else spent after the effect of a event may it be a fraud, loosing a customer sale, or a employee accident. All such events can be regulated in favor of enterprise goals and profits by creating and deploying a solid real time data capability that empowers AI and its usage.
How to build such a capability?
Define business needs
Before even you look into engineering or technologies around data or even into the data itself ensure that you know and you have well defined business goals and needs in place. Such examples could be what are the key goals of business, such as grow revenues, capture high market share, expand into new territories, reduce cost, or drive customer retention. These key goals will help you discover needs around your high value use cases, which is the next step.
Define high value use cases
Given that you are clear on the most pressing business goal, now its time to define or list down use cases which will enable these business goals. As an example if your company operates across diverse channels to sell and engage customers, and one of your key business goal is to increase revenue then you would like to identify means to increase revenue from existing customers while chasing new customer conversions faster across the marketing funnel. In order to do you will review your business’s upselling and cross selling strategies for existing customer across channels such as contact center, online shops, brick and mortar retail, and others. You would also look at your demand generation engine across channels, may it be email campaigns, social media campaigns, customer referrals and more to explore how would you ensure that you are not missing our potential prospects from becoming customers to your competitors. Given you high value uses and business needs around these are in place now its time to move on to map the required data sources.
Map data sources
For the prioritized high value use cases you will now request your SMEs, Project managers and data analysts to map required data to enable the use case. As an example if you want to go all our for upselling and be able to offer recommendations to your customer by offering customized offers in the moment when they are at the shopping cars on your ecommerce website, or over a call with you sales contact center you may now would need to think what data can help AI algorithms on your ecommerce website and chat bots or humans in contact center to be able to make these offers made available to your customers in that very moment. You will need to query and explore all your data bases within enterprise or beyond enterprise including second party and third-party data to map these to each use case. Some of these data sources could be your ERP, CRM, and Third-party API. Once the data mapping is done you know now what data is needed to fuel your use cases and thus now its time to move on to considerations behind managing and storing the data, which is your data infrastructure.
Data Infrastructure
Now as you know the data that needs to fetch for your target use cases, you can quantify and size up volume, variety and velocity of the data that your systems need to handle. Based on these three measures you can ask your IT department, Data architects and Cloud vendors to help you design a data ingestion, storage and analytics infrastructure. This infrastructure will leverage your existing and as need new tools, including open source to save your cost to implement these new data driven use cases. Two key components of your real time data infrastructure would be ETL or ELT tools, along with target storage system such as a data warehouse, data lake or data lake house. Now is the time for final step which involves choosing a real time analytics engine which will analyses the data and make available the insights for the end action which could be a recommendation engine on your website, an edge device such as camera to trigger an alert, or a report published on a laptop or mobile device.
Real time data analytics tools
Choosing a real time data analytics is a function of context, users and data volume. You need to start from users to see who these users are and what will they use the data for, make sure its ease and intuition that matters more than data itself for end users, and finally you will have to factor in security and governance of data as well. A wide number of analytics tools are available to choose from and you will have to undertake a well thought assessment to pick the one that aligns with your unique needs. Some examples are snowflake, data bricks, big query, athena, redshift
