The Impact of Big Data on Modern Society

It’s impossible for the human brain to fathom the quantity of data generated today. In 2008, an estimated 14.7 exabytes (that’s a billion gigabytes) of data was created. From 2014 to 2016, 90% of the world’s existing data was created.¹ Every day, people interacting with their devices via the internet contribute to this, creating big data with every click, share, like, and swipe.²

Big data and society

The focus of big data in society is moving towards the value that organizations can extract from it. Data science is a field that has grown from this, incorporating disciplines including programming; statistics; data engineering, mining, preparation, and visualization; machine learning; and predictive analytics.³

Using machine learning technology, predictive analytics shows the value in large amounts of data. It learns from current and historical data trends to predict the likelihood of those trends recurring in the future.⁴ For example, predictive analytics can use available data about an individual to determine the likelihood of them buying a specific product, contracting a certain disease, or enjoying certain media. Based on these insights, organizations can make more informed decisions.⁵

While some may be justifiably skeptical about the implications of big data for privacy and security, big data has the potential to transform society for the better. Discover its potential and application in the following industries:

Big data in business

By analyzing large-scale social media and browsing behavior, businesses can create a more complete profile of customers and stream them into narrow segments of preferences. With this level of specificity and insight, businesses can make more informed marketing decisions to promote their product or service to those more likely to want it.⁶

Netflix is an example of a company using big data to understand their customers and target them with personalized suggestions based on their viewing history.

As much as 80% of your Netflix stream is influenced by its algorithm-powered recommendation system.⁷ This behavioral data is used to create a better experience for the customer – one they’re almost guaranteed to enjoy – and gain better brand loyalty as a result.

On the Data Analysis for Management online certificate course from the London School of Economics and Political Science (LSE), you’ll learn how to analyze, interpret, and communicate data for a competitive advantage. Watch the course trailer to find out more:

How can big data be used to understand customers?

By gathering information through customers’ buying habits, large-scale surveys, and case studies, organizations can predict customer needs and create new, innovative products and services that best meet those needs.⁸ Big data presents the opportunity for businesses to create consumer-responsive products based on precise data predictions, instead of relying on the lengthy process of customer feedback.⁹

Amazon Fresh capitalizes on this by collecting data to better understand how suppliers interact with grocers and how customers buy groceries. This allows Amazon to know whenever there is a need for change and improvement in the process or product.¹⁰

How can big data predictions help reduce costs?

For businesses stocking inventory, products, or produce, knowing when and how much stock is needed at any given time can save money and prevent waste. Big data analysis makes it possible to predict when sales will occur, helping organizations to order the precise amount of stock needed to meet demand. This results in less produce being wasted and avoids keeping capital tied up in inventory, or incurring unnecessary carrying costs.¹¹

PepsiCo relies on big data to manage its supply chain effectively to reduce costs and minimize waste.

With reports from suppliers and warehouse inventory, PepsiCo is able to forecast which retailers need which products, the volumes they need, and the time they need them – saving them money and effort.¹²

Big data in healthcare

As more data is collected and analyzed, it’s becoming possible to save more people’s lives. With patient health records increasingly being digitized, there is more and more data to train artificial intelligence (AI) to identify things like kidney disease and breast cancer, or the likelihood of leukemia remission, for example. However, there is still a lot of work needed to optimize electronic health records (EHR) and make them consistent so they can be used in algorithms.¹³

A lot of attention is focused on developing decision support tools. An IBM client integrated the AI-powered Watson Health clinical decision support tool into its EHR and saw 73% of its searches answered in less than a minute.¹⁴

Big data and AI are also being used to aid in more mundane healthcare tasks like workflow and administration. Researchers at the Center for Clinical Data Science trained a machine-learning algorithm to detect movement when patients are undergoing CT scans, which could ruin the scan. This saves the time of potentially having to call the patient back for another session.¹⁵

How big data personalizes medical care

In much the same way businesses reach customers through targeted marketing, the goal in healthcare is to have enough data on each patient to provide evidence-based personalized treatment. People have more control over their health data than before, making them more engaged in decisions around healthcare and data-sharing for research purposes.¹⁶ The Internet of Things is enabling smart home healthcare innovations. These range from glucose and heart-rate monitors to devices that track physiological changes linked to mental state, or assess how chronic disease symptoms fluctuate throughout the day.¹⁷

Data around patient demographics, lifestyle, history, and genetics can aid doctors in predicting the likelihood of future disease, judge the impact of comorbidities on therapy treatments, and even expedite FDA requirements to deliver a particular treatment or drug.¹⁸

Genome sequencing is another exciting possibility in the medical field. It took 13 years to map the first human genome; today it can be done in an hour. As big data technology continues to expand and larger amounts can be processed, everyday genome sequencing is looking like a regular possibility.¹⁹ Genome sequencing played a large part in developing vaccines for COVID-19, demonstrating the crucial role that this tool can play in healthcare.²⁰

How big data reduces medical costs

US healthcare costs have surpassed GDP growth for decades, creating a major driver for companies and government to harness big data applications in order to reduce these costs.²¹

Fraud, waste, and abuse is a major factor in these costs. Using machine-learning algorithms to scour historical records, patients files, and billing information can help detect anomalies like prescriptions being double-filled or hospital services being overused in a short space of time.

The Centers for Medicare & Medicaid Services used predictive analytics to preempt more than $210.7 million in healthcare fraud in one year alone.²²

Analyzing data points like seasonal demands and expected demographic changes can help organizations forecast demand for medication, equipment, and supplies, and ensure those resources are used optimally.²³ The Global Fund, the Bill & Melinda Gates Foundation, USAID, and GAVI have been investing in supply chain information systems for more than a decade to increase medical supply chain visibility. Many countries can now draw on three or more years’ worth of historical consumption data, enabling them to forecast with greater accuracy.²⁴

Patient, clinical, and departmental data can also yield insights into which patients may stay in hospital longer than the average time needed to treat their condition. These can be used to adjust treatment on the fly, reducing the likelihood of costly overstays.²⁵

Big data in government

The application of big data analytics in Western democratic governments is subject to much debate. While there are foreseeable benefits, democracy’s prioritization of privacy has the potential to hinder data’s potential.

It also has the potential to be exploited. One example is the use of data by China’s communist leadership. The country’s Draft Security Law categorizes data according to its impact on national security, among other things, and requires companies to conduct risk assessments on their data.²⁶

While major data conglomerates like Alibaba, Tencent, and Baidu are frequently cited for their cooperation with the China Communist Party, foreign businesses wishing to operate in China are also subject to a raft of laws. These require disclosure of information like business contracts, strategies, employee and customer personal data, and intellectual property. The Corporate Social Credit System introduced in 2020 tracks regulatory compliance, with poorly performing companies potentially suffering sanctions or blacklistings.²⁷

The Chinese government is also using data collected from millions of surveillance cameras to build profiles of its population. Algorithms are being trained to cross-reference information including purchases, reading habits, friends and associates, and travel records to identify and quash political dissent in advance. Nowhere is the reach of the government’s surveillance apparatus more evident than in Xinjiang, where Muslim Uighurs are forced to install apps on their phones that monitor for Quran verses or sermons, Arabic script, or associations with mosques.²⁸

However, while only time will tell how China’s authoritarian approach to data dovetails with its innovation ambitions, the rest of the world is by no means ignoring the potential application of big data in government sectors.

How big data transforms cities

Barcelona started implementing data-driven smart city technology over a decade ago. Today it’s recognized as one of the leaders in terms of incorporating big data into city planning and management. It’s built an ecosystem of operations centers, horizontal information platforms, dashboards, training programs and educational institutes, research centers, innovation labs, and strategic planning offices. These all help manage data collected about traffic, public transportation, power supply, utilities, emergency services, weather, social media networks, and information directly provided by the public. In the process, the city government is working to democratize its information technology infrastructure, encouraging citizens to engage in urban planning and policy development, helping to ensure that technology is used for the public benefit.²⁹

How big data collaborations assist in safety

Data-sharing collaboration may become more common in order to facilitate greater safety and security.

AEGIS is an EU-funded project that analyzes data collected from sources including wearables, cellphones, smart home platforms, environmental sensors, smart city systems, databases, and social media platforms. The goal is to better understand past events in order to predict the future. AEGIS is used by an Italian insurance company to provide early warnings for hailstorms, for example, while another application involves altering authorities and drivers to poorly maintained roads.³⁰

In Kenya, analysis of data taken from social media, cellphones, and official accident reports revealed that 50% of road fatalities occurred on 5% of roads.

This sort of insight provides clear directions for interventions that can save lives.³¹

Big data is also being used in policing and crime prevention. Risk terrain modeling (RTM) is a form of analysis that combines data from numerous sources and connects them to geographic locales, diagnosing environmental conditions that lead to crime.³² In response to a spate of convenience store robberies, the Atlantic City Police Department in the US implemented a location-based intervention informed by RTM. The intervention focused on risk reduction rather than law enforcement against people and resulted in a 63% drop in robberies over four months.³³

The risk inherent in predictive law enforcement, however, is that it can create a feedback loop that results in over-policing of communities and ethnicities prone to being victimized by the police. In 2020, Santa Cruz became the first US city to ban public use of predictive policing, despite having been one of the first to adopt it in 2011.³⁴

How big data governance empowers you

Smarter government systems mean an easier government experience. X-Road is a decentralized data exchange network developed in Estonia that automatically syncs information across government agencies.

The country’s data exchange network, X-Road, saves over 240 hours of work every three minutes by automatically syncing information across government agencies.

The aim of X-Road is to enable public services that make people’s lives easier, like automatically paying an allowance for a newborn child in Finland, for example.³⁵

In Taiwan, people are encouraged to participate in government processes, creating greater citizen engagement. Using a system called vTaiwan, citizens post on a discussion forum about new political decisions. Through this method of crowdsourcing, people are invited to post principles that should underpin legislation. Suggestions are rated by other citizens and then clustered to show statements that hold true across the nation. After further deliberation among experts, the government decides whether to implement legislation based on the people’s response, or explains why it won’t be doing so.³⁶

Stay ahead of the big data curve

The need for people to interpret, understand, and execute decisions based on big data analytics is rapidly increasing. GetSmarter offers a large portfolio of data science and analysis courses, for both technical and non-technical professionals who are interested in using data to produce powerful business insights.

Learn in-demand data and analysis skills