How Big Data is Reshaping Society

7 minutes   |  BUSINESS & MANAGEMENT, SYSTEMS & TECHNOLOGY

Big Data Banner Big Data Banner

It’s impossible for the human brain to fathom the quantity of data generated today. In 2003, the world had created a total of 1.8 zettabytes of data. In 2011, that same amount was created every two days.1 Now, in 2018, “over 2.5 quintillion bytes of data are created every day, and by 2020, it’s estimated that 1.7 megabytes of data will be created every second for every person on earth”.2 With each click, share, like, and swipe, society is creating big data.3 Every day, billions of people interacting with their devices via the internet contribute to a world of valuable information.

As Meglena Kuneva, European Consumer Commissioner said, “Personal data is the new oil of the internet and the new currency of the digital world”.4

The Value of Big Data

The focus of big data in society should not only be on the extraordinary volume of information, but rather the value that organisations can extract from it. Using machine learning technology, a field of data science known as predictive analytics shows the value in large amounts of data. In a nutshell, predictive analytics learns from data to predict the way individuals will behave in the future. Machine learning detects patterns in data sets to consider the probability of certain outcomes. For example, the predictive model uses everything known about an individual to determine the likelihood of them buying a specific product, contracting a certain disease, being influenced by an economic trend, or any desired outcome. Based on that insight, organisations can make more informed decisions.5

Sandy Pentland, Director of MIT’s Human Dynamics and Media Lab aptly states, “It’s not that it’s big or that it’s fast. The part that’s important is that it’s about people…. By understanding these things, we begin to understand society and social interaction in ways we never could before”.6

While some may be sceptical about the implications of big data on privacy and security, big data stands to transform society for the better. Discover its potential and application in the following industries:

Business

Analysing large scale social media and browsing behaviour, businesses can create a more complete profile of customers and stream them into narrow segments of preferences, likes, and dislikes. With this level of specificity and insight, businesses can make more informed marketing decisions to promote their product or service to those more likely to convert.8

Netflix is an example of a company using big data to understand their customers and target them with personalised suggestions based on their viewing history. As much as 80% of your Netflix stream is influenced by its recommendation system powered by algorithms. This behavioural data is used to create a better experience for the customer, one they are almost guaranteed to enjoy, and gain better brand loyalty as a result.9

Personal data is the new oil of the internet and the new currency of the digital world.

Meglena Kuneva
European Consumer Commissioner

How big data helps understand customers

By gathering information through customers’ buying habits, large-scale surveys, and case studies, organisations can begin to create new innovative products based on what customers are looking for. Companies can predict what needs customers are wanting to fulfil, and create products that best meet those needs. It’s the first opportunity for businesses to create consumer-responsive products based on data prediction instead of relying on the lengthy process of customer feedback.10

AmazonFresh and Whole Foods Market capitalise on this by collecting data to better understand how suppliers interact with grocers and how customers buy groceries. This allows Amazon to know whenever there is a need for change and improvement in the process or product.11

How big data predictions help reduce cost

For businesses stocking inventory, products, or produce, knowing when and how much stock is needed at any given time can save money and prevent waste. Big data analysis makes it possible to predict when sales will occur, helping organisations to order the precise amount of stock needed to adhere to demand without wasting produce, keeping capital tied up in inventory, or incurring unnecessary carrying costs.12

PepsiCo relies on big data to manage its supply chain effectively to reduce costs and minimise waste. With reports from suppliers and warehouse inventory, PepsiCo is able to forecast which retailers need which products, the volumes they need, and the time they need them, saving them money and effort.13

Healthcare

By 2020, IBM predicts a 20% decrease in patient mortality.14 As more data is collected and analysed, it will be possible to save more people’s lives. With artificial intelligence (AI) able to understand questions, read through 200 million pages of data, and provide an answer in seconds, doctors can consider all available resources before making a decision regarding a patient’s condition. This can change the nature of emergency rooms in the future.15

In a consultation context, medical professionals will be able to analyse past trials, trends, and current data. With this information and access to data concerning a patient’s lifestyle, history, and genetics, a holistic picture can aid doctors in providing the most effective care.16

How big data personalises medical care

In much the same way businesses reach customers through targeted marketing, the goal in healthcare is to have enough personal data on each person to provide evidence-based personalised treatment.17 People would be empowered to play an active role in their health and wellbeing with information tailored to their specific needs. To push the envelope even further, scientists foresee the use of innovative smart devices at home. We are already surrounded by devices that process large amounts of data, why not imagine a medical future where your toothbrush, toilet, or scale is capable of reporting instantly on the condition of your health.18

Genome sequencing is another exciting possibility in the medical field. It took 10 years to decode the first human genome, today it takes a week. As big data technology continues to expand and larger amounts can be processed, everyday genome sequencing for regular people may be possible.19 The unique set of variations in your DNA sequence affect your appearance, your behaviour, and from a medical perspective, your susceptibility to disease. Today, there are only certain parts of the genome that are well understood and that influence our health care decisions. However, as more people are sequenced, scientists and doctors will have access to a larger set of data to learn about genes they previously couldn’t understand, including certain genomes relationships to diseases.20

How big data reduces medical costs

With US healthcare expenses in excess of $600 billion, fiscal concerns are among the greatest demand for big data application.21

Analytics can help organisations forecast demand for medication. The Clinton Health Access Initiative (CHAI) uses analytics to determine the need for HIV/AIDS, malaria, and tuberculosis medication, which has led to negotiations for lower prices and wider availability of medical care in countries needing it most. CHAI has shared information with the United Nations and the World Health Organisation, aiming to identify how best to spend limited resources.22

Fifty hospitals across Queensland, Australia are using a tool which predicts the number of patients who will arrive and the injuries they will have days, weeks, months, or even years in advance.23 It’s able to identify patterns in historical admission and discharge data, which allows staff to know when to expect patients with specific injuries when preparing and planning for surgery. This has prevented long waits and cancellations by up to 20%. It can also be used to predict arrivals by the hour to alert staff of the number of beds needed at all times. Queensland hospitals are saving as much as $2.5 million USD every year, and the value of improved patient outcomes in the state could be up to $77.5 million.24

Government

The application of big data analytics in Western democratic governments is subject to much debate. While there are foreseeable benefits, democracy’s prioritisation of privacy has the potential to hinder progress. Today, the big data revolution seems to play more into the hands of China’s communist leadership. With uninterrupted access to data through “close collaboration with a few state-licensed commercial data conglomerates Alibaba, Tencent or Baidu,”25 the Chinese government could become a data powerhouse.

However, while only time will tell how China’s authoritarian approach to data will accelerate its technological advance, the rest of the world is by no means ignoring the potential application of big data in government sectors, despite the often strict privacy regulations.

How big data transforms cities

Barcelona developed plans to incorporate data into city planning over 10 years ago. The smart city it’s hoping to achieve spans across various departments including water, energy, communication, housing, and mobility, all with the intention of improving people’s quality of life through technological innovation. The developments are underpinned with intelligent data systems collecting information from smart assets and public organisations with the intention of creating a more open, transparent government to engage citizens and provide greater public independence. According to government officials, Barcelona “is eager to connect global cities through big data analytics to address shared issues in the future”.26

How big data collaborations assist in safety

Collaboration across countries and between public and private sectors may become more common as big data gains prominence. The Haiti earthquake in 2010 is an example of collaboration between businesses, emergency services, the government, and people in a disaster. To help the government, people, and emergency services cope, InStedd, a technology company specialising in emergency services, set up systems to decode data based on time, geolocation, and route of transmission. The quality of data is paramount in prioritising information and minimising incorrect usage of limited rescue resources. Together with various crowdsourcing agencies, they were able to “accurately geotag information to provide coordinates to the search and rescue teams working on the ground,” managing to save and support many more people.

Similar to disaster management, around the US and 30 other countries across the world, Risk-Terrain modelling is being used to predict crime. Buildings are linked with types of crimes, and potentially dangerous areas are recorded on a digital map overlaid with past crime locations. Buildings are then ranked according to the number and type of crimes that have occurred close to their location and are flagged with police to alert them where to focus their efforts and attention. As a result, gun violence has dropped by 35% in Newark, New Jersey; 33% of car robberies stopped in Colorado Springs; and in Glendale, Arizona, overall crime was reduced by over 40%.28

How big data governance empowers you

Smarter government systems mean an easier government experience. In Estonia, Europe, citizens are never asked to fill in the same information twice. The country’s data exchange network, X-Road, saves over 240 hours of work every three minutes by automatically syncing information across government agencies. X-Road works by allowing approved databases to request and share data automatically. Whenever an official needs information, it can be retrieved without a manual request. Finland is also adopting the technology due to its success.29

In Taiwan, people are encouraged to participate in government processes, creating greater citizen engagement. Using a system called vTaiwan, citizens post on a discussion forum about new legislation. Through this method of crowdsourcing, people are invited to post principles that should underpin legislation. Suggestions are rated by other citizens and then clustered to show statements that hold true across the nation. After further deliberation among experts, the government decides whether to implement legislation based on the people’s response, or explains why it won’t be doing so.30

It’s not that it’s big or that it’s fast. The part that’s important is that it’s about people…. By understanding these things, we begin to understand society and social interaction in ways we never could before.

Sandy Pentland
Director of MIT’s Human Dynamics and Media Lab

How an efficient data system saves on cost

Efficiency is everything when it comes to reducing costs in government. With more efficient processes in place and inter-organisational compatibility, analysts estimate potential savings could be as much as $500 billion USD in the United States alone.31

Stay ahead of the big data curve

The need for people to interpret, understand, and execute plans and decisions based on big data analytics is rapidly increasing as industries recognise opportunities in digital information. The Harvard Business Review named Data Analyst “the sexiest job of the 21st century”, attesting to the demand for skilled data professionals and the culture they will drive.32

Help your company to evolve with innovation from either hiring the right talent, creating a data-driven ethos, or becoming skilled in data science yourself.


  • 1 Kumar Roy, A (Apr, 2016). ‘Impact of big data analytics on business, economy, health care and society’. Retrieved from Journal of Biometrics and Biostatistics.
  • 2 (2018). ‘Everyone on the same page, all the time. See how data’s connecting everyone to everything across the world, every minute’ Retrieved from 6th Edition of the Data Never Sleeps Report.
  • 3 Ahmad, I. (Jun, 2018). ‘How much data is generated every minute?”. Retrieved from Social Media Today.
  • 4 Kuneva, M. (Mar, 2009). “Keynote speech: Roundtable on online data collection, targeting and profiling’. Retrieved from European Commission Press Release Database.
  • 5 Siegal, E. (Oct, 2018). ‘3 common mistakes that can derail your team’s predictive analytics efforts’. Retrieved from Harvard Business Review.
  • 6 Moreira, P. (Oct, 2016). ‘How data is reshaping society BDC2016’. Retrieved from Entrevestor.
  • 7 Kumar Roy, A (Apr, 2016). ‘Impact of big data analytics on business, economy, health care and society’. Retrieved from Journal of Biometrics and Biostatistics.
  • 8 Mills, T. (Jul, 2018). ‘Eight ways big data and AI are changing the business world’. Retrieved from Forbes.
  • 9 Kopanakis, J. (Jun, 2018). ‘5 real-world examples of how brands are using big data analytics’. Retrieved from Mentionlytics.
  • 10 Mills, T. (Jul, 2018). ‘Eight ways big data and AI are changing the business world’. Retrieved from Forbes.
  • 11 Kopanakis, J. (Jun, 2018). ‘5 real-world examples of how brands are using big data analytics’. Retrieved from Mentionlytics.
  • 12 Mills, T. (Jul, 2018). ‘Eight ways big data and AI are changing the business world’. Retrieved from Forbes.
  • 13 Kopanakis, J. (Jun, 2018). ‘5 real-world examples of how brands are using big data analytics’. Retrieved from Mentionlytics
  • 14 Kumar Roy, A (Apr, 2016). ‘Impact of big data analytics on business, economy, health care and society’. Retrieved from Journal of Biometrics and Biostatistics.
  • 15 Kumar Roy, A (Apr, 2016). ‘Impact of big data analytics on business, economy, health care and society’. Retrieved from Journal of Biometrics and Biostatistics.
  • 16 Kumar Roy, A (Apr, 2016). ‘Impact of big data analytics on business, economy, health care and society’. Retrieved from Journal of Biometrics and Biostatistics.
  • 17 Buckley, J. (Dec, 2015). ‘How is big data going to change the world?’. Retrieved from World Economic Forum.
  • 18 Kumar Roy, A (Apr, 2016). ‘Impact of big data analytics on business, economy, health care and society’. Retrieved from Journal of Biometrics and Biostatistics.
  • 19 Kumar Roy, A (Apr, 2016). ‘Impact of big data analytics on business, economy, health care and society’. Retrieved from Journal of Biometrics and Biostatistics.
  • 20 Personal Genetics Education Project. (Accessed Oct, 25, 2018). “What is personal genome sequencing?”. Retrieved from PgEd.
  • 21 Kumar Roy, A (Apr, 2016). ‘Impact of big data analytics on business, economy, health care and society’. Retrieved from Journal of Biometrics and Biostatistics.
  • 22 Kumar Roy, A (Apr, 2016). ‘Impact of big data analytics on business, economy, health care and society’. Retrieved from Journal of Biometrics and Biostatistics.
  • 23 Apolitical. (Sep, 2017). ‘Five ways government is using big data to make your life better’. Retrieved from Medium.
  • 24 Apolitical. (Sep, 2017). ‘Five ways government is using big data to make your life better’. Retrieved from Medium.
  • 25 Heilmann, S. (2017). ‘Big data reshapes China’s approach to governance’. Retrieved from Financial Times.
  • 26 Morabito, V. (2015). ‘Big data Analytics for government innovation’. Retrieved from Springer International Publishing Switzerland.
  • 27 Morabito, V. (2015). ‘Big data Analytics for government innovation’. Retrieved from Springer International Publishing Switzerland.
  • 28 Apolitical. (Sep, 2017). ‘Five ways government is using big data to make your life better’. Retrieved from Medium.
  • 29 Apolitical. (Sep, 2017). ‘Five ways government is using big data to make your life better’. Retrieved from Medium.
  • 30 Apolitical. (Sep, 2017). ‘Five ways government is using big data to make your life better’. Retrieved from Medium.
  • 31 Bisk. (2018). ‘How the government uses big data to save people money and improve efficiency’. Retrieved from Villanova University.
  • 32 Davenport, T. Patil, D.J. (Oct, 2012). ‘Data scientist: The sexiest job of the 21st century”. Retrieved from Harvard Business Review.