What’s the big deal with big data?

While the term “big data” is relatively new – the act of gathering and storing large amounts of information for analysis has been an age-old practice. To be honest… it’s not that big and it’s not that new (***that’s what she said***). One thing that has changed is the ability collect, store, analyse and distribute or implement the data set.

These high volumes of data can come from a range of different sources, such as business sales records, the collected results of scientific experiments or real-time sensors used in the internet of things. Data may be raw or preprocessed using separate software tools before analytics are applied.


Data may also exist in a wide variety of file types, including structured data, such as SQL database stores;  unstructured data, such as document files; or streaming data from sensors. Further, big data may involve multiple, simultaneous data sources, which may not otherwise be integrated. For example, a big data analytics project may attempt to gauge a product’s success and future sales by correlating past sales data, return data and online buyer review data for that product.

By recognising that data is now being generated at a speed and volume never before experienced, there is a growing demand for answering increasingly complex questions, but also answering simple questions faster and differently. Big data is often characterised by 3Vs: the extreme volume of data, the wide variety of data types and the velocity at which the data must be processed. Although big data doesn’t equate to any specific volume of data, the term is often used to describe terabytes or even exabytes of data captured over time.

Screen Shot 2018-04-10 at 11.19.45

In Africa, most of the data generated on the continent is still manual such as those in hospitals, official state records and schools. Aside of the propensity of manual recording of data on the continent, the data is sometimes not reliable as it may exclude some subset of the population. In addition recording the data may be different from town to town, even in the same country, because of language and culture barriers.

Globally, one major problem with collection of data is that people lie. Sometimes not intentionally or with mal-intent, but generally people answer surveys in a way that makes them seem better to the person conducting the survey. So the data collected in manual surveys may not be truly representative of the true information.

Where is one place where people don’t lie? Google. A book by Seth Stevens Davidowitz called Everybody Lies examines this issue. One interesting case study he uses is that of Netflix. Originally Netflix asked their audience which films they would like to see in future and queued these up for them. The films the audience marked were aspirational films that people thought they should say they want to see… but they didn’t really have an immediate interest to see them. So the Netflix viewing was dropping. Then Netflix added an algorithm where they used information from sources such as Google Trends to analyse which content is being searched for and then queue the trending content for their users. This saw Netflix viewership surge and it continues to use this principle today.

If you would like to see a trends analysis for your business in Africa, contact us at NubiaNetwork


While interesting opportunities for for Big Data in research and business exist, we also need to be aware that Big Data presents some concerns, such as ethical issues – particularly in developing countries. Additionally, extracting the full benefit from opportunities presented by Big Data, will only be realised with the establishment of infrastructure, systems and corresponding capacity to manage and use the data.

Currently, South Africa is finalising a legal framework called POPI. In simple terms, the purpose of the PoPI Act is to ensure that all South African institutions conduct themselves in a responsible manner when collecting, processing, storing and sharing another entity’s personal information by holding them accountable should they abuse or compromise your personal information in any way.

Kenya and Nigeria currently do not have an omnibus law or comprehensive piece of legislation that provides broad data protection principles covering all sectors engaged in processing of personal data

A couple of startups in Africa are looking to take advantage of the vacuum in this space to produce applications that could solve problems plaguing the continent.  Here are some companies and sectors that are implementing IoT and Big Data to solve Africa’s intricate problems.

Safaricom, Kenya’s biggest telecom company invested in Eneza Education, a startup that aims to help learners in schools brush up on their education using the mobile phone. The company says it has captured 1.8 million users in Africa and aims to spread their services to capture 50 million people.

Students can subscribe to daily questions either through USSD or the Android app and they send answers. They are then graded.

Such innovation enables tutors and teachers to easily track the progress of each child or a group of children through data analysis – all of this by using relevant and ready technology, the mobile phone. The company has completed 14 million questions since its inception, and they have seen general improvement of students using the service.

We are in the information age and using technology is crucial for formulating new ideas and strategies, that is no lie.



Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s


Up ↑

%d bloggers like this: