A Bluffer’s Guide to AI, Algorithmics and Big Data

 Big Data—More Than “Big”

A few years ago, the keyword big data resounded throughout the land. What

is meant is the emergence and the analysis of huge amounts of data that

is generated by the spreading of the Internet, social media, the increasing

number of built-in sensors and the Internet of Things, etc.

The phenomenon of large amounts of data is not new. Customer and

credit card sensors at the point of sale, product identification via barcodes

or RFID as well as the GPS positioning system have been producing large

amounts of data for a long time. Likewise, the analysis of unstructured data,

in the shape of business reports, e-mails, web form free texts or customer

surveys, for example, is frequently part of internal analyses. Yet, what is

new about the amounts of data falling under the term “big data” that has

attracted so much attention recently? Of course, the amount of data avail-

able through the Internet of Things (Industry 4.0), through mobile devices

and social media has increased immensely (Fig. 2.1).

A decisive factor is, however, that due to the increasing orientation of

company IT systems towards the end customer and the digitalisation of

business processes, the number of customer-oriented points of contact that

can be used for both generating data and systematically controlling commu-

nication has increased. Added to this is the high speed at which the corre-

sponding data is collected, processed and used. New AI approaches raise the

analytical value creation to a new level of quality.

Big Data—What Is Not New

The approach of gaining insights from data for marketing purposes is noth-

ing new. Database marketing or analytical CRM has been around for more

than 20 years. The phenomenon of large amounts of data is equally nothing

new: Point of sale, customer and credit cards or web servers have long been

producing large amounts of data. Equally, the analysis of unstructured data

in the shape of emails, web form free texts or customer surveys, for example,

frequently form a part of marketing and research.

is generated by the spreading of the InterBig Data—What Is New

It goes without saying that the amount of data has increased immensely

thanks to the Internet of Things, mobiles and social media—yet this is

rather a gradual argument. The decisive factor is that thanks to the possi-

bilities of IT and the digitalisation of business processes, customer-oriented

points of contact for both generating data and for systematically controlling

communication have increased. Added to this is the high speed at which the

corresponding data is collected, processed and used. Equally, data mining

methods of deep learning and semantic analytics raise the analytical value

creation to a new level of quality.

Definition of Big Data

As there are various definitions of big data, one of the most common ones

will be used here:

“Big data” refers to datasets whose size is beyond the ability of typical database

software tools to capture, store, manage, and analyse.