Big Data Hackathon participants learn get a crash course in data analysis. (Images via Eman Mostafa)

Go big or go home could have been a good motto for the third edition of Egypt's Big Data Hackathon last week.

Over 100 people applied for the three-day event on Febraury 18, but only 65 students, graduates and technology sector employees were chosen to join 11 teams that would embue meaning into their selected data.

Big data is a term that refers to analysis of large amounts of raw data generated every second by people and machines, to help make fast, calculated decisions.

"This year's event was about providing big data on five main tracks in Egypt: social networks for telecommunication companies, airlines and stock markets from the private sector, water and agricultural land from the public sector," said Heba Saleh, head of Egypt's Information Technology Institute (ITI). "TeraData helped provide a database regarding the private sector tracks, with the Ministry of Housing and that of scientific research providing databases for the public sector track."

In the first two days, the teams were taught to read and analyze data using software provided by TeraData. On the third day, the teams presented their results to panel of judges.

"We separated the public from the private sector because the nature of data coming from government bodies is different from those coming from the private sector,” said Hesham Farouk, technology consultant at ITI and event judge. “The first is based on studies and tangible results, lacks order and unfortunately, often has [data] missing.”

“As for the private sector data, it is always curated, easy to process by using some technical tools or specialized programs. So separating the two becomes inevitable to keep the competition fair and just," he said.

Participants were evaluated according to team skills and the value of results delivered, Farouk said.

Two teams were declared winners, one working with private sector data (social networks data for telcos) and one working with public sector data (agricultural data).

A team that analyzed telcos data presents its findings.

Giving data meaning

"Despite the lack of time and the large amount of data available, we were able to find the weaknesses and strengths of the three telecommunication companies in Egypt,” said Abada Sayed, from the winning team dealing with social networks data. “The results of the available data analysis stopped at the complaints submitted by customers on the social networks of each company, regarding internet services, network power and response speed."

Mustafa Alaa, working with agricultural land data, took a different approach.

"We separated desert areas from agricultural areas from urban areas via satellite imagery,” he said. “We were also able to add other data using network programs. Eventually, we came up with the shape of these areas by the end of 2016. We also reached important results regarding desertification, which turns out to be a bigger threat to agricultural lands than turning them into buildings."

Hisham El Razzaz, a data sciences student at ITI, won the best idea award for his sensor that evaluates water purity by measuring the intensity of the light going through it when placed on water pipes attached to the house. His device is still in the idea stage.

"The results show on a built-in screen, and according to the findings, we will know the missing or extra components in the water. We will then analyze the data while comparing the results found with the standard results, to discover the dangers and start working on fixing them."

The event dovetailed with growing interest in big data in Egypt, and the realization of the importance in organizing, processing and curating big data.

According to the founder of Gartner for Marketers, just 15 percent of companies are making good use of big data, and financial indicators say these companies have witnessed a 20 percent increase in effectiveness.