Hadoop on the network: How to prep for big data

The volume of unstructured information will require the use of new tools, potentially including the popular open source in-memory database. This is not just an enterprise play


Big Data means potentially big things for the network — finding strategic value in deeper analytics, particularly as more organizations seek to find faster ways of sifting through raw data to boost productivity and make better business-critical decisions.  But as Big Data is set to be a $50 Billion Industry by 2017, it’s telling the Globe and Mail recently cited an IDC Canada survey that revealed “about 10 per cent of nearly 300 respondents had some version of a formal strategy to capitalize on big data opportunities.”

Canadian companies are facing growing volumes of raw and unstructured data, which naturally means more load on the network, not to mention a potential impact on overall network performance. Given the rise of the data network and many problems organizations have handling big data, how do IT departments get that competitive edge when thinking about big data and the network?

As GigaOM Pro notes, big data is fundamentally changing networking, forcing organizations to rethink how data is analyzed and structured. Managing unstructured data is a big deal in Canada, notes Canadian IT, Michelle Warren, principal analyst at MW Research & Consulting in Toronto, especially when factoring in in BYOD mobility trends, cloud computing and Web 2.0 and social networking factors presently transforming traditional information processing models.

Warren explains that IT departments should already be looking into testing and pilot programs around Big Data and the network and server architecture. In thinking of network performance, top of mind should be new big data technologies such as open source Hadoop, designed to managing complex and structured data not easily integrated into tables.

“You’ll more see the larger and traditional organizations like retailers and financial services firms (adopting it), but average Canadian organizations should still see big data as something that is a high priority,” says Warren.

While organizations look to crunch massive amounts to data, there are more tools emerging that purport to help streamline the process.  Looking at the Big Data landscape, an emerging technologies are making some noise, with vendors like SAP taking steps to improve Hadoop support in its HANA in-memory product or Cisco and Pentaho partnering on open source big data solutions.

“(Hadoop) is well embraced and has a lot of proponents,” says Warren. “It’s still a relatively niche player in Canada but something that larger firms are embracing,” says Warren.

From a network manager perspective (and in thinking of basic reporting and predictive analytics), Hadoop can be seen as a more cost-effective method to simplify data, overcome storage limitations, and scale computing power when analyzing distributed, unstructured data.

But ultimately, big data is more than Hadoop and analytics. In thinking of big data on the network also involves an integrated and layered approach, says Warren.

“(Such strategies) should be already baked into IT architecture and deployment plans. The management component is truly crucial. The IT department simply needs to know what’s going on with the data at all times. There’s too much liability tied up,” she notes.

This includes understanding that a typical big data environment effectively optimizes and simplifies search –meaning that moving beyond a traditional data analysis architecture towards one that incorporates both large scale real time databases along with analytical databases is key moving forward. And as cloud computing tools and big data applications mature and become more pervasive, a new data ecosystem is the new corporate reality that network managers need to strive for.

Sooner rather than later, Warren notes.

“(Big Data is) here to stay because we’re looking at vast quantities of unstructured data in Canadian organizations,” says Warren. “Information is knowledge. There’s greater access to data than ever before – so the more data points that can be collected, the more organizations can have greater insight. It’s a definite competitive advantage.”

Lay the foundation for big data by downloading our free white paper from Frost & Sullivan, ‘The IP Imperative: The Time to Build an Optimized Network is Now.’

 

Image by NTTCOM.tv

Comments are closed.