By: Elise Ackerman

The software, called Hadoop, is part of Yahoo's massive computing grid and is transforming the way Yahoo and corporate giants such as IBM extract meaning from enormous streams of data.
Universities are also using the code - an open-source version of software Google relies on for daily operation - to train a new generation of computer scientists and engineers.

Hadoop improves the relevance of ads Yahoo shows on the internet by analysing the company's endless flow of data - now more than 10TB daily - on the fly. As users click from Yahoo Mail to Yahoo Search to Yahoo Finance and back again, Hadoop helps figure out what ad, if any, is likely to catch someone's attention.

Google quickly got on board, launching an initiative with IBM to provide universities such as Stanford, University of California, MIT and Carnegie Mellon with clusters of several hundred computers, so students could learn new techniques for parallel programming. Since Google's MapReduce was a trade secret, Google and IBM announced the students would be taught on Hadoop.

To continue reading this interesting article posted in the Australian IT, click here.