When Yahoo! announced that it’s outsourcing search to Microsoft, some folks became concern that Yahoo! could end its sponsorship of Hadoop. This is cleared up in the Yahoo! Developer Network Blog: Yahoo! will continue to sponsor this effort.
Open source software requires sponsors to staff engineers in order to develop and maintain the software; and Yahoo! is the key sponsor for Hadoop. Yahoo! has other uses for Hadoop in addition to search. Like Linkedin and Facebook, Yahoo! has lots of users and groups, and like these other web 2.0 companies, Yahoo! can run data mining and analytics for its purposes, such as targeted advertising. Many data analytics algorithms, including machine learning, can scale linearly with a Hadoop architecture. So it’s very believable that Yahoo! has a big stake in having a robust Hadoop implementation,.
For discussion sake, if there were no Hadoop, the existing alternatives are somewhat limited:
At the same time, if Yahoo! were to pull Hadoop sponsorship, it’d likely that others could come out to sponsor Hadoop. My guess would be Linkedin, Facebook, and even Google.
Copyright (c) 2009 by Waiming Mok
