On Wed, 2008-11-12 at 09:43 -0500, Ed Summers wrote:
> On Wed, Nov 12, 2008 at 9:30 AM, Phil Cryer <phil.cryer_at_mobot.org> wrote:
> > Thanks for the Tahoe mention, I hadn't heard of that. Looking at it now
> > for differences from Hadoop.
>
> I think the main difference is the locality-of-reference you get with
> Hadoop, which allows you to distribute processing as well as data.
> This can be important in intensive data crunching exercises, where
> having the data you are working with right there on a local disk,
> rather than coming over the network is important. But distributing
> processing in this way may not be important to you.
>
> //Ed
Good point Ed, my thought is that in the future with all the data we'll
be consuming we'll *need* distributed filesystems for redundancy/fault
tolerance (and sanity), but to have something that can distribute big
jobs around, like indexing huge datasets, the dist processing would be a
bonus.
P
--
Phil Cryer | Open Source Dev Lead | web www.mobot.org | skype phil.cryer
Received on Wed Nov 12 2008 - 09:14:30 EST