Also see this recent thread where various options for large/out of memory data processing is discussed: