langreiter.com plain, simple: comment-2006-03-20-1

LANGREITER.COM plain, simple

START

INDEX

comment-2006-03-20-1

CREATED BY gavin • LAST EDITED BY gavin 6611 days AGO

If the compute cycles are far away, change what it is you compute to accomodate. Guan's example of video is a good one to refactor:

"[...] consider the example where users upload video files, but we need to do some post-processing on the video at a later date. This would involve downloading huge files from Amazon, doing the processing, and uploading them again [...]"

Downloading "huge files" isn't really the best use of something like S3 (or any sort of tuplespace-like thing for that matter.) Download individual frames of video (still big but not as big!) Sure you have to do a lot more but at least you're using the various parts in the stack appropriately and you could scale this sanely/predictably as well. Coordination of all this is another issue tho....