[Lustre-discuss] Distributed Object storage lookup of small files
Peter Grandi
pg_lus at lus.for.sabi.co.UK
Sat Aug 15 12:03:17 PDT 2009
>From Pranas Baliuka:
pranas> Sent: Monday, August 10, 2009 12:11 AM
pranas> To: lustre-discuss at lists.lustre.org
pranas> Subject: [Lustre-discuss] Distributed Object storage lookup of small files
pranas> Dear Lustre experts/users,
pranas> I looking for optimal solution of the task:
pranas> Internet-scale applications must be designed to process
pranas> high volumes of transactions. Describe a design for a
pranas> system that must process on average 30,000 HTTP requests
pranas> per second. For each request, the system must perform a
pranas> lookup into a dictionary of 50 million words, using a
pranas> key word passed in via the URL query string. Each
pranas> response will consist of a string containing the
pranas> definition of the word (10 KB or less).
pranas> Task is hypothetical, but would be nice to get feedback
pranas> from specific technology experts... Some ideas ;)
This looks like to me an attempt at cheating on a university
assignment or a job interview challenge.
Especially given that something like Lustre looks ridiculous
overkill for such a task (50m word, each on average 7-10 chars
long => only 500MB table, and read-only too), so posting the
question here makes little sense.
But then I have seen a number of ignoramuses happy to use
filesystems instead of incore or storage databases (typical
questions about having several million or hundred million files
almost all of which less than a block long, or directories with
hundreds of thousands or millions of files, especially on the
XFS mailing list).
More information about the lustre-discuss
mailing list