[Lustre-discuss] Distributed Object storage lookup of small files

[Pranas Baliuka]

Dear Lustre experts/users,

 

I looking for optimal solution of the task:

Internet-scale applications must be designed to process high volumes of
transactions. 

Describe a design for a system that must process on average 30,000 HTTP
requests per second. 

For each request, the system must perform a lookup into a dictionary of 50
million words, using a key word passed in via the URL query string. 

Each response will consist of a string containing the definition of the word
(10 KB or less).

 

My initial though was using MySQL/Berkeley DB pointing to SAN, but probably
lower level solution would be more affordable. 

Can I use e.g. QFS storage via Java without DB severer instead. Can SAN be
avoided and local HDDs joined to Lustre system?

 

Task is hypothetical, but would be nice to get feedback from specific
technology experts...

Some ideas ;) 

 

I've send similar request to QFS forum and really not sure which product
would fit better. Both works as distributed file systems ... and both sounds
as convenient storage for particular task.

 

Thanks,

Pranas

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.lustre.org/pipermail/lustre-discuss-lustre.org/attachments/20090810/249d09c6/attachment.htm>