[Lustre-discuss] WBC subcomponents.

Vladimir V. Saveliev Vladimir.Saveliev at Sun.COM
Thu Jan 31 11:55:46 PST 2008


Hello

On Wed, 2008-01-23 at 00:10 +0300, Nikita Danilov wrote:
> Hello,
> 
> below is a tentative list of tasks into which WBC effort can be
> sub-divided. I also provided a less exact list for the EPOCH component,
> and an incomplete list for the STL component.
> 
> WBC tasks are estimated in lines-of-code with the total of (9100 + 3000)
> LOC, where LOC is a non-comment, non-whitespace line of the source
> file. I believe it is too early to estimate all EPOCH tasks, hopefully
> there will be more clear understanding of the situation during next
> week. I tried to estimate the EPOCH tasks that are absolutely necessary
> during the early stages of WBC development.
> 
> Vladimir is the best person to complete the list of STL tasks,
> together with the estimations.
> 
> C-* tasks are for the client, S-* tasks are for the server.
> 
> ESTIMATION
> 
>                      scope                                                effort
>                                                                              loc
> WBC
> 
>     C-VFS-MM         integration with vfs: inodes, dentries, memory         2500
>                      pressure. Executing operation effects locally.
> 
>     C-ops-caching    tracking operations: list vs. fragments. Tracking      2500
>                      dependencies.
> 
>     C-write-out      policy deciding when to write-out cached state         1000
>                      updates, and with what granularity: age, amount,
>                      max-in-flight
> 
>     C-dir-pages      caching of directory pages and using them for local    1000
>                      lookups
> 
>     C-new-files      creation of new files locally                           200
> 
>     C-new-objects    creation of new objects locally                         200
> 
>     C-DLM            invoking reintegration on a lock cancel, lock           200
>                      weighting
> 
>     C-data           dependencies between cached data and meta-data          200
> 
>     C-IO             switching between whole-file mds-based locking and      250
>                      extent locking
> 
>     C-grants         unified resource range leasing mechanism                250
> 
>     S-grants         unified resource range leasing mechanism                250
> 
>     C-misc           sync, fsync, compatibility flag, mount option           200
> 
>     S-misc           compatibility flags                                     100
> 
> STL

Components of STL (tentatively)

> 
C-policy         track usage statistics                            250
                 and use them to decide when to ask for a STL

S-policy         track usage statistics                            250
                 and use them to decide when to grant a STL

S-LDLM           new lock type, etc                                100

C-S-STL split    split STL into STLs of subdirs on callback        1000

Persistent STL   store subtree lock on persistent storage           2000

S-path revalidation
                 with STL access to ".." requires special handling  1000

C-filesystem operations
                 this is where requests for subtree lock come from  1000

S-conflicts      special cases in lock conflicts resolution,        1500
                 switching  to ordinary locks,
                 lock acquiring under STL, CMD

C-WBC            interaction with WBC                               100

C-S-recovery     resending locks after reconnection                 100

> EPOCHS
> 
>     formalization    formal reintegration model with "proofs" of recovery
>                      correctness and concurrency control description
> 
>     C-reintegration  reintegration, including concurrency control,          1000
>                      integration with ptlrpc
> 
>     S-compound       implementation of the compound operations on           1000
>                      the server
> 
>     S-reintegration  reintegration of batches on the server, thread         1000
>                      scheduling
> 
>     S-undo           keeping undo logs
> 
>     S-cuts           implementation of the CUTs algorithm
> 
>     C-gc             garbage collection: when to discard cached batches
> 
>     S-gc             garbage collection: when to discard undo logs
> 
>     C-recovery       replay, including optional optimistic "pre-replay"
> 
>     S-recovery-0     roll-back of the uncommitted epochs
> 
>     S-recovery-1     roll-forward from the clients
> 
> EXTERNAL DEPENDENCIES
> 
>     ost-fid          ost understanding fids, and granting fid sequences
>                      to the clients
> Nikita.

Best regards,
Vladimir




More information about the lustre-discuss mailing list