[Lustre-discuss] WBC subcomponents.
Vladimir V. Saveliev
Vladimir.Saveliev at Sun.COM
Thu Jan 31 11:55:46 PST 2008
Hello
On Wed, 2008-01-23 at 00:10 +0300, Nikita Danilov wrote:
> Hello,
>
> below is a tentative list of tasks into which WBC effort can be
> sub-divided. I also provided a less exact list for the EPOCH component,
> and an incomplete list for the STL component.
>
> WBC tasks are estimated in lines-of-code with the total of (9100 + 3000)
> LOC, where LOC is a non-comment, non-whitespace line of the source
> file. I believe it is too early to estimate all EPOCH tasks, hopefully
> there will be more clear understanding of the situation during next
> week. I tried to estimate the EPOCH tasks that are absolutely necessary
> during the early stages of WBC development.
>
> Vladimir is the best person to complete the list of STL tasks,
> together with the estimations.
>
> C-* tasks are for the client, S-* tasks are for the server.
>
> ESTIMATION
>
> scope effort
> loc
> WBC
>
> C-VFS-MM integration with vfs: inodes, dentries, memory 2500
> pressure. Executing operation effects locally.
>
> C-ops-caching tracking operations: list vs. fragments. Tracking 2500
> dependencies.
>
> C-write-out policy deciding when to write-out cached state 1000
> updates, and with what granularity: age, amount,
> max-in-flight
>
> C-dir-pages caching of directory pages and using them for local 1000
> lookups
>
> C-new-files creation of new files locally 200
>
> C-new-objects creation of new objects locally 200
>
> C-DLM invoking reintegration on a lock cancel, lock 200
> weighting
>
> C-data dependencies between cached data and meta-data 200
>
> C-IO switching between whole-file mds-based locking and 250
> extent locking
>
> C-grants unified resource range leasing mechanism 250
>
> S-grants unified resource range leasing mechanism 250
>
> C-misc sync, fsync, compatibility flag, mount option 200
>
> S-misc compatibility flags 100
>
> STL
Components of STL (tentatively)
>
C-policy track usage statistics 250
and use them to decide when to ask for a STL
S-policy track usage statistics 250
and use them to decide when to grant a STL
S-LDLM new lock type, etc 100
C-S-STL split split STL into STLs of subdirs on callback 1000
Persistent STL store subtree lock on persistent storage 2000
S-path revalidation
with STL access to ".." requires special handling 1000
C-filesystem operations
this is where requests for subtree lock come from 1000
S-conflicts special cases in lock conflicts resolution, 1500
switching to ordinary locks,
lock acquiring under STL, CMD
C-WBC interaction with WBC 100
C-S-recovery resending locks after reconnection 100
> EPOCHS
>
> formalization formal reintegration model with "proofs" of recovery
> correctness and concurrency control description
>
> C-reintegration reintegration, including concurrency control, 1000
> integration with ptlrpc
>
> S-compound implementation of the compound operations on 1000
> the server
>
> S-reintegration reintegration of batches on the server, thread 1000
> scheduling
>
> S-undo keeping undo logs
>
> S-cuts implementation of the CUTs algorithm
>
> C-gc garbage collection: when to discard cached batches
>
> S-gc garbage collection: when to discard undo logs
>
> C-recovery replay, including optional optimistic "pre-replay"
>
> S-recovery-0 roll-back of the uncommitted epochs
>
> S-recovery-1 roll-forward from the clients
>
> EXTERNAL DEPENDENCIES
>
> ost-fid ost understanding fids, and granting fid sequences
> to the clients
> Nikita.
Best regards,
Vladimir
More information about the lustre-discuss
mailing list