#LyX 1.3 created this file. For more info see http://www.lyx.org/ \lyxformat 221 \textclass newbook \language english \inputencoding auto \fontscheme default \graphics default \paperfontsize default \spacing single \papersize Default \paperpackage a4 \use_geometry 0 \use_amsmath 0 \use_natbib 0 \use_numerical_citations 0 \paperorientation portrait \secnumdepth 3 \tocdepth 3 \paragraph_separation indent \defskip medskip \quotes_language english \quotes_times 2 \papercolumns 1 \papersides 1 \paperpagestyle fancy \layout Section MDS and OSS Capabilities \layout Subsection Introduction \layout Standard Capabilities are pieces of data generated by one service, the \series bold master service \series default , passed to a client and presented by the client to another service, the \series bold slave service, \series default to authorize an action. \layout Subsection Specification \layout Subsubsection Feature control \layout Standard For each of the MDS and OST the feature of capability checking and generation can be turned on or off with an lconf parameter and a /proc/sys variable. \layout Standard The duration of validity can be tuned through /proc (important for acceptance tests). \layout Subsubsection Special user \layout Standard The security context for a user, both on the MDS and OSS may indicate that the user has special priviliges for the target and can access fids and/or objects in the target without capabilities. If the MDS detects this, a 0 capability is generated. \layout Subsubsection MDS capabilities (not required for cmd2 - skip initially) \layout Standard There are a few MDS capabilities: \layout Enumerate A fid capability: this proves that a client obtained access to a fid \layout Enumerate Replay capability for an operation associated with a fid. \begin_deeper \layout Enumerate At present only open for a fid (together with the opening mode) are important \layout Enumerate Chdir and mount may be additional capabilities \end_deeper \layout Enumerate Each request to the MDS will include the capability or capabilities associated with the fid(s) in that request. \layout Enumerate Each reply to a lookup request or open request from the MDS contains a capabilit y. \layout Subsubsection OSS capabilities \layout Enumerate When objects for a file are created the MDS id and storage id of the file inode is stored in an EA in the object. \layout Enumerate When a file is opened successfully a capability is sent to the client allowing create, read/write or read access to the objects. \layout Enumerate The capability is verified by the OSS. \layout Subsubsection Capability Renewal \layout Enumerate the client keeps capabilities in a list. When they are close (like 90%) to expiring they are renewed with the MDS. Notice that renewal of a capability is based on the original open, not on current permissions of a file. \layout Subsubsection Capability cryptography \layout Paragraph HMAC-SHA1 function \layout Standard A MAC is used to prove that a capability was not modified since it was generated by a master. The size of the MAC is 128 bits. \layout Standard We need a kernel level implementation of this function to compute hashes of capabilities, to protect their integrity. It seems that HMAC-MD5 or HMAC-SHA1 would be acceptable for this purpose. \layout Paragraph capability cache \layout Standard Signing the capabilities is probably time consuming. A cache of fixed size (maybe some 1000's) of signed capabilities should be retained. To increase cache hits, timeouts in capabilities should be rounded to the nearest 1000 seconds. \layout Paragraph Encryption \layout Standard Capabilities need to be encrypted on the wire so that they cannot be snooped. There are two mechanisms for encrypting capabilities: \layout Enumerate Use the GSS infrastructure - this is the simplest solution, good for cmd2. \layout Enumerate Build a special purpose encryption mechanism for capabilities. \layout Standard When OSS capabilities are enabled both client - MDS and client - OSS connections need to use privacy on the GSS connection. \layout Subsubsection Managing and setting shared keys with slaves \layout Standard The master and the slave need to share a key to verify the authenticity of capabilities, this key is called the \series bold base key \series default and the base key must be identified so that it can be found by the slave. Slaves may have base keys from different masters. The capability will identify what key was used, but not contain the key. \layout Standard The master will be an MDS, the slave will be an MDS (possibly the same server as the master) or an OSS. \layout Enumerate The master will connect using GSS to the slave using a special principal allowing administrative actions. \layout Enumerate At any time a slave will have one or two keys available for any master, labeled as red and black. Keys expire after a hard coded amount of hours, after which a new key will be generated by the master. To avoid a storm of new requests, the key that expired remains valid for another time interval and is then discarded. Hence, when a new key arrives on the slave: \begin_deeper \layout Enumerate the black key is discarded, with all its associated cache information \layout Enumerate the red key becomes black \layout Enumerate the new key is given the red label \end_deeper \layout Enumerate The keys will be identified by the a master server number and an increasing key sequence number. \layout Enumerate Each time a key expires, a new key is generated by the master the sequence number is increased in the MDS disk data. \layout Enumerate The keys generated by the master service and sent, encrypted using GSS to the slave service. \layout Enumerate This network command is an obd-ioctl or set info command that tells the OSS: \begin_deeper \layout Enumerate the new key \layout Enumerate the identifier of the key \layout Enumerate the duration for which the key will be used \end_deeper \layout Enumerate When capabilities are verified both the red and the black key can be used. \layout Enumerate When keys need to be found, they are stored in a hash table, based on the key identifier. \layout Enumerate The keys should be stored in an on disk directory to avoid having to renew keys at recovery. \layout Subsection Logic \layout Standard The design falls into natural parts \layout Enumerate Capabilities without encryption for the OSS \begin_deeper \layout Enumerate capabilities are somehow shared between the mdc's and osc's \begin_deeper \layout Enumerate mdc is responsible for renewal of the capability \layout Enumerate osc is responsible to include capability in further request(s). The capability can be used many times. \end_deeper \layout Enumerate the logical location of capability code is probably in the class driver: they might / should be used all over the place. Let's consider very carefully in the DLD how a capability is found, referenced and released, in conjunction with the cache of signed capabilities on the server nodes. \layout Enumerate structure of the capability \layout LyX-Code struct lustre_capa { \end_deeper \layout LyX-Code u64 lcapa_objid[3]; // object/inode on what server \layout LyX-Code u32 lcapa_operation; // operation allowed \layout LyX-Code u32 lcapa_expiry; // expiry time: servers have clocks \layout LyX-Code u32 lcapa_keyid; // key used for the capability \layout LyX-Code u32 lcapa_secflags // security features for capa \layout LyX-Code } \layout Enumerate Capabilities without encryption for the MDS \layout Enumerate Renewing nearly expired capabilities \begin_deeper \layout Enumerate maintain a list for each user of capabilities \layout Enumerate manage capabilities in the MDC / OSC, not \end_deeper \layout Enumerate Signing capabilities and a cache of signed capabilities \layout Enumerate Shared key management between master and slave capability nodes \layout Subsection Future work \layout Standard Possibly the use of GSS connections is associated with serious overhead. This needs to be profiled. \the_end