Showing posts with label Kernel. Show all posts
Showing posts with label Kernel. Show all posts

Friday, April 19, 2013

NTLM Authentication and Signing.


SMB connection is established over 3 steps.
1) Negotiation: The client and the server exchange a list of their own capabilities.
We have 1 Negotiation call per connection to the server.
2) Session Setup: Here the user authentication takes place.
There is one Session Setup per user using the connection negotiated in step 1.
3) Tree Connect: We connect to the share available to the user.
We have one tcon per smb share.

NTLM authentication is based on a challenge response mechanism. On responding to the Negotiate call, the server sends over a 'Challenge' which is used in the authentication process. The client then calculates a response using
1) NTLM Hash: This is created using the users password. This will be CIFS_AUTH_RESP_SIZE ie. 24 bytes long.
http://ubiqx.org/cifs/SMB.html#SMB.8.4
2) Server Challenge: This was sent by the server in response to the Negotiate call.
http://ubiqx.org/cifs/SMB.html#SMB.8.4
http://ubiqx.org/cifs/SMB.html#SMB.8.3.4

When calculating the Response to the server, we also calculate the Session Key. This will be used to Sign packets if the feature is chosen. To calculate the Session Key, we simply obtain a md4 hash of the NTLM hash. This will be CIFS_SESS_KEY_SIZE ie. 16 bytes long.
http://ubiqx.org/cifs/SMB.html#SMB.8.9.1

We first describe how the NTLM hash is calculated and the "client Response" calculated.

IC -> Implementing CIFS by Chris Hertel. http://ubiqx.org/cifs/SMB.html
MAC ->  Message Authentication Code

On Negotiate, we receive a Challenge from the server. We store this Server Challenge into ses->cryptKey.
int
CIFSSMBNegotiate(unsigned int xid, struct cifsSesInfo *ses)
{
..
        if (pSMBr->EncryptionKeyLength == CIFS_CRYPTO_KEY_SIZE) {
                memcpy(ses->cryptKey, pSMBr->u.EncryptionKey,
                       CIFS_CRYPTO_KEY_SIZE);
..
}


We then call CIFS_SessSetup to initiate a session.

int
CIFS_SessSetup(unsigned int xid, struct cifsSesInfo *ses,
               const struct nls_table *nls_cp)
{
..
        type = ses->server->secType;
..
        } else if (type == NTLM) {
..
                /* calculate ntlm response and session key */
                rc = setup_ntlm_response(ses);
..
}

We call setup_ntlm_response() to build the NTLM hash and the Response to the 'Server Challenge'


/* first calculate 24 bytes ntlm response and then 16 byte session key */
int setup_ntlm_response(struct cifsSesInfo *ses)
{
//The key-response combo length Total: 16 + 24 = 40 bytes.
        unsigned int temp_len = CIFS_SESS_KEY_SIZE + CIFS_AUTH_RESP_SIZE;
//A temporary buffer to hold it.
        char temp_key[CIFS_SESS_KEY_SIZE];

        if (!ses)
                return -EINVAL;

//We allocate the memory which will be used to store this response.
        ses->auth_key.response = kmalloc(temp_len, GFP_KERNEL);
        if (!ses->auth_key.response) {
                cERROR(1, "NTLM can't allocate (%u bytes) memory", temp_len);
                return -ENOMEM;
        }
//Store the key len.
        ses->auth_key.len = temp_len;

//First we get the NTLM Response
//Look at notes for SMBNTencrypt() below.
//The response created is stored at the buffer 
//starting at ses->auth_key.response + CIFS_SESS_KEY_SIZE(16 bytes) 
        SMBNTencrypt(ses->password, ses->cryptKey,
                        ses->auth_key.response + CIFS_SESS_KEY_SIZE);

//We re-create the NTLM hash.
//See notes for E_md4hash in SMBNTencrypt() below
//The NTLM key is stored in temp_key
        E_md4hash(ses->password, temp_key);

//The md4 NTLM hash of the NTLM hash is then stored at buffer starting
//at ses->auth_key.response. This session key is 16 bytes long
        mdfour(ses->auth_key.response, temp_key, CIFS_SESS_KEY_SIZE);

        return 0;
}
void
SMBNTencrypt(unsigned char *passwd, unsigned char *c8, unsigned char *p24)
{
//Temporary buffer.
        unsigned char p21[21];

//Zero temporary buffer
        memset(p21, '\0', 21);

//We first find the NTLM hash
        E_md4hash(passwd, p21);
-->
We just need the password to generate the NTLM hash
void
E_md4hash(const unsigned char *passwd, unsigned char *p16)
{
int len;
__u16 wpwd[129];


/* Password cannot be longer than 128 characters */
if (passwd) {
//We first trim to password to 128 bytes if longer.
len = strlen((char *) passwd);
if (len > 128)
       len = 128;

//We then convert it to unicode.
/* Password must be converted to NT unicode */
_my_mbstowcs(wpwd, passwd, len);
} else
len = 0;

wpwd[len] = 0;  /* Ensure string is null terminated */
/* Calculate length in bytes */
//We get the new length. This is 2 times the length of the password.
len = _my_wcslen(wpwd) * sizeof(__u16);

//We then get a md4 hash. This is 16 bytes long.
mdfour(p16, (unsigned char *) wpwd, len);
//We then zero the temporary buffer to avoid leaking the passwd hash.
memset(wpwd, 0, 129 * 2);
}
<-- blockquote="">
//We then create the response.
//Look at the notes for SMBOWFencrypt below
        SMBOWFencrypt(p21, c8, p24);

//At this stage, we have the response in the p24 array.
}


http://ubiqx.org/cifs/SMB.html#SMB.8.3.4 
For this we need the NTLM hash and the Server provided Challenge.


/* Does the des encryption from the NT or LM MD4 hash. */
static void
SMBOWFencrypt(unsigned char passwd[16], const unsigned char *c8,
              unsigned char p24[24])
{
//Temporary buffer
        unsigned char p21[21];

//Zero temporary buffer.
        memset(p21, '\0', 21);

//Copy 16 byte NTLM hash to temporary buffer.
        memcpy(p21, passwd, 16);
//Calculate response
        E_P24(p21, c8, p24);
-->
The response is simply calculated by padding the 16 byte NTLM hash to 21 bytes.
We then divide the 21 byte key into 3 keys of 7 bytes each.
We use each 7 byte key to encrypt the Server Challenge separately into 3 
seperate 8 byte values. 
These values are stored in a concatenated form in the p24 buffer.
void
E_P24(unsigned char *p21, const unsigned char *c8, unsigned char *p24)
{
smbhash(p24, c8, p21, 1);
smbhash(p24 + 8, c8, p21 + 7, 1);
smbhash(p24 + 16, c8, p21 + 14, 1);
}
<-- div="">
}

At this point,
ses->auth_key points to the Session Key.
ses->auth_key+CIFS_SESS_KEY_SIZE points to the response we send to the server.

The response generated is used in the SESSION_SETUP_ANDX call to the server. There are 2 copies of the response sent in
SMB->SESSION_SETUP_ANDX->Ansi Password and SMB->SESSION_SETUP_ANDX->Unicode Password


int
CIFS_SessSetup(unsigned int xid, struct cifsSesInfo *ses,
               const struct nls_table *nls_cp)
{
..
        } else if (type == NTLM) {
..
                /* copy ntlm response */
                memcpy(bcc_ptr, ses->auth_key.response + CIFS_SESS_KEY_SIZE,
                                CIFS_AUTH_RESP_SIZE);
                bcc_ptr += CIFS_AUTH_RESP_SIZE;
                memcpy(bcc_ptr, ses->auth_key.response + CIFS_SESS_KEY_SIZE,
                                CIFS_AUTH_RESP_SIZE);
                memcpy(bcc_ptr, ses->auth_key.response + CIFS_SESS_KEY_SIZE,
                                CIFS_AUTH_RESP_SIZE);
                bcc_ptr += CIFS_AUTH_RESP_SIZE;
..
}


At this stage the user is authenticated..

The first time this session-key / response key is created for a server, it is copied over to
server->auth_key. This is then used for calculation of signatures for all sessions on the server.


int cifs_setup_session(unsigned int xid, struct cifsSesInfo *ses,
                        struct nls_table *nls_info)
{
..
//Call Session setup.
        rc = CIFS_SessSetup(xid, ses, nls_info);
..
                mutex_lock(&ses->server->srv_mutex);
//If this is the first time we create a session on the server, then save 
//The session key and response for use in creating signatures.
                if (!server->session_estab) {
//Copy over auth_key to the server.
                        server->session_key.response = ses->auth_key.response;
                        server->session_key.len = ses->auth_key.len;
//Change sequence number to 2 (1 & 2 are used by the SESSION_SETUP_ANDX call)
                        server->sequence_number = 0x2;
//Set established to true.
                        server->session_estab = true;
//Set this to NULL so that the response string isn't freed by kfree() below.
                        ses->auth_key.response = NULL;
                }
                mutex_unlock(&server->srv_mutex);

                cFYI(1, "CIFS Session Established successfully");
                spin_lock(&GlobalMid_Lock);
                ses->status = CifsGood;
                ses->need_reconnect = false;
                spin_unlock(&GlobalMid_Lock);
        }

//Free Session key and response.
        kfree(ses->auth_key.response);
        ses->auth_key.response = NULL;
        ses->auth_key.len = 0;

        return rc;
}

We now look at the Signature generation for each SMB message.

To calculate the Message Authentication Code ( MAC - Signature) for a SMB message, we need the following 3
1) The session key
2) The response
both of which are calculated during the initial session setup and are available in server->auth_key
3) The SMB Message with a sequence number(http://ubiqx.org/cifs/SMB.html#SMB.8.9.2) set at smb_hdr->Extra->Signature.
The 3 values above are concatenated and a md5 hash generated.

http://ubiqx.org/cifs/SMB.html#SMB.8.9.3
The function setup_ntlm_response() is used to calculate the first part of the MAC
This is the Session-key & Response part of the MAC(shown as MAC_Key in IC).
At the end of this function, we have
1) Session key at ses->auth_key.response. 16 bytes
2) md4 hash of NTLM response in ses->auth_key.response+16. 24 bytes

We sign the SMB in SendReceive()


SendReceive(const unsigned int xid, struct cifsSesInfo *ses,
            struct smb_hdr *in_buf, struct smb_hdr *out_buf,
            int *pbytes_returned, const int long_op)
{
..
        rc = cifs_sign_smb(in_buf, ses->server, &midQ->sequence_number);
..
}

int cifs_sign_smb(struct smb_hdr *cifs_pdu, struct TCP_Server_Info *server,
                  __u32 *pexpected_response_sequence_number)
{
        int rc = 0;
        char smb_signature[20];

        if ((cifs_pdu == NULL) || (server == NULL))
                return -EINVAL;

        if ((cifs_pdu->Flags2 & SMBFLG2_SECURITY_SIGNATURE) == 0)
                return rc;

//Obtain global lock on the MIDs
        spin_lock(&GlobalMid_Lock);
//Set the sequence number in the smb_hdr->Extra->Signature field.
// http://ubiqx.org/cifs/SMB.html#SMB.8.9.2
        cifs_pdu->Signature.Sequence.SequenceNumber =
                        cpu_to_le32(server->sequence_number);
        cifs_pdu->Signature.Sequence.Reserved = 0;

//Store the seq number for the response. 
//This is used to check the signature of the server sent response
// and is stores in the MID->sequence_number for the request.
        *pexpected_response_sequence_number = server->sequence_number++;
//Increment it one more time so that it is an even number for use by the next request.
        server->sequence_number++;
        spin_unlock(&GlobalMid_Lock);

//Now calculate signature.
        rc = cifs_calculate_signature(cifs_pdu, server, smb_signature);
-->
http://ubiqx.org/cifs/SMB.html#SMB.8.9.3
static int cifs_calculate_signature(const struct smb_hdr *cifs_pdu,
               struct TCP_Server_Info *server, char *signature)
{
struct  MD5Context context;

if (cifs_pdu == NULL || signature == NULL || server == NULL)
return -EINVAL;

//Initialise MD5
cifs_MD5_init(&context);
//Add the Session key and response which are concatanated at server->session_key.response
cifs_MD5_update(&context, server->session_key.response,
       server->session_key.len);
//Now update the md5 hash with the entire smb packet.
cifs_MD5_update(&context, cifs_pdu->Protocol, cifs_pdu->smb_buf_length);

cifs_MD5_final(signature, &context);
return 0;
}
<-- i="">

//We now overwrite the sequence number stored in smb_hdr->Extra->Signature field
//With the first 8 bytes of the signature we generated above.
        if (rc)
                memset(cifs_pdu->Signature.SecuritySignature, 0, 8);
        else
                memcpy(cifs_pdu->Signature.SecuritySignature, smb_signature, 8);

        return rc;
}

The packet so generated is sent to the server.

When receiving a response from the server, we verify the signature on the packet.


int
SendReceive(const unsigned int xid, struct cifsSesInfo *ses,
            struct smb_hdr *in_buf, struct smb_hdr *out_buf,
            int *pbytes_returned, const int long_op)
{
..
        rc = smb_send(ses->server, in_buf, in_buf->smb_buf_length);
..
//The stored sequence number is the sequence number of the request.
//We need to increment it for the response.
                        rc = cifs_verify_signature(out_buf,
                                                ses->server,
                                                midQ->sequence_number+1);
..
}

int cifs_verify_signature(struct smb_hdr *cifs_pdu,
                          struct TCP_Server_Info *server,
                          __u32 expected_sequence_number)
{
..
        /* save off the origiginal signature so we can modify the smb and check
                its signature against what the server sent */
        memcpy(server_response_sig, cifs_pdu->Signature.SecuritySignature, 8);
//We copy over the expected sequence number to the packet in place of the Signature.
        cifs_pdu->Signature.Sequence.SequenceNumber =
                                        cpu_to_le32(expected_sequence_number);
        cifs_pdu->Signature.Sequence.Reserved = 0;

//We then calculate the signature.
//Notes for cifs_calculate_signature() are given above.
        rc = cifs_calculate_signature(cifs_pdu, server,
                what_we_think_sig_should_be);

        if (rc)
                return rc;

//We then compare the first 8 bytes of the expected signature we generated with
//the signature sent by the server.
        if (memcmp(server_response_sig, what_we_think_sig_should_be, 8))
//If they do not match, it could be a man in the middle attack.
                return -EACCES;
        else
//It matches. So is fine.
                return 0;

}

The additional steps required for verifying signatures increases the load on the client/server. This is therefore switched off by default and only enabled when the user requests it.

NTLMv2 Authentication:
NTLMv2 authentication differs in the following ways from NTLM authentication.
1) The NTLMv2 hash is created using the NTLM hash and additional data.
3) The NTLMv2 Session key is created using additional data.
2) The client generates a random string of bytes to use as a client challenge(referred to as blob/blip) which is then used to create the response. It seperately sends the client challenge along with the response calculated in the NTLMv2 response to the server in the SESSION_SETUP_ANDX request. The server then creates its own response using the client challenge and compares it with the response to confirm authentication.

Monday, March 18, 2013

CIFS: Async Reads


A read call on the cifs fs goes through the following code path

sys_read -> vfs_read -> file->f_op->read() -> do_sync_read() -> filp->f_op->aio_read -> do_generic_file_read()


Within do_generic_file_read,

static void do_generic_file_read(struct file *filp, loff_t *ppos,
                read_descriptor_t *desc, read_actor_t actor)
{
..
        //Index of the page which contains the ppos.
        index = *ppos >> PAGE_CACHE_SHIFT;
..
        for (;;) {
..
find_page:
                //Find the page and get a reference.
                page = find_get_page(mapping, index);
                if (!page) {
                        page_cache_sync_readahead(mapping,
                                        ra, filp,
                                        index, last_index - index);
..
}

We use the read ahead code to fetch the read pages using the async read code.
This is implemented using the address_operations->readpages().

We go through the following code path before we get to the address_operations
page_cache_sync_readahead() -> ondemand_readahead() -> __do_page_cache_readahead()

We allocate all the pages required in the __do_page_cache_readahead following which we
call read_pages()

static int
__do_page_cache_readahead(struct address_space *mapping, struct file *filp,
                        pgoff_t offset, unsigned long nr_to_read,
                        unsigned long lookahead_size)
{
..
        /*
         * Preallocate as many pages as we will need.
         */
        for (page_idx = 0; page_idx < nr_to_read; page_idx++) {
..
                page = page_cache_alloc_readahead(mapping);
                if (!page)
                        break;
                page->index = page_offset;
                list_add(&page->lru, &page_pool);
                if (page_idx == nr_to_read - lookahead_size)
                        SetPageReadahead(page);
                ret++;
        }
..
        //then call read_pages().
        if (ret)
                read_pages(mapping, filp, &page_pool, ret);

..
}

static int read_pages(struct address_space *mapping, struct file *filp,
                struct list_head *pages, unsigned nr_pages)
{
..
        if (mapping->a_ops->readpages) {
                ret = mapping->a_ops->readpages(filp, mapping, pages, nr_pages);
..
}

const struct address_space_operations cifs_addr_ops = {
..
        .readpages = cifs_readpages,
..
}

We first start by locking the page. This lock on the page is not unlocked until the
read response from the server is received or the request times out. This is the mechanism
used to block the thread which started the read while it waits for the response to return.

static int cifs_readpages(struct file *file, struct address_space *mapping,
        struct list_head *page_list, unsigned num_pages)
{
..
        while (!list_empty(page_list)) {
..
                //We first lock the page, then we add it to the page cache.
                __set_page_locked(page);
                rc = add_to_page_cache_locked(page, mapping,
                                              page->index, GFP_KERNEL);

                /* give up if we can't stick it in the cache */
                if (rc) {
                        __clear_page_locked(page);
                        break;
                }
..
                rdata = cifs_readdata_alloc(nr_pages, cifs_readv_complete);
..
}

Within cifs_readdata_alloc, we initialise the struct work and set it to call
cifs_readv_complete()

static struct cifs_readdata *
cifs_readdata_alloc(unsigned int nr_pages, work_func_t complete)
{
..
                INIT_WORK(&rdata->work, complete);
..
}

Back to cifs_readpages()

static int cifs_readpages(struct file *file, struct address_space *mapping,
        struct list_head *page_list, unsigned num_pages)
{
..
        while (!list_empty(page_list)) {
..
                rdata = cifs_readdata_alloc(nr_pages, cifs_readv_complete);
                //If there was an error, free up all the pages we allocated
                // and return an error
                if (!rdata) {
                        /* best to give up if we're out of mem */
                        list_for_each_entry_safe(page, tpage, &tmplist, lru) {
                                list_del(&page->lru);
                                lru_cache_add_file(page);
                                unlock_page(page);
                                page_cache_release(page);
                        }
                        rc = -ENOMEM;
                        break;
                }
                //We get a reference to the open_file.
                rdata->cfile = cifsFileInfo_get(open_file);
                rdata->mapping = mapping;
                rdata->offset = offset;
                rdata->bytes = bytes;
                rdata->pid = pid;
                rdata->pagesz = PAGE_CACHE_SIZE;
                rdata->read_into_pages = cifs_readpages_read_into_pages;

                list_for_each_entry_safe(page, tpage, &tmplist, lru) {
                        list_del(&page->lru);
                        rdata->pages[rdata->nr_pages++] = page;
                }

                rc = cifs_retry_async_readv(rdata);
..
}

static int
cifs_retry_async_readv(struct cifs_readdata *rdata)
{
..
        do {
                //We re-open the file if the handle is invalid.
                if (rdata->cfile->invalidHandle) {
                        rc = cifs_reopen_file(rdata->cfile, true);
                        if (rc != 0)
                                continue;
                }
                //The async readv call is protocol specific.
                rc = server->ops->async_readv(rdata);
        } while (rc == -EAGAIN);
..
}

struct smb_version_operations smb1_operations = {
..
        .async_readv = cifs_async_readv,
..
}

int
cifs_async_readv(struct cifs_readdata *rdata)
{
..
        rc = small_smb_init(SMB_COM_READ_ANDX, wct, tcon, (void **)&smb);
..
        rc = cifs_call_async(tcon->ses->server, &rqst, cifs_readv_receive,
                             cifs_readv_callback, rdata, 0);
..
        return rc;
}

We call cifs_call_async() with the arguments for
mid_receive_t *receive is set to cifs_readv_receive
and for
mid_callback_t *callback is set to cifs_readv_callback

int
cifs_call_async(struct TCP_Server_Info *server, struct smb_rqst *rqst,
                mid_receive_t *receive, mid_callback_t *callback,
                void *cbdata, const int flags)
{
..
        //We first allocate a mid for the async_request.
        mid = server->ops->setup_async_request(server, rqst);
..
        //We set the receive function pointer to cifs_readv_receive()
        mid->receive = receive;
        //We set the callback function pointer to cifs_readv_callback()
        mid->callback = callback;
        mid->callback_data = cbdata;
        mid->mid_state = MID_REQUEST_SUBMITTED;

        /* put it on the pending_mid_q */
        spin_lock(&GlobalMid_Lock);
        list_add_tail(&mid->qhead, &server->pending_mid_q);
        spin_unlock(&GlobalMid_Lock);
..
        rc = smb_send_rqst(server, rqst);
..
        if (rc == 0)
                return 0;

        //If there was an error, delete the mid and return an error.
        cifs_delete_mid(mid);
..
        return rc;
}

The data is sent to the server using the smb_send_rqst() call. cifs_call_async() then returns immediately.
If we had encoountered an error at this point, the error is propagated back to cifs_readpages() where the
allocated pages are freed.

static void do_generic_file_read(struct file *filp, loff_t *ppos,
                read_descriptor_t *desc, read_actor_t actor)
{
..
                if (!page) {
                        page_cache_sync_readahead(mapping,
                                        ra, filp,
                                        index, last_index - index);
                        //Fetch the page again. This should get you the page since we have allocated and
                        //attached the page to the page cache above.
                        page = find_get_page(mapping, index);
                        if (unlikely(page == NULL))
                                goto no_cached_page;
                }
..
                if (!PageUptodate(page)) {
                        if (inode->i_blkbits == PAGE_CACHE_SHIFT ||
                                        !mapping->a_ops->is_partially_uptodate)
                                goto page_not_up_to_date;
                        //We should hit this page condition since the page is locked in cifs_readpages()
                        if (!trylock_page(page))
                                goto page_not_up_to_date;
..
page_not_up_to_date:
                //We first attempt to lock the page. It blocks here since we have locked the page in
                //cifs_readpages() and do not unlock it until we have received the response from the server.
                /* Get exclusive access to the page ... */
                error = lock_page_killable(page);
..
}

The thread which triggered the read is blocked at this point waiting for the response to the read call sent earlier.

The response to the read call is received by the demultiplex thread for that share.

static int
cifs_demultiplex_thread(void *p)
{
..
        struct mid_q_entry *mid_entry;
..
        while (server->tcpStatus != CifsExiting) {
..
                mid_entry = server->ops->find_mid(server, buf);
..
                if (!mid_entry || !mid_entry->receive)
                        length = standard_receive3(server, mid_entry);
                else
                        length = mid_entry->receive(server, mid_entry);
..
                if (mid_entry != NULL) {
                        if (!mid_entry->multiRsp || mid_entry->multiEnd)
                                mid_entry->callback(mid_entry);
..
}

mid->receive is set to cifs_readv_receive()
mid->callback is set to cifs_readv_callback()
with the call made to cifs_call_async() in cifs_async_readv()

We first call the receive function.

int
cifs_readv_receive(struct TCP_Server_Info *server, struct mid_q_entry *mid)
{
..
        //We receive the data
        length = cifs_readv_from_socket(server, &rdata->iov, 1, len);
        if (length < 0)
                return length;
        server->total_read += length;

        //If we did encounter an error, discard the data returned.
        /* Was the SMB read successful? */
        rdata->result = server->ops->map_error(buf, false);
        if (rdata->result != 0) {
                cFYI(1, "%s: server returned error %d", __func__,
                        rdata->result);
                return cifs_readv_discard(server, mid);
        }
..
        //read the read data
        data_offset = server->ops->read_data_offset(buf) + 4;
..
       length = rdata->read_into_pages(server, rdata, data_len);
..
}

We then call the callback function from the demultiplex thread.

static void
cifs_readv_callback(struct mid_q_entry *mid)
{
        struct cifs_readdata *rdata = mid->callback_data;
        struct cifs_tcon *tcon = tlink_tcon(rdata->cfile->tlink);
        struct TCP_Server_Info *server = tcon->ses->server;
        struct smb_rqst rqst = { .rq_iov = &rdata->iov,
                                 .rq_nvec = 1,
                                 .rq_pages = rdata->pages,
                                 .rq_npages = rdata->nr_pages,
                                 .rq_pagesz = rdata->pagesz,
                                 .rq_tailsz = rdata->tailsz };

..
//The work is queued in the cifsiod workqueue.
        queue_work(cifsiod_wq, &rdata->work);
        DeleteMidQEntry(mid);
        add_credits(server, 1, 0);
}

The work function is set to
cifs_readv_complete()
in
cifs_readpages() -> cifs_readdata_alloc()

static void
cifs_readv_complete(struct work_struct *work)
{
        unsigned int i;
        struct cifs_readdata *rdata = container_of(work,
                                                struct cifs_readdata, work);

        //For each page
        for (i = 0; i < rdata->nr_pages; i++) {
                struct page *page = rdata->pages[i];

                //add the page to the lru_cache
                lru_cache_add_file(page);

                //Set the page as Uptodate.
                if (rdata->result == 0) {
                        flush_dcache_page(page);
                        SetPageUptodate(page);
                }

                //And unlock the page.
                unlock_page(page);
..
}

As we unlock the page, we unblock the read thread which was blocked
and waiting for the page lock at do_generic_file_read().

We thus complete a successful read.

What happens if we encounter an error when performing an Async read?

We look at the code path when we encounter an error in the response from the server

int
cifs_readv_receive(struct TCP_Server_Info *server, struct mid_q_entry *mid)
{
..
        //We receive the data
        length = cifs_readv_from_socket(server, &rdata->iov, 1, len);
        if (length < 0)
                return length;
        server->total_read += length;
        //If we did encounter an error, discard the data returned.
        /* Was the SMB read successful? */
        rdata->result = server->ops->map_error(buf, false);
        if (rdata->result != 0) {
                cFYI(1, "%s: server returned error %d", __func__,
                        rdata->result);
                return cifs_readv_discard(server, mid);
        }
..
}


The result is stored in rdata->result. A non zero value indicates an error. On encountering
an error, the mid is dequeued.

static int
cifs_readv_discard(struct TCP_Server_Info *server, struct mid_q_entry *mid)
{
..
        dequeue_mid(mid, rdata->result);
        return 0;
}


It returns 0 which is propagated to cifs_readv_receive() -> cifs_demultiplex_thread()

static int
cifs_demultiplex_thread(void *p)
{
..
        struct mid_q_entry *mid_entry;
..
        while (server->tcpStatus != CifsExiting) {
..
                mid_entry = server->ops->find_mid(server, buf);
..
                if (!mid_entry || !mid_entry->receive)
                        length = standard_receive3(server, mid_entry);
                else
                        length = mid_entry->receive(server, mid_entry);

                if (length < 0)
                        continue;
..
                if (mid_entry != NULL) {
                        if (!mid_entry->multiRsp || mid_entry->multiEnd)
                                mid_entry->callback(mid_entry); <-- callback="" i="">
..
}

It then proceeds to call the callback function cifs_readv_callback().
We queue the work and deletes the mid.

static void
cifs_readv_callback(struct mid_q_entry *mid)
{
..
        queue_work(cifsiod_wq, &rdata->work);
        DeleteMidQEntry(mid);
..
}

The thread cifsiod runs the workqueue. This calls the function cifs_readv_complete()

static void
cifs_readv_complete(struct work_struct *work)
{
        unsigned int i;
        struct cifs_readdata *rdata = container_of(work,
                                                struct cifs_readdata, work);

        //For each page
        for (i = 0; i < rdata->nr_pages; i++) {
                struct page *page = rdata->pages[i];

                //Since we hit an error, we do not set
                //page uptodate here
                if (rdata->result == 0) {
                        flush_dcache_page(page);
                        SetPageUptodate(page);
                }

                //We unlock the page.
                unlock_page(page);
..
}

As we unlock the page, we unblock the read thread which was blocked
and waiting for the page lock at do_generic_file_read().

static void do_generic_file_read(struct file *filp, loff_t *ppos,
                read_descriptor_t *desc, read_actor_t actor)
{
..
page_not_up_to_date:
                /* Get exclusive access to the page ... */
                error = lock_page_killable(page);
                if (unlikely(error))
                        goto readpage_error;
                //Once we receive the lock, we proceed.

                //the page is not set as uptodate. We therefore skip this.
                /* Did somebody else fill it already? */
                if (PageUptodate(page)) {
                        unlock_page(page);
                        goto page_ok;
                }
..
readpage:
..
                //Call a_ops->readpage() which calls the sync version of cifs_read
                /* Start the actual read. The read will unlock the page. */
                error = mapping->a_ops->readpage(filp, page);
..
}

On receiving an error when attempting an async read, we ignore the error and drop back to using sync reads.


Sunday, July 08, 2012

Building modules for the latest Raspberry Pi firmware

The latest firmware for the Raspberry Pi is made available through the git repository. The firmware built is based on the kernel sources which are also made available through a separate git repository.

Users may want to install the latest firmware to avail of the latest fixes available for the Raspberry Pi kernel. However changes in the kernel necessitates the rebuild of any customer kernel modules installed. These are kernel modules which are not available on the usual kernel which was distributed. In my case, I need to recompile the drivers for my wireless card which aren't available on the stock kernel. 

The steps given below is to be used when you recompile your drivers from within the Raspberry Pi.

Before we start, we need to make sure that we have the required tools.
# apt-get update
# apt-get install git gcc make
To update the firmware to the latest one from the git repo, follow the steps below

Download the latest firmware repo
#  git clone --depth 1 https://github.com/raspberrypi/firmware.git
The argument --depth 1 limits the firmware downloaded to only the latest available in the repo.
OR
Update your existing firmware repo by first changing into the firmware repo directory and running the command
# git pull 
You now have to download the latest source code from the git repo. The 256 MB of memory available on the Raspberry Pi isn't sufficient to run the git clone command. To get around this, you can download the git repo on a different machine and then copy over the repo to the Raspberry Pi.
To clone the git repo, use the command
# git clone --branch rpi-patches https://github.com/raspberrypi/linux.git rapi-linux
The argument --branch rpi-patches specifies that you would like to download the branch rpi-patches which is the one on which the kernel is built.
Now copy over the directory rapi-linux created to the Raspberry Pi.
OR
Update the existing sources directory by first changing into the sources repo directory and running the command
# git checkout rpi-patches
# git pull
I have the git repos downloaded to the following location on the Raspberry Pi
/root/firmware
/root/rapi-linux
Now to build the kernel headers required to build the module.

1) Determine the git hash which represents the kernel used to build the firmware and check out the sources.
# cat /root/firmware/extra/git_hash 
85b7821857dd0b9cabab59d47f08eabed74679a3
Use this hash to checkout the kernel sources.
# cd /root/rapi-linux
# git checkout 85b7821857dd0b9cabab59d47f08eabed74679a3


2) Prepare the sources so that they can be used to build the kernel module.
Reset the sources to clear any artifacts from an earlier build
# make mrproper
Copy over the kernel configuration and prepare sources.
# zcat /proc/config.gz > .config
# make oldconfig
# make modules_prepare
3) Copy over the files require to build the module from the firmware repo
# cp /root/firmware/extra/Module.symvers /root/rapi-linux
The kernel sources required to build the kernel module is now complete.

To build the module, change into the directory containing the module sources and run the command
# cd /root/rtl8192_8188CU_linux_v3.0.2164.20110715/
# make -C /root/rapi-linux/ M=`pwd`
We pass the location of the kernel sources directory with the -C parameter. 
We pass the location to the module sources using the M= parameter.

To update the kernel on the Raspberry Pi, change into the firmware directory and copy over the required files.
First backup the existing files 
# cd /root/firmware
mkdir -p /root/fw.backups/
# rsync -av /boot /root/fw.backups/
# rsync -av /lib/modules/3.1.9+ /root/fw.backups/
Now copy over the new files
# cp -av boot/* /boot/
# cp -av modules/3.1.9+/* /lib/modules/3.1.9+/
Run depmod to build the module dependencies for modprobe
# depmod -a
Perform any additional steps required for your custom modules. In my case, I need to copy over my newly built wireless module to the new modules folder.

# cp ~/rtl8192_8188CU_linux_v3.0.2164.20110715/8192cu.ko /lib/modules/3.1.9+/kernel/drivers/net/wireless/
# depmod -a 
Now reboot the Raspberry Pi so that the new kernel can be loaded.

Wednesday, July 04, 2012

Building kernel modules on the Raspberry Pi

To enable wireless on my RaPi, I bought a Edimax wireless usb dongle. This is based on the Realtek chipset which unfortunately was not supported out of the box on the 2012-06-18-wheezy-beta image I was running.

I was following the instructions posted here to enable the wireless dongle but soon found that the binary driver posted doesn't match the kernel version(3.1.9+ #144) I was running on my RaPi.  This meant that I had to build the kernel module required to run the wireless USB dongle from sources. The kernel headers for the running kernel are not provided in the Debian repository or the Rapi firmware git repository. Instead, the kernel sources used to build the kernel have to be used to setup the build environment.

Given below are instructions to build the kernel module for the latest development kernel on the Rapi as well as cross compile on a Linux based laptop or desktop. The limited resources on the RaPi means that building the kernel on the RaPi will take a long time. For this reason, I recommend that you cross compile on your desktop and then copy over the results to the RaPi.

Building on the RaPi.

1) Install git. Also make sure gcc and make are available on your Rapi.

# apt-get update
# apt-get install git make gcc

2) Clone the RaPi kernel git repository

# git clone --branch rpi-patches --depth 1 https://github.com/raspberrypi/linux.git rapi-linux

In my case, I cloned the branch on another machine and copied the resulting directory rapi-linux over to the RaPi at the location /root/rapi-linux.

3) Create a symlink to the kernel build directory used by the build process

# cd /lib/modules/3.1.9+/
# ln -s /root/rapi-linux/ build

4) Copy over the config file and prepare the sources

# cd /root/rapi-linux
# make mrproper
# zcat /proc/config.gz > .config
# make oldconfig
# make modules_prepare

You can build your kernel module at this stage but will find that the build complains about Module.symvers being missing. Attempting to load the resulting module fails with the error "Error: could not insert module ./8192cu.ko: Invalid module format" and the following line in dmesg "no symbol version for module_layout".

To complete the build environment, you will have to generate a Module.symvers file by building a kernel. To do this, run the command

# make vmlinux


This will take some time to build.


To build the module on the RaPi, simply change in to the module sources directory and run

# make -C /lib/modules/`uname -r`/build/ M=`pwd`


OR

Cross-compiling on a x86 based environment.

If you have a Linux based system, you could cross compile on your laptop and copy the result to the Rapi.

The following commands work for Fedora. You will need to adjust to get it working for your distribution


1) Install git, gcc, make and  gcc-arm-linux-gnu required to cross compile the kernel


# yum install -y git make gcc  gcc-arm-linux-gnu

2) Clone the RaPi kernel git repository

# git clone --branch rpi-patches --depth 1 https://github.com/raspberrypi/linux.git rapi-linux

3) Copy over the config file from the RaPi and prepare the sources

On the RaPi
zcat /proc/config.gz > /tmp/config


Copy over /tmp/config on the RaPi to your own system. Now copy it over to the git repo you just cloned.


On the Laptop/Desktop
# cd rapi-linux
# cp /tmp/config .config
# make ARCH=arm CROSS_COMPILE=/usr/bin/arm-linux-gnu- oldconfig
# make ARCH=arm CROSS_COMPILE=/usr/bin/arm-linux-gnu- vmlinux

At this point, your build environment is complete.


To build a module on the linux desktop, change into the module sources and run

make ARCH=arm CROSS_COMPILE=/usr/bin/arm-linux-gnu- -C ../rapi-linux/ M=`pwd`

Replace ../rapi-linux/ with the location to your own RaPi kernel sources.



Setting a frost alarm on Home assistant

One of the issues with winter is the possibility of ice covering your windscreen which needs to be cleared before you can drive. Clearing ou...