Chris,
I will follow-up as soon as I have the answers.
Thanks,
Edgar
-----Original Message-----
From: Christopher R. Hertel [mailto:crh at samba.org]
Sent: Saturday, June 18, 2011 10:53 PM
To: Edgar Olougouna
Cc: cifs-protocol at samba.org
Subject: Re: [cifs-protocol] [REG:111061756137964] Encryption of the key for "netsh branchcache” importkey and exportkey.
Okay, I'm wrong. The IV does make a difference.
So... My questions again:
QUESTION: From what input is the Initialization Vector (IV) derived?
Is it constant, derived from the initial passphrase, or derived
from the SHA256 of the passphrase?
QUESTION: What algorithm is used to derive the IV or, if the IV is
constant, what values are used?
QUESTION: Is the passphrase handled as Unicode or ASCII?
I'm Assuming Unicode.
QUESTION: Is the AES mode used AES-CBC? AES-ECB? Other?
Two More
QUESTION: There isn't any salt used, is there?
QUESTION: You did write "prepended", so if the file I exported is 80
bytes long, it should be:
[32-bytes SHA256(X)] + [48-bytes X]
...where X is the server secret.
I just can't seem to get this to decipher, and it's probably just something basic that I'm missing... like the IV.
Chris -)-----
Christopher R. Hertel wrote:
> Okay, never mind the IV part. I don't think that the IV actually
> makes a difference when decrypting.
>> ...but I could be wrong.
>> Christopher R. Hertel wrote:
>> Edgar,
>>>> There is a little more information that I will need in order to make
>> this work. In particular, the AES encryption algorithm makes use of
>> an Initialization Vector which is typically derived from the
>> passphrase, as is the key.
>>>> Your description tells me how the key is derived, but not how the IV is derived.
>>>> QUESTION: Is the IV derived from the initial passphrase or from the
>> SHA256 of the passphrase? ...or is it a constant?
>>>> QUESTION: What algorithm is used to derive the IV or, if the IV is
>> constant (including NULL), what values are used?
>>>> QUESTION: Is the passphrase handled as Unicode or ASCII?
>> I'm Assuming Unicode.
>>>> QUESTION: Is the AES mode used AES-CBC? AES-ECB? Other?
>>>> Thanks.
>>>> Chris -)-----
>>>> Edgar Olougouna wrote:
>>> Chris,
>>>>>> The algorithm is as follows.
>>>>>> Let’s assume, Bs is the server configured binary string (or whatever is generated if none was configured) for the secret.
>>> Let’s assume, Ks is the server secret, by definition Ks is SHA-256 hash of Bs.
>>>>>> When you run "netsh branchcache exportkey”, the key file is encrypted as follows:
>>> • A buffer is generated, consisting of Bs prepended with Ks.
>>> • The buffer is encrypted using the AES-256 algorithm. The SHA-256 hash of the passphrase string, excluding the string’s NULL terminator, is used as the encryption key.
>>> • The resulting encrypted buffer is written to the output file.
>>>>>> During import (netsh branchcache importkey), the input file (which was the output file in the export) is decrypted using the SHA-256 hash of the passphrase as key. The value of Bs is validated against Ks, which is the SHA-256 hash of Bs.
>>>>>> NOTE:
>>> I am pretty sure you have seen this is the document, but for a refresher for a larger audience, the server secret key is defined in MS-PCCRC, as a SHA-256 hash of an arbitrary length binary string (see excerpts below).
>>>>>> MS-PCCRC
>>> 1.1 Glossary
>>> server secret: A SHA-256 hash of an arbitrary length binary string stored on the server.
>>> 1.3 Overview
>>> In order to ensure that cache content retrieval communications are at least as secure as a normal client's communications with a content server, all content servers must be configured with a binary secret value of arbitrary length. This secret is used as a key to derive secret keys to be used to secure communications between the peers or between peers and a Hosted Cache. If the secret value is not configured, it is automatically generated.
>>>>>> Best regards,
>>> Edgar
>>>>>> -----Original Message-----
>>> From: Edgar Olougouna
>>> Sent: Friday, June 17, 2011 10:49 AM
>>> To: 'Christopher R. Hertel'
>>> Cc: cifs-protocol at samba.org>>> Subject: RE: [cifs-protocol] [REG:111061069959469] BranchCache and SMB2: Questions specific to BranchCache.
>>>>>> Chris,
>>>>>> Regarding the question on the server secret, I will be replying shortly and describe the algorithm in a new thread [REG:111061756137964] Encryption of the key for "netsh branchcache” importkey and exportkey.
>>> That way you can achieve the decryption of the exported server secret and verify your implementation.
>>>>>> Regarding your feedback on the glossary definition of "segment secret" in [MS-PCCRC]... I have passed on your observations to the product team.
>>>>>> Thanks,
>>> Edgar
>>>>>> -----Original Message-----
>>> From: Christopher R. Hertel [mailto:crh at ubiqx.com]
>>> Sent: Friday, June 17, 2011 12:11 AM
>>> To: Edgar Olougouna
>>> Cc: cifs-protocol at samba.org>>> Subject: Re: [cifs-protocol] [REG:111061069959469] BranchCache and SMB2: Questions specific to BranchCache.
>>>>>> Edgar,
>>>>>> Thanks for the clarifications.
>>>>>> Regarding the dwOffsetInFirstSegment and dwReadBytesInLastSegment fields...
>>>>>> I have a working prototype, written as a CGI script and running under Apache. It seems to be giving correct results, when compared to a W2K8r2 server, but I cannot verify that I am correctly generating the Segment Secret correctly because I do not know the Server Secret generated by the Windows server.
>>>>>> I can extract the Windows Server Secret using a netsh command, but the command encrypts the Server Secret with a password, and I do not know what algorithm is used to perform the encryption. If I did, I could decrypt the Server Secret and use it in my test program to verify that I am generating the hash correctly.
>>>>>> Do you know where I could find documentation that would tell me what algorithm is being used to encrypt the extracted Server Secret?
>>>>>>>>>>>> Regarding the glossary definition of "segment secret" in [MS-PCCRC]...
>>>>>> I would suggest that the use of "authenticated" in the context of the description of "segment secret" is misleading. [MS-PCCRC] covers a variety of server types, and in many cases there is no authentication required. I'm testing using HTTP, for example, and my client is being given Content Information without performing any sort of authentication.
>>>>>> The fact that SMB2 requires authentication before providing access to a content or content information is an aspect of SMB2, not BranchCache. There is no authorization required by the Peer Content Caching and Retrieval:
>>> Content Identification protocol itself.
>>>>>> Thanks.
>>>>>> See you next week.
>>>>>> Chris -)-----
>>>>>>>>> Edgar Olougouna wrote:
>>>> Chris,
>>>>>>>> For the question regarding "authorized client", BranchCache considers possession of the Content Information data structure to be equivalent to possession of data for access control purposes. If a client was able to obtain hashes from a file server, we will allow that client to retrieve data from peers or the hosted cache server. A file server is expected to protect the content information data structure the same way it protects actual files. If a file server authenticates and authorizes clients before allowing them to read files, it should do the same before providing them with hashes.
>>>>>>>> Yes, your interpretation regarding dwOffsetInFirstSegment and dwReadBytesInLastSegment appears to be correct.
>>>> Basically, dwReadBytesInLastSegment is a count of bytes you can read in the last segment. If the range is one segment long then the last segment is the only segment so dwOffsetInFirstSegment and dwReadBytesInLastSegment both apply to it. If the range is multiple segments long then dwOffsetInFirstSegment applies to the first segment and dwReadBytesInLastSegment applies to the last segment.
>>>>>>>> In my previous communication, it should be Windows Server 2008 R2 instead of Windows 7. Only Windows Server 2008 R2 servers could be BranchCache servers, so both OS versions cannot be used interchangeably in this case. Thanks for catching this.
>>>>>>>> Finally, I have passed on your feedback to the product team and suggested that an example of range request being added to the document.
>>>> Thanks again for the numerous observations.
>>>>>>>> Regards,
>>>> Edgar
>>>>>>>> -----Original Message-----
>>>> From: Edgar Olougouna
>>>> Sent: Friday, June 10, 2011 2:14 PM
>>>> To: 'Christopher R. Hertel'
>>>> Cc: cifs-protocol at samba.org>>>> Subject: RE: [cifs-protocol] [REG:111051857287367] BranchCache and SMB2: Questions specific to BranchCache.
>>>>>>>> Chris,
>>>>>>>> Thanks for the numerous observations. I will review the questions and follow-up as soon as I have an update.
>>>>>>>> Regards,
>>>> Edgar
>>>>>>>> -----Original Message-----
>>>> From: Christopher R. Hertel [mailto:crh at ubiqx.com]
>>>> Sent: Friday, June 10, 2011 12:57 AM
>>>> To: Edgar Olougouna
>>>> Cc: cifs-protocol at samba.org>>>> Subject: Re: [cifs-protocol] [REG:111051857287367] BranchCache and SMB2: Questions specific to BranchCache.
>>>>>>>> Edgar,
>>>>>>>> One more observation and another quick question:
>>>>>>>> Question:
>>>>>>>> The [MS-PCCRC] Glossary (section 1.1) gives the following definition:
>>>>>>>> segment secret: The content-specific hash that is sent to authorized
>>>> clients along with the rest of the content information. It is
>>>> generated by hashing the concatenation of the segment hash of data
>>>> and the server secret.
>>>>>>>> What does the phrase "authorized clients" mean? How is a client
>>>> authorized in this context?
>>>>>>>> Observation:
>>>> Now that I understand the purpose of the dwOffsetInFirstSegment and
>>>> dwReadBytesInLastSegment fields it appears that the descriptions of
>>>> these fields in [MS-PCCRC], though inscrutable, are plausible.
>>>>>>>> In particular, the following now makes sense:
>>>>>>>> dwReadBytesInLastSegment (4 bytes): Total number of bytes of the
>>>> content range which lie within the final segment in the Content
>>>> Information data structure.
>>>>>>>> If the requested range includes multiple segments, then the offset
>>>> of the dwReadBytesInLastSegment bytes to be read is 0, relative to
>>>> the segment. What is interesting to note is that if the requested
>>>> range is included in only ONE segment, then the
>>>> dwReadBytesInLastSegment bytes to be read start at the offset
>>>> indicated by dwOffsetInFirstSegment.
>>>>>>>> Chris -)-----
>>>>>>>>>>>> Christopher R. Hertel wrote:
>>>>> Edgar,
>>>>>>>>>> Thanks again for this reply. I have some comments which I hope
>>>>> you will pass along to the documentation team and engineers.
>>>>>>>>>> On Tue, May 31, 2011 at 08:43:22PM +0000, Edgar Olougouna wrote:
>>>>>> Chris,
>>>>>>>>>>>> Please find our answers as follows.
>>>>>>>>>>>> BranchCache divides content into 32 megabyte segments and further
>>>>>> subdivides these segments into 64 kilobyte blocks.
>>>>> Yes. [MS-PCCRC] does a good job of explaining that part.
>>>>>>>>>>> The content information encodes:
>>>>>> (1) ullOffsetInContent - the offset within the content at which
>>>>>> the first segment present in the content information begins and
>>>>> Again yes, this makes sense to me.
>>>>>>>>>>> (2) dwOffsetInFirstSegment - the offset from ullOffsetInContent
>>>>>> at which the first block of the first segment present in the
>>>>>> content information begins.
>>>>> This is where things get confusing.
>>>>>>>>>>> Let's walk through this with an example.
>>>>>> Let's assume the client requests a content range starting at
>>>>>> content offset 33554433 (a.k.a. 32 megabytes plus one byte) and
>>>>>> extending for
>>>>>> 10 bytes.
>>>>> Problem #1 (for me): The examples given in the document
>>>>> specifically state that they show requests for entire files. From section 3.1:
>>>>>>>>>> "A client requests the entirety of the 125 kilobyte content from the
>>>>> server. The server responds with Content Information of the following
>>>>> form."
>>>>>>>>>> >From section 3.2:
>>>>>>>>>> "The same server S now receives a client request for a 125-megabyte
>>>>> file."
>>>>>>>>>> ...so, the examples in [MS-PCCRC] do not cover the use of ranges.
>>>>> These are whole-file requests, so there is no example of how
>>>>> dwOffsetInFirstSegment or dwReadBytesInLastSegment might be used
>>>>> (though there are descriptions).
>>>>>>>>>>> ullOffsetInContent will be 32 megabytes because that is where the
>>>>>> first segment present in the content information begins.
>>>>> Makes sense.
>>>>>>>>>>> dwOffsetInFirstSegment will be 1 because the first segment in the
>>>>>> content information is the second segment in the actual content
>>>>>> because the requested content range starts at 32 megabytes plus one.
>>>>> So... If I understand this correctly, the situation is that
>>>>> BranchCache must operate in terms of Segments, and when it returns
>>>>> the Segment Information for this segment, it is including an
>>>>> offset, relative to the start of the Segment, at which the *requested* content begins.
>>>>>>>>>> Is that correct?
>>>>>>>>>>> dwReadBytesInLastSegment will be 10 because that is the length of
>>>>>> the content range that lies within the final segment present in
>>>>>> the content information.
>>>>> If my new understanding of dwOffsetInFirstSegment is correct, then
>>>>> I believe that my understanding of dwReadBytesInLastSegment is now
>>>>> also correctly updated.
>>>>>>>>>> Basically, if the client requests a range the server must send
>>>>> Content Information covering the segments that completely include
>>>>> the requested range. These two values indicate how to handle the
>>>>> cached segments when the client retrieves them from the cache, as follows:
>>>>>>>>>> dwOffsetInFirstSegment - Discard this many leading bytes of the first
>>>>> segment retrieved from cache.
>>>>>>>>>> Retain all bytes of all subsequent segments
>>>>> retrieved from cache except the last one.
>>>>>>>>>> dwReadBytesInLastSegment - Retain only this many leading bytes of the
>>>>> last segment retrieved from cache.
>>>>>>>>>> Is that correct?
>>>>>>>>>>> Note that dwReadBytesInLastSegment may be zero if the client did
>>>>>> not request a content range (i.e. the client asked for the entire content).
>>>>> So a dwReadBytesInLastSegment value of zero is interpreted as "all bytes"
>>>>> in the segment.
>>>>>>>>>>> The scenarios in which dwReadBytesInLastSegment will be zero are
>>>>>> Windows specific behavior. In Windows 7, this happens whenever
>>>>>> the content information covers the entire content. In order to
>>>>>> participate in the MS-PCCRC protocol, a client must be able to
>>>>>> parse content information that contains a zero value for
>>>>>> dwReadBytesInLastSegment. The client must react by treating the
>>>>>> segment size for the final segment encoded in the content
>>>>>> information as the amount of data in the final segment. If
>>>>>> dwReadBytesInLastSegment is non-zero then the client must only
>>>>>> attempt to read dwReadBytesInLastSegment from the final segment, regardless of the segment size of the final segment.
>>>>> Not quite... Assuming that I'm catching on here, and that such a
>>>>> bug could happen, the goal would be something like the following:
>>>>>>>>>> if 0 == dwReadBytesInLastSegment
>>>>> readbytes = cbSegment
>>>>> else
>>>>> readbytes = min( dwReadBytesInLastSegment, cbSegment )
>>>>>>>>>> If, due to some bug, the dwReadBytesInLastSegment were greater
>>>>> than the number of actual bytes in the segment then you would
>>>>> still want to read only the number of actual bytes in the segment.
>>>>> I can envision something like this happing if the requested range
>>>>> extended beyond the end of the Content and the server did not clean up the results properly.
>>>>>>>>>> Okay, so I am being picky here... but it serves to verify whether
>>>>> or not I am understanding these fields.
>>>>>>>>>>> On Windows 7, if the content fits in one segment and the client
>>>>>> issued a request for the entire content then,
>>>>>> dwReadBytesInLastSegment should be zero and dwOffsetInFirstSegment should be zero.
>>>>> It was my understanding, based on my reading of a number of
>>>>> several MSDN web pages, that only Windows 2008R2 servers could be BranchCache servers.
>>>>> Windows 7 systems could only be clients. Vista and Windows 2008
>>>>> systems could also be clients if an additional package was installed.
>>>>> (W2Kr2 servers can also be BranchCache clients, I believe.)
>>>>>>>>>> The idea that Windows 7 cannot be a BranchCache Server is somewhat
>>>>> supported by the following Windows Behavior note from [MS-PCCRC]:
>>>>>>>>>> "<2> Section 2.3: In Windows 7, the [MS-PCCRR] implementation can only
>>>>> accept segment IDs generated using SHA-256, whereas the Windows Server
>>>>> 2008 R2 implementation of the hash generation part can generate segment
>>>>> IDs using one of three hashing algorithms: SHA-256, SHA- 384, or
>>>>> SHA-512."
>>>>>>>>>> Note that the behavior note only talks about Windows 7 in client
>>>>> mode and says nothing about its capabilities in server mode. I
>>>>> believe that's because Windows 7 doesn't support server mode.
>>>>> Also, only a Windows
>>>>> 2008r2 server can act as a hosted cache. Windows 7 clients, I
>>>>> believe, can operate as peer caches.
>>>>>>>>>> So... Do you mean to say Windows 7 or Windows 2Kr2 in the statement above?
>>>>> If Windows 7, then are you talking about W7 clients sending
>>>>> content requests to the hosted cache server or as broadcasts on the local LAN?
>>>>> If so, that part of BranchCache is outside of the scope of [MS-PCCRC] ...
>>>>> which leaves me confused as to what you are trying to explain to
>>>>> me when you describe Windows 7 behavior.
>>>>>>>>>>> The product team will be reflecting the above clarification in
>>>>>> the protocol document. Let me know whether this answers your questions.
>>>>> It would be quite helpful if an additional example were presented
>>>>> in order to show how these values are used. I found the
>>>>> descriptions of these values to be difficult to interpret when reading the documentation.
>>>>>>>>>> Unless you tell me that my understanding, given above, is
>>>>> incorrect I believe that your description has clarified things nicely.
>>>>>>>>>> Thanks.
>>>>>>>>>> Chris -)-----
>>>>>>>>>>> Regards,
>>>>>> Edgar
>>>>>>>>>>>> -----Original Message-----
>>>>>> From: Edgar Olougouna
>>>>>> Sent: Tuesday, May 31, 2011 11:07 AM
>>>>>> To: 'Christopher R. Hertel'
>>>>>> Cc: cifs-protocol at samba.org>>>>>> Subject: RE: [cifs-protocol] [REG:111051857287367] BranchCache and SMB2: Questions specific to BranchCache.
>>>>>>>>>>>> Chris,
>>>>>>>>>>>> I am actively working on this with the product team to clarify a few more points. I will update you as soon as possible.
>>>>>>>>>>>> Regards,
>>>>>> Edgar
>>>>>>>>>>>> -----Original Message-----
>>>>>> From: Christopher R. Hertel [mailto:crh at ubiqx.com]
>>>>>> Sent: Sunday, May 29, 2011 5:57 PM
>>>>>> To: Edgar Olougouna
>>>>>> Cc: MSSolve Case Email; cifs-protocol at samba.org>>>>>> Subject: Re: [cifs-protocol] [REG:111051857287367] BranchCache and SMB2: Questions specific to BranchCache.
>>>>>>>>>>>> Edgar,
>>>>>>>>>>>> I am just checking in on this ticket. I understand that the resolution to these issues can take some time, but I want to test my implementation at the FileSharing Plugfest in a few weeks time. Having the answers would help me complete my testing implementation.
>>>>>>>>>>>> Thanks.
>>>>>>>>>>>> Chris -)-----
>>>>>>>>>>>> Obaid Farooqi wrote:
>>>>>>> Hi Chris:
>>>>>>> Thank you for contacting Microsoft regarding your query on branch cache. A member of the protocol documentation team will be in touch soon.
>>>>>>>>>>>>>> Regards,
>>>>>>> Obaid Farooqi
>>>>>>> Escalation Engineer | Microsoft
>>>>>>>>>>>>>> Exceeding your expectations is my highest priority. If you
>>>>>>> would like to provide feedback on your case you may contact my
>>>>>>> manager at allisong at microsoft.com>>>>>>>>>>>>>>>>>>>>> -----Original Message-----
>>>>>>> From: Christopher R. Hertel [mailto:crh at ubiqx.com]
>>>>>>> Sent: Tuesday, May 17, 2011 9:12 PM
>>>>>>> To: Interoperability Documentation Help
>>>>>>> Subject: BranchCache and SMB2: Questions specific to BranchCache.
>>>>>>>>>>>>>> I have a couple of questions regarding BranchCache and [MS-PCCRC]. As you know, BranchCache--particularly the Content Information format specified in [MS-PCCRC]--can be used as an extension to SMB2.
>>>>>>>>>>>>>> 1) In the Content Information structure ([MS-PCCRC:2.3]), there is the
>>>>>>> following field definition:
>>>>>>>>>>>>>> dwOffsetInFirstSegment (4 bytes): Number of bytes into the first
>>>>>>> segment within the Content Information data structure at which
>>>>>>> the content range begins.
>>>>>>>>>>>>>> I can't make heads nor tails of that description, and the captures I
>>>>>>> have done on the wire typically show that field set to zero.
>>>>>>> Similarly, the examples in section 3 of [MS-PCCRC] show the value of
>>>>>>> this field to be zero.
>>>>>>>>>>>>>> The field is 4 bytes in length, so it cannot be an offset into the
>>>>>>> content itself, since content offsets are (should be) 8 bytes (64-bit).
>>>>>>>>>>>>>> Please clarify: What does dwOffsetInFirstSegment actually represent?
>>>>>>>>>>>>>> 2) I am not getting what I expect in the dwReadBytesInLastSegment field.
>>>>>>> This field is defined in [MS-PCCRC:2.3], as follows:
>>>>>>>>>>>>>> dwReadBytesInLastSegment (4 bytes): Total number of bytes of the
>>>>>>> content range which lie within the final segment in the Content
>>>>>>> Information data structure.
>>>>>>>>>>>>>> It seems to me that this value represents the number of bytes
>>>>>>> of actual content included in the calculation of the final
>>>>>>> Segment block of the Content Information. That interpretation
>>>>>>> is supported by the examples in section 3.
>>>>>>>>>>>>>> In actual captures, however, Windows IIS server returns zero
>>>>>>> (0x00000000) in the dwReadBytesInLastSegment field.
>>>>>>>>>>>>>> Is this a documentation bug or is the IIS server returning the
>>>>>>> wrong value?
>>>>>>>>>>>>>> This is IIS running on W2K8r2 with fairly up-to-date patches
>>>>>>> applied.
>>>>>>>>>>>>>> Looking forward to the replies.
>>>>>>>>>>>>>> Thanks.
>>>>>>>>>>>>>> Chris -)-----
>>>>>>>>>>>>>> --
>>>>>>> "Implementing CIFS - the Common Internet FileSystem" ISBN: 013047116X
>>>>>>> Samba Team -- http://www.samba.org/ -)----- Christopher R. Hertel
>>>>>>> jCIFS Team -- http://jcifs.samba.org/ -)----- ubiqx development, uninq.
>>>>>>> ubiqx Team -- http://www.ubiqx.org/ -)----- crh at ubiqx.mn.org>>>>>>> OnLineBook -- http://ubiqx.org/cifs/ -)----- crh at ubiqx.org>>>>>>>>>>>>> --
>>>>>> Christopher R. Hertel -)----- Storage Architect & CIFS Geek
>>>>>>http://www.ubiqx.com/ Data Storage and Systems Consulting
>>>>>> "Implementing CIFS - the Common Internet FileSystem" ISBN: 013047116X
>>>>>>>
--
"Implementing CIFS - the Common Internet FileSystem" ISBN: 013047116X
Samba Team -- http://www.samba.org/ -)----- Christopher R. Hertel
jCIFS Team -- http://jcifs.samba.org/ -)----- ubiqx development, uninq.
ubiqx Team -- http://www.ubiqx.org/ -)----- crh at ubiqx.mn.org
OnLineBook -- http://ubiqx.org/cifs/ -)----- crh at ubiqx.org