[chef-dev] Re: Re: Re: Re: Error downloading cookbooks in Chef 11

[Ignasi < >]

Hi Seth,

That is exactly the case. The failures occur when requesting those files.

I've been investigating a bit more and I've found a pattern for the
failure. I've seen that the signature validation only fails when the
URL of the resource has URL encoded characters in the "Signature"
query parameter. For example, the following two URLs are failing:

In Enterprise Chef:
https://s3-external-1.amazonaws.com:443/opscode-platform-production-data/organization-<secret>/checksum-f111ba97dc5160e26eee6473dc8e86d5?AWSAccessKeyId=<secret>&Expires=1377590794&Signature=QPnT5XLRam%2Br8p09hXra5N3jd6A%3D

In Chef 11:
https://ubuntu64-1104.defaultdomain:443/bookshelf/organization-00000000000000000000000000000000/checksum-818a38d41bd95ebec58b6d430eabd7cb?AWSAccessKeyId=<secret>&Expires=1377588824&Signature=31OUx%2BUBdZSyzzA9xpHlFIOxjHc%3D

Both have the '%2B' character in the signature. I've also seen URLs
with a '%20' that are also failing. However, if the signature does not
have those URL encoded characters, it just works fine (the padding
'%3D' character is not causing any failure). Here is an example URL
that is working:

https://ubuntu64-1104.defaultdomain:443/bookshelf/organization-00000000000000000000000000000000/checksum-a7daae249eca97adcd4b3617c745940d?AWSAccessKeyId=<secret>&Expires=1377589454&Signature=Mp7TFuAyObhtsjgE7wLSv3Sm6WU%3D


Is there something special that should be done to treat those encoded
characters when signing? I've been taking a look at the knife and mix
lib-authentication source code (a knife cookbook download works fine)
but still haven't found anything relevant.


Many thanks for your help,


Ignasi



On 27 August 2013 00:18, Seth Falcon 
<
 >
 wrote:
>
> Hi Ignasi,
>
> 
 
>  writes:
>> I am the maintainer of jclouds-chef [1], a Java library to access
>> Chef, and just started testing it against Enterprise Chef and Chef 11.
>> It used to work with Chef 10 and Hosted Chef, and I am just trying to
>> figure out if there is something that has changed in the signature
>> verification that is making it fail now with Chef 11 and Enterprise
>> Chef.
>>
>> The signature generation is generic [2] and the same code is used to
>> sign every request, so I don't understand why I only get failures when
>> trying to download the contents of a file (and why it only fails in
>> Chef 11 and Hosted Chef).
>
> There are a number of significant changes introduced in Chef 11 and more
> recently deployed into Hosted Enterprise Chef (HEC).
>
> Here's my understanding of the problem you're seeing, please let me know
> if I'm off:
>
> 1. You can use jclouds-chef to make some successful API requests against
> Open Source Chef 11 Server (OSC) and against your org in Hosted
> Enterprise Chef.
>
> 2. You are seeing intermittent 403 errors when doing GETs of cookbook
> content.
>
>> I can provide the logs of the generated requests and the obtained
>> responses, if it helps.
>
> It would help to have examples of the type of URL you are attempting to
> access (you can put in placeholders to maintain privacy if you will be
> posting this to the list).
>
> Similarly, associated responses would be helpful.
>
> Cookbook content, by which I mean the files in a cookbook stored by
> checksum, are handled outside of the core server in both OSC 11 and HEC.
>
> OSC 11 uses a newly added component called "bookshelf" to serve the
> cookbook content. It provides a think S3 API.
>
> HEC stores cookbook content in AWS S3.
>
> In both cases, when fetching a cookbook, the client receives a cookbook
> version object containing pre-signed URLs that it can use to fetch the
> cookbook content out of either bookshelf or S3. Although it is OK to
> sign those requests like other API requests, both bookshelf and S3 use
> S3-style authentication. If these are the requests that are failing,
> inspecting the response body might give us a clue of where to look.
>
> + seth
>
>
>
> --
> Seth Falcon | Development Lead | Opscode | @sfalcon
>