General discussion about Chef

[chef] Re: RE: RE: Remote file Copy from Windows Share


Chronological Thread 
  • From: Carlos Camacho < >
  • To: " " < >
  • Subject: [chef] Re: RE: RE: Remote file Copy from Windows Share
  • Date: Thu, 7 Aug 2014 16:28:51 +0200
I usually use xcopy to get data from shares...


To copy files from a folder
Chef::Log.warn("I need to copy: "+z[:app_name])      
Chef::Log.warn("From: "+default_app_web_sources_path)          
Chef::Log.warn("To: "+default_app_web_path)
COPY = "xcopy "+default_app_web_sources_path+z[:app_name]+".7z "+default_app_web_path+" /s /e"
Chef::Log.warn("COPY: "+COPY)

unless File.exists?(default_app_web_path+z[:app_name])
  batch 'populate_web_app' do
  code COPY
  action :run
end

To copy all folder content
Chef::Log.warn("I need to copy: "+z[:folder])      
Chef::Log.warn("From: "+default_dlls_sources_path)          
Chef::Log.warn("To: "+default_dlls_path)
COPY = "xcopy "+default_dlls_sources_path+z[:folder]+" "+default_dlls_path+z[:folder]+"\\ /s /e"
Chef::Log.warn("COPY: "+COPY)

unless File.exists?(default_dlls_path+z[:folder])
  batch 'populate_dlls' do
 code COPY
 action :run
end



For debugging i always print the command in the chef-client execution, and then execute it in the CLI.

Regards.


On Thu, Aug 7, 2014 at 4:22 PM, "> < " target="_blank"> > wrote:

Actually, forget all that, I think it know it is.

Seems to be an issue parsing the path and converting into windows.

If I use this path: \\ATLQ-CHEF01.airwatch.qa/server.installs/dll.drops/

I get the following error using the same path on the node in question:

I’ve tried a couple of permutations but I cannot get it to parse /server without hiding /s as it’s a special char. I’ve tried escaping with an additional forward or backward slash to no avail.

 

Any suggestions? I’ve seen there is a bug related to parsing with the chef-client on windows.

 

 

 

Hi Adam,

  I’ve tried that and it seems the formatting wasn’t liked by my system.

I’ve had to alter it to this so that the appearance in the logs looks correct; using single quotes just generated errors and featured lots of slashes where slashes should not be.

The resource now looks like this;

“fname = "7.1_HF05_Overlay_SAML_fix.zip"

Chef::Log.info("file:////#{managementServer}/server.installs/dll.drops/#{fname}")

 

remote_file "d:/installs/#{fname}" do

  atomic_update false

  source "file:////#{managementServer}/server.installs/dll.drops/#{fname}"

end”

 

Back to permission denied errors.

“Errno::ENOENT: No such file or directory - /server.installs/dll.drops/7.1_HF05_Overlay_SAML_fix.zip”

 

Chef Client Log is giving me;

“[2014-08-07T09:58:02-04:00] INFO: file://ATLQ-CHEF01.airwatch.qa/server.installs/dll.drops/7.1_HF05_Overlay_SAML_fix.zip

[2014-08-07T09:58:02-04:00] DEBUG: Loading Recipe windows via include_recipe

[2014-08-07T09:58:02-04:00] DEBUG: Found recipe default in cookbook windows

[2014-08-07T09:58:02-04:00] INFO: Processing chef_gem[win32-api] action install (windows::default line 23)

[2014-08-07T09:58:02-04:00] DEBUG: chef_gem[win32-api] detected omnibus installation in C:/opscode/chef/embedded/bin

[2014-08-07T09:58:02-04:00] DEBUG: chef_gem[win32-api] using gem from running ruby environment

[2014-08-07T09:58:02-04:00] DEBUG: chef_gem[win32-api] found installed gem win32-api version 1.4.8 matching win32-api (>= 0)

[2014-08-07T09:58:02-04:00] DEBUG: chef_gem[win32-api] is already installed - nothing to do

[2014-08-07T09:58:02-04:00] INFO: Processing chef_gem[win32-service] action install (windows::default line 23)

[2014-08-07T09:58:02-04:00] DEBUG: chef_gem[win32-service] detected omnibus installation in C:/opscode/chef/embedded/bin

[2014-08-07T09:58:02-04:00] DEBUG: chef_gem[win32-service] using gem from running ruby environment

[2014-08-07T09:58:02-04:00] DEBUG: chef_gem[win32-service] found installed gem win32-service version 0.8.2 matching win32-service (>= 0)

[2014-08-07T09:58:02-04:00] DEBUG: chef_gem[win32-service] is already installed - nothing to do

[2014-08-07T09:58:02-04:00] INFO: Processing chef_gem[windows-api] action install (windows::default line 31)

[2014-08-07T09:58:02-04:00] DEBUG: chef_gem[windows-api] detected omnibus installation in C:/opscode/chef/embedded/bin

[2014-08-07T09:58:02-04:00] DEBUG: chef_gem[windows-api] using gem from running ruby environment

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[windows-api] found installed gem windows-api version 0.4.2 matching windows-api (>= 0)

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[windows-api] is already installed - nothing to do

[2014-08-07T09:58:03-04:00] INFO: Processing chef_gem[windows-pr] action install (windows::default line 31)

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[windows-pr] detected omnibus installation in C:/opscode/chef/embedded/bin

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[windows-pr] using gem from running ruby environment

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[windows-pr] found installed gem windows-pr version 1.2.2 matching windows-pr (>= 0)

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[windows-pr] is already installed - nothing to do

[2014-08-07T09:58:03-04:00] INFO: Processing chef_gem[win32-dir] action install (windows::default line 31)

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[win32-dir] detected omnibus installation in C:/opscode/chef/embedded/bin

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[win32-dir] using gem from running ruby environment

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[win32-dir] found installed gem win32-dir version 0.4.5 matching win32-dir (>= 0)

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[win32-dir] is already installed - nothing to do

[2014-08-07T09:58:03-04:00] INFO: Processing chef_gem[win32-event] action install (windows::default line 31)

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[win32-event] detected omnibus installation in C:/opscode/chef/embedded/bin

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[win32-event] using gem from running ruby environment

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[win32-event] found installed gem win32-event version 0.6.1 matching win32-event (>= 0)

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[win32-event] is already installed - nothing to do

[2014-08-07T09:58:03-04:00] INFO: Processing chef_gem[win32-mutex] action install (windows::default line 31)

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[win32-mutex] detected omnibus installation in C:/opscode/chef/embedded/bin

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[win32-mutex] using gem from running ruby environment

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[win32-mutex] found installed gem win32-mutex version 0.4.1 matching win32-mutex (>= 0)

[2014-08-07T09:58:03-04:00] DEBUG: chef_gem[win32-mutex] is already installed - nothing to do

[2014-08-07T09:58:03-04:00] DEBUG: Converging node ATLQ-CHEF04.airwatch.qa

[2014-08-07T09:58:03-04:00] INFO: Processing remote_directory[c:/chef/handlers] action nothing (chef_handler::default line 23)

[2014-08-07T09:58:03-04:00] DEBUG: Skipping remote_directory[c:/chef/handlers] due to action :nothing

[2014-08-07T09:58:03-04:00] INFO: Processing cookbook_file[c:/chef/handlers/AW-ReportHandler.rb] action create (chef_handler::default line 40)

[2014-08-07T09:58:03-04:00] DEBUG: cookbook_file[c:/chef/handlers/AW-ReportHandler.rb] checksumming file at c:/chef/handlers/AW-ReportHandler.rb.

[2014-08-07T09:58:03-04:00] INFO: Processing chef_handler[AWHandle::SendEmail] action enable (chef_handler::default line 45)

[2014-08-07T09:58:03-04:00] DEBUG: AWHandle::SendEmail has not been loaded.

[2014-08-07T09:58:03-04:00] INFO: Enabling chef_handler[AWHandle::SendEmail] as a report handler

[2014-08-07T09:58:03-04:00] INFO: Enabling chef_handler[AWHandle::SendEmail] as a exception handler

[2014-08-07T09:58:03-04:00] INFO: Processing remote_file[d:/installs/7.1_HF05_Overlay_SAML_fix.zip] action create (AW_DLLDrop::default line 30)

[2014-08-07T09:58:03-04:00] DEBUG: touching d:/installs/7.1_HF05_Overlay_SAML_fix.zip to create it

[2014-08-07T09:58:03-04:00] INFO: remote_file[d:/installs/7.1_HF05_Overlay_SAML_fix.zip] created file d:/installs/7.1_HF05_Overlay_SAML_fix.zip

[2014-08-07T09:58:03-04:00] DEBUG: remote_file[d:/installs/7.1_HF05_Overlay_SAML_fix.zip] checking for changes

[2014-08-07T09:58:03-04:00] DEBUG: remote_file[d:/installs/7.1_HF05_Overlay_SAML_fix.zip] staging //ATLQ-CHEF01.airwatch.qa/server.installs/dll.drops/7.1_HF05_Overlay_SAML_fix.zip to C:/Users/CHALL~1.AIR/AppData/Local/Temp/7.1_HF05_Overlay_SAML_fix.zip20140807-3340-t9q63r

[2014-08-07T09:58:03-04:00] WARN: remote_file[d:/installs/7.1_HF05_Overlay_SAML_fix.zip] cannot be downloaded from file:////ATLQ-CHEF01.airwatch.qa/server.installs/dll.drops/7.1_HF05_Overlay_SAML_fix.zip: Permission denied - //ATLQ-CHEF01.airwatch.qa/server.installs/dll.drops/7.1_HF05_Overlay_SAML_fix.zip

[2014-08-07T09:58:04-04:00] INFO: Running queued delayed notifications before re-raising exception

[2014-08-07T09:58:04-04:00] DEBUG: Re-raising exception: Errno::EACCES - remote_file[d:/installs/7.1_HF05_Overlay_SAML_fix.zip] (AW_DLLDrop::default line 30) had an error: Errno::EACCES: Permission denied - //ATLQ-CHEF01.airwatch.qa/server.installs/dll.drops/7.1_HF05_Overlay_SAML_fix.zip”

 

Any further thoughts?

 

Thanks

Chris

 

From: Adam Edwards [ " target="_blank">mailto: ]
Sent: 06 August 2014 19:04
To: " target="_blank">
Subject: [chef] RE: RE: RE: RE: RE: Remote file Copy from Windows Share

 

So here’s an example of what I did to make remote_file work – I also added the ‘atomic_update’ attribute because at one point this was required (not sure that’s needed in latest versions of chef-client):

 

remote_file "c:/users/myuser/myfile.txt" do

  atomic_update false

  source 'file:////server/share/file.txt'

end

 

Basically you need two additional ‘/’ chars in your url. We should probably add a path_syntax attribute to automate this…

 

-Adam

 

 

Hi Adam,

  Thanks for that, it seems to have helped. Authenticating via AD seems to be working ok.

I’m now getting an error saying the dir isn’t there (which is clearly is – I have checked the paths).

I think it’s looking for the path on the local system rather that the remote one.  I suspect this is me messing up some syntax J

Here is the section out of the recipe:

“

# Dll drop filename (in full, no path)

# ie   fname = "7.1_HF05_Overlay_SAML_fix.zip"

fname = "7.1_HF05_Overlay_SAML_fix.zip"

path = "file://#{managementServer}/server.installs/dll.drops/#{fname}"

 

remote_file "d:\\installs\\#{fname}" do

  source path

end

”

 

Here is the error:

“

remote_file("d:\installs\7.1_HF05_Overlay_SAML_fix.zip") do

Win-Node2   provider Chef::Provider::RemoteFile

Win-Node2   action "create"

Win-Node2   retries 0

Win-Node2   retry_delay 2

Win-Node2   guard_interpreter :default

Win-Node2   path "d:\\installs\\7.1_HF05_Overlay_SAML_fix.zip"

Win-Node2   backup 5

Win-Node2   atomic_update true

Win-Node2   source ["file://ATLQ-CHEF01.airwatch.qa/server.installs/dll.drops/7.1_HF05_Overlay_SAML_fix.zip"]

Win-Node2   use_etag true

Win-Node2   use_last_modified true

Win-Node2   cookbook_name "AW_DLLDrop"

Win-Node2   recipe_name "default"

Win-Node2 end

 

Win-Node2 [2014-08-06T13:41:36-04:00] FATAL: Stacktrace dumped to c:/chef/cache/chef-stacktrace.out

DEBUG: :relay_output_from_backend => ["Win-Node2", "[2014-08-06T13:41:36-04:00] FATAL: Errno::ENOENT: remote_file[d:\\installs\\7.1_HF05_Overlay_SAML_fix.zip] (AW_DLLDrop::default line 28) had an error: Errno::ENOENT: No such file or directory - /server.installs/dll.drops/7.1_HF05_Overlay_SAML_fix.zip\r\n"]

”

 

From: Adam Edwards [ " target="_blank">mailto: ]
Sent: 06 August 2014 17:43
To: " target="_blank">
Subject: [chef] RE: RE: RE: Remote file Copy from Windows Share

 

Hmm – so you are running Chef remotely via WinRM? If that’s the case, you should add credssp to your winrm listener. You can do that with powershell by setting the value below to true:

 

ls WSMan:\localhost\Service\Auth\CredSSP

# if that shows false, you can set it to true:

si WSMan:\localhost\Service\Auth\CredSSP $true

 

That should allow your credentials to do one extra hop off the machine.

 

 

Hi Adam,

  It’s windows to windows.  Only the Chef server is Linux but that has little to do with anything (I think/hope).

Without Chef involvement the shares are perfectly accessible, even using the user account that chef client is run as.

 

Firewall is off completely and I’m using AD creds and local creds.  When Chef run is performed against the two windows nodes, one will complete successfully (it is the node hosting the share) and the other node will fail with ‘authentication failed – password incorrect’ (which I think is a catch-all error).

The security log of the second node shows authentication error with a null sid.

  The first node (hosting the share) shows no errors.

  The security of the D:\installs folder also show effective permissions as full control for both ad login and local account.

 

I’m wondering if it’s something to do with the user that the chef client is run as on each node? Maybe machine localsystem?  Does chef-client run with the creds that winrm provides to it or do those creds simply auth against WinRM and then localsystem takes over?  Chef-Client is being run on-demand, it is not installed as a service due to the way we will be using Chef.

 

Thanks

Chris

 

 

Christopher, to clarify a few things here:

 

1.       Are you copying from a windows machine to a windows machine? Or is one of the systems a Linux system

2.       You’re saying that even without any Chef / Ruby involved you can’t access the share?

 

If those things are true, I’d try the following:

1.       Turn off the firewall

2.       Try using credentials for a local account as well as a domain credentials

3.       Check the security log to see if you get an audit event for a failed logon

 

-Adam

 

 

Hi Chefs

I’m having trouble again J

 

I’m trying to copy a file from a share on a windows server from a target node via chef. The purpose for it to copy dll drop files from a share and dumped into d:\installs so it can later be unzipped. At the moment I am having issues trying to connect to the share.

I’ve tried running the chef run as a local admin on the target node (which used to work), using AD creds and also a different local account to no avail.

The linux server was recently added to our domain, but this was failing before that happened.

I’ve tried a batch file mapping a drive then copy, a powershell script doing the same, native ruby and now this remote file resource. Same error everytime, it looks like a permissions error on Windows, but the file share is currently set full permission to everyone on both Share permissions and filesystem permissions.

I’m seeing NULL_SID event log entries where the auth fails, but this is on the target node, not the one that hosts the share.

 

Can any offer anything to look at?

 

Chef server is Ubuntu 14.04 using PBIS to auth via AD  (might I need Samba installing now AD is in use for Auth?)

Windows nodes are running Server 2008R2 patched up-to-date.  All are VM’s. All one the same AD and DNS domain.

 

“myserver

DEBUG: :relay_output_from_backend => ["Win-Node2", "================================================================================\r\nError executing action `create` on resource 'remote_file[d:\\installs\\7.1_HF05_Overlay_SAML_fix.zip]'\r\n================================================================================\r\n\r\nErrno::EACCES\r\n-------------\r\nPermission denied - //targetserver//server.installs//dll.drops//7.1_HF05_Overlay_SAML_fix.zip\r\n\r\nResource Declaration:\r\n---------------------\r\n# In c:/chef/cache/cookbooks/AW_DLLDrop/recipes/default.rb\r\n\r\n 27: remote_file \"d:\\\\installs\\\\\#{fname}\" do \r\n 28:   source \"file:////\#{managementServer}//server.installs//dll.drops//\#{fname}\"\r\n 29: end\r\n 30: \r\n\r\nCompiled Resource:\r\n------------------\r\n# Declared in c:/chef/cache/cookbooks/AW_DLLDrop/recipes/default.rb:27:in `from_file'\r\n\r\nremote_file(\"d:\\installs\\7.1_HF05_Overlay_SAML_fix.zip\") do\r\n  provider Chef::Provider::RemoteFile\r\n  action \"create\"\r\n  retries 0\r\n  retry_delay 2\r\n  guard_interpreter :default\r\n  path \"d:\\\\installs\\\\7.1_HF05_Overlay_SAML_fix.zip\"\r\n  backup 5\r\n  atomic_update true\r\n  source [\"file:////targetserver//server.installs//dll.drops//7.1_HF05_Overlay_SAML_fix.zip\"]\r\n  use_etag true\r\n  use_last_modified true\r\n  cookbook_name \"AW_DLLDrop\"\r\n  recipe_name \"default\"\r\nend\r\n\r\n"]”

 

 

Thanks

Chris



Archive powered by MHonArc 2.6.16.

§