[chef] Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Chef and LDAP Users

[AJ Christensen < >]

The debug logs may be hard to understand to a non-native English
speaker. There's no internationalization in Chef.

--aj

On Fri, Aug 1, 2014 at 8:53 AM, Joseph Smith 
<
 >
 wrote:
> I realize that some things are said tongue-in-cheek and that people become
> frustrated when communication breaks down.
>
> However, I do feel we should still stick to some general standards of
> politeness and respect.
>
> Joseph
>
>
>
> On Thu, Jul 31, 2014 at 1:24 PM, AJ Christensen 
> <
 >
>  wrote:
>>
>> The recipe isn't dictating how it is done. You are. There is no
>> "problem with the recipe", if anything it sounds like a bug regarding
>> directory defaulting to root/root, or the interaction between lazy and
>> the outputs of the Library you're using. I don't know, I can't tell
>> without trying more :-)
>>
>> Put your keys in /etc/ssh/keys/%u as has been suggested (that zap
>> cookbook looks super cool) to avoid the problem altogether.
>> Alternatively store keys in LDAP. Both methods ensure a user cannot
>> inject any old key into authenticated keys between Chef converges.
>>
>> I don't know what is causing the directory to be created root/root, if
>> the ShellGetent lookups w/ lazy are working. You can run with debug
>> mode to see what values the directory is being created with, by Chef
>> itself.
>>
>> Post your debug logs if you can't read English.
>>
>> --AJ
>>
>> On Fri, Aug 1, 2014 at 8:13 AM, Douglas Garstang
>> <
 >
>>  wrote:
>> > I'd prefer not to allow a problem with the recipe dictate the way it's
>> > done.
>> > PAM currently sets the directory to be user.group, and I'd like to stick
>> > with that.
>> >
>> > Doug.
>> >
>> >
>> > On Thu, Jul 31, 2014 at 1:04 PM, Joe Nuspl 
>> > <
 >
>> >  wrote:
>> >>
>> >> Nope.  The directory and files can be root:root.  Just need to be world
>> >> readable.
>> >>
>> >> I like this because chef controls the contents and it can be audited.
>> >> Users can’t do things like put
>> >> Wile E. Coyote’s public key in their authorized keys file (for those
>> >> who
>> >> don’t have a data bag entry).
>> >>
>> >> <shameless-plug>
>> >> You will probably want to https://github.com/nvwls/zap that directory.
>> >> </shameless-plug>
>> >>
>> >>         Joe
>> >>
>> >> On Jul 31, 2014, at 12:54 PM, AJ Christensen 
>> >> <
 >
>> >>  wrote:
>> >>
>> >> > Does that directory still need to have uid/gid permissions,
>> >> > 0700/0600?
>> >> >
>> >> > On Fri, Aug 1, 2014 at 7:53 AM, AJ Christensen 
>> >> > <
 >
>> >> > wrote:
>> >> >> Boom! That's even simpler.
>> >> >>
>> >> >> --aj
>> >> >>
>> >> >> On Fri, Aug 1, 2014 at 7:52 AM, Joe Nuspl 
>> >> >> <
 >
>> >> >>  wrote:
>> >> >>> Or you could configure sshd to look for authorized keys some place
>> >> >>> other
>> >> >>> than the user's home directory.  For example:
>> >> >>>
>> >> >>> AuthorizedKeysFile     /etc/ssh/keys/%u
>> >> >>>
>> >> >>> And then have your recipe populate that directory…
>> >> >>>
>> >> >>> Joe
>> >> >>>
>> >> >>> On Jul 31, 2014, at 12:37 PM, Douglas Garstang
>> >> >>> <
 >
>> >> >>> wrote:
>> >> >>>
>> >> >>> AJ,
>> >> >>>
>> >> >>> There home directories are being created by LDAP and PAM on first
>> >> >>> login.
>> >> >>> However, that does not handle the setup of the ssh authorized keys
>> >> >>> file. For
>> >> >>> select users, I wanted chef to create the home directories, with
>> >> >>> the
>> >> >>> correct
>> >> >>> uid and gid pulled from LDAP, and then pull the ssh public key from
>> >> >>> a
>> >> >>> data
>> >> >>> bag and drop it into their home directory. Obviously, the LDAP
>> >> >>> recipe
>> >> >>> has to
>> >> >>> run first in the run list. This should be pretty simple.
>> >> >>>
>> >> >>> My latest attempt, based on your suggestion, is not working
>> >> >>> however. I
>> >> >>> believe the uid and gid are correct (although I can't print them
>> >> >>> because
>> >> >>> that causes it to fail. How do I print lazy variables? I tried
>> >> >>> using
>> >> >>> lazy in
>> >> >>> the log statement and it failed), but I can't verify because I
>> >> >>> can't
>> >> >>> print.
>> >> >>> When the recipe has run, /home/doug is owned by root.root.
>> >> >>>
>> >> >>> Doug
>> >> >>>
>> >> >>> # recipe
>> >> >>> include_recipe "foo-ldap"
>> >> >>> bag = data_bag("ssh-keys")
>> >> >>> for item in bag do
>> >> >>>    user = data_bag_item('ssh-keys', item)
>> >> >>>    user_name = user['id']
>> >> >>>    directory "/home/#{user_name}" do
>> >> >>>        owner lazy { ShellGetent.uid "#{user_name}" }
>> >> >>>        group lazy { ShellGetent.gid "#{user_name}" }
>> >> >>>    end
>> >> >>> end
>> >> >>>
>> >> >>> # library:
>> >> >>> class ShellGetent
>> >> >>>    include Chef::Mixin::ShellOut
>> >> >>>
>> >> >>>    class << self
>> >> >>>        def uid(username)
>> >> >>>            return if username.nil? || username.empty?
>> >> >>>            x = new.getent(:passwd, username)[2]
>> >> >>>        end
>> >> >>>
>> >> >>>        def gid(username)
>> >> >>>            return if username.nil? || username.empty?
>> >> >>>            new.getent(:passwd, username)[3]
>> >> >>>        end
>> >> >>>    end
>> >> >>>
>> >> >>>    # Returns a new slice of parts of the output. Maybe make a
>> >> >>> struct
>> >> >>> or
>> >> >>> something? Be creative!
>> >> >>>    def getent(type, name)
>> >> >>>        return unless %w(passwd).include?(type.to_s)
>> >> >>>        cmd = shell_out!("getent #{type.to_s} #{name}")
>> >> >>>        cmd.stdout.split(":")
>> >> >>>    end
>> >> >>>
>> >> >>> end
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> On Thu, Jul 31, 2014 at 11:23 AM, AJ Christensen 
>> >> >>> <
 >
>> >> >>> wrote:
>> >> >>>>
>> >> >>>> Another way to run the code during convergence time is to put the
>> >> >>>> code
>> >> >>>> in the body of an LWRP action, when the action fires (converge
>> >> >>>> time)
>> >> >>>> the code will run (not compile time!)
>> >> >>>>
>> >> >>>> A way to tidy this is to put re-used logic in Libraries and refer
>> >> >>>> to
>> >> >>>> those method/function calls from the LWRP itself.
>> >> >>>>
>> >> >>>> I believe that is most of the ways.
>> >> >>>>
>> >> >>>> Copying data in/out of resources at compile/convergence phase is
>> >> >>>> one
>> >> >>>> of the most advanced/nasty pieces of knowledge you will gain
>> >> >>>> regarding
>> >> >>>> recipe authorship, I expect; it should be avoided.
>> >> >>>>
>> >> >>>> I would consider re-designing your entire user management system
>> >> >>>> and
>> >> >>>> extract it from Chef; have like a little process that runs on
>> >> >>>> every
>> >> >>>> node, listening to LDAP; when new users are made, use skel to
>> >> >>>> construct them. I could have sworn I saw functionality in PAM,
>> >> >>>> specifically for creating users home dirs and so-on, when logged
>> >> >>>> in
>> >> >>>> from LDAP for the first time.
>> >> >>>>
>> >> >>>>
>> >> >>>>
>> >> >>>> http://manpages.ubuntu.com/manpages/maverick/man8/pam_mkhomedir.8.html
>> >> >>>>
>> >> >>>> Many ways to shave dat yak.
>> >> >>>>
>> >> >>>> Chef -> configure LDAP -> configure PAM -> Users log in -> Home
>> >> >>>> dirs
>> >> >>>> created automatically via PAM -> No compile/converge phase
>> >> >>>> shenanigans.
>> >> >>>>
>> >> >>>> Be careful configuring PAM, you can easily hose your system.
>> >> >>>>
>> >> >>>> --AJ
>> >> >>>>
>> >> >>>> On Fri, Aug 1, 2014 at 6:17 AM, AJ Christensen 
>> >> >>>> <
 >
>> >> >>>> wrote:
>> >> >>>>> My specific example shows the code being called at Convergence
>> >> >>>>> Time
>> >> >>>>> through a Lazy Attribute evaluation (a new Chef11 feature).
>> >> >>>>>
>> >> >>>>> Specifically: the attributes to the resource when lazy aren't
>> >> >>>>> evaluated at compile time, they are "finalized" so to speak at
>> >> >>>>> convergence time, when the action actually needs it!
>> >> >>>>>
>> >> >>>>> --aj
>> >> >>>>>
>> >> >>>>> On Fri, Aug 1, 2014 at 6:15 AM, Douglas Garstang
>> >> >>>>> <
 >
>> >> >>>>>  wrote:
>> >> >>>>>> AJ,
>> >> >>>>>>
>> >> >>>>>> Thanks. Now, what are the practical implications of compile and
>> >> >>>>>> converge in
>> >> >>>>>> relation to variable scope? How does one 'Run the ShellOut code
>> >> >>>>>> during
>> >> >>>>>> convergence'? Does it have to be in a library as you've done?
>> >> >>>>>> Can
>> >> >>>>>> it be
>> >> >>>>>> done
>> >> >>>>>> another way? What exactly in a recipe makes something run at
>> >> >>>>>> compile or
>> >> >>>>>> converge time? How do you tell? How do you pass variables
>> >> >>>>>> obtained
>> >> >>>>>> during
>> >> >>>>>> compilation (or is it convergence) back to the other?
>> >> >>>>>>
>> >> >>>>>> Doug
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>> On Thu, Jul 31, 2014 at 11:12 AM, AJ Christensen
>> >> >>>>>> <
 >
>> >> >>>>>> wrote:
>> >> >>>>>>>
>> >> >>>>>>> http://docs.getchef.com/essentials_nodes_chef_run.html
>> >> >>>>>>>
>> >> >>>>>>> * Compile the resource collection
>> >> >>>>>>>
>> >> >>>>>>> The chef-client identifies each resource in the node object and
>> >> >>>>>>> builds
>> >> >>>>>>> the resource collection. All libraries are loaded (to ensure
>> >> >>>>>>> that
>> >> >>>>>>> all
>> >> >>>>>>> language extensions and Ruby classes are available). And then
>> >> >>>>>>> all
>> >> >>>>>>> attributes are loaded. And then all lightweight resources are
>> >> >>>>>>> loaded.
>> >> >>>>>>> And then all definitions are loaded (to ensure that any
>> >> >>>>>>> pseudo-resources are available). Finally, all recipes are
>> >> >>>>>>> loaded
>> >> >>>>>>> in
>> >> >>>>>>> the order specified by the expanded run-list; each action
>> >> >>>>>>> specified in
>> >> >>>>>>> each recipe is identified before any Ruby code within a recipe
>> >> >>>>>>> is
>> >> >>>>>>> evaluated.
>> >> >>>>>>>
>> >> >>>>>>> * Converge the node
>> >> >>>>>>>
>> >> >>>>>>> The chef-client configures the system based on the information
>> >> >>>>>>> that
>> >> >>>>>>> has been collected. Each resource is executed in the order
>> >> >>>>>>> identified
>> >> >>>>>>> by the run-list, and then by the order in which each resource
>> >> >>>>>>> is
>> >> >>>>>>> listed in each recipe. Each resource in the resource collection
>> >> >>>>>>> is
>> >> >>>>>>> mapped to a provider. The provider examines the node, and then
>> >> >>>>>>> does
>> >> >>>>>>> the steps necessary to complete the action. And then the next
>> >> >>>>>>> resource
>> >> >>>>>>> is processed. Each action configures a specific part of the
>> >> >>>>>>> system.
>> >> >>>>>>> This process is also referred to as convergence.
>> >> >>>>>>>
>> >> >>>>>>> Run the ShellOut code during convergence after the LDAP code
>> >> >>>>>>> has
>> >> >>>>>>> converged (instead of at compile time as your example
>> >> >>>>>>> inaccurately
>> >> >>>>>>> attempts to represent mine)
>> >> >>>>>>>
>> >> >>>>>>> YMMV
>> >> >>>>>>>
>> >> >>>>>>> --aj
>> >> >>>>>>>
>> >> >>>>>>> On Fri, Aug 1, 2014 at 6:10 AM, AJ Christensen 
>> >> >>>>>>> <
 >
>> >> >>>>>>> wrote:
>> >> >>>>>>>> You ignored all of my examples and ran the code at Shell Out
>> >> >>>>>>>> code
>> >> >>>>>>>> at
>> >> >>>>>>>> compile time. It will be executed before the other LDAP stuff
>> >> >>>>>>>> is
>> >> >>>>>>>> actually converged.
>> >> >>>>>>>>
>> >> >>>>>>>> Don't be such a support vampire bro.
>> >> >>>>>>>>
>> >> >>>>>>>> --AJ
>> >> >>>>>>>>
>> >> >>>>>>>> On Fri, Aug 1, 2014 at 5:59 AM, Douglas Garstang
>> >> >>>>>>>> <
 >
>> >> >>>>>>>>  wrote:
>> >> >>>>>>>>> AJ,
>> >> >>>>>>>>>
>> >> >>>>>>>>> Thanks. I'm really not a ruby guy. It will take me
>> >> >>>>>>>>> considerable
>> >> >>>>>>>>> time to
>> >> >>>>>>>>> work
>> >> >>>>>>>>> out your gist. I tried this. I thought the call to
>> >> >>>>>>>>> Mixlib::ShellOut
>> >> >>>>>>>>> would
>> >> >>>>>>>>> fork a new process and therefore get me the correct result.
>> >> >>>>>>>>> However,
>> >> >>>>>>>>> cmd.stdout is empty on the first run, and contains the
>> >> >>>>>>>>> correct
>> >> >>>>>>>>> getent
>> >> >>>>>>>>> passwd
>> >> >>>>>>>>> reply on the second run, so I assume it's the same issue...
>> >> >>>>>>>>>
>> >> >>>>>>>>> include_recipe "foo-ldap"
>> >> >>>>>>>>>
>> >> >>>>>>>>> bag = data_bag("ssh-keys")
>> >> >>>>>>>>> for item in bag do
>> >> >>>>>>>>>
>> >> >>>>>>>>>    user = data_bag_item('ssh-keys', item)
>> >> >>>>>>>>>    user_name = user['id']
>> >> >>>>>>>>>
>> >> >>>>>>>>>    execute "create_homedir" do
>> >> >>>>>>>>>        command "mkdir /home/#{user_name}"
>> >> >>>>>>>>>        creates "/home/#{user_name}"
>> >> >>>>>>>>>        action :run
>> >> >>>>>>>>>    end
>> >> >>>>>>>>>
>> >> >>>>>>>>>    log "USER_NAME = #{user_name}"
>> >> >>>>>>>>>    cmd = Mixlib::ShellOut.new("getent passwd #{user_name}")
>> >> >>>>>>>>>    cmd.run_command
>> >> >>>>>>>>>    log ">>>>>>>>>>>>>>>>>>>>>>>>>>> NODE_DATA  =
>> >> >>>>>>>>> #{cmd.stdout}"
>> >> >>>>>>>>>    ent_array = cmd.stdout.split(':')
>> >> >>>>>>>>>    user_id = ent_array[2]
>> >> >>>>>>>>>    group_id = ent_array[3]
>> >> >>>>>>>>>
>> >> >>>>>>>>>    execute "homedir_perms" do
>> >> >>>>>>>>>        command "chown #{user_id}.#{group_id}
>> >> >>>>>>>>> /home/#{user_name}"
>> >> >>>>>>>>>        action :run
>> >> >>>>>>>>>    end
>> >> >>>>>>>>>
>> >> >>>>>>>>> end
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>> On Wed, Jul 30, 2014 at 5:25 PM, AJ Christensen
>> >> >>>>>>>>> <
 >
>> >> >>>>>>>>> wrote:
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> I meant to link to this too re: Mixlib ShellOut:
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> https://github.com/opscode/chef/blob/master/lib/chef/mixin/shell_out.rb#L29-L39
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> 'git grep shell_out' in the Chef codebase, haha! You'll see
>> >> >>>>>>>>>> all
>> >> >>>>>>>>>> of
>> >> >>>>>>>>>> the
>> >> >>>>>>>>>> places this is used.
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> --aj
>> >> >>>>>>>>>>
>> >> >>>>>>>>>> On Thu, Jul 31, 2014 at 12:22 PM, AJ Christensen
>> >> >>>>>>>>>> <
 >
>> >> >>>>>>>>>> wrote:
>> >> >>>>>>>>>>> If you 'include Chef::Mixin::ShellOut' in a custom library,
>> >> >>>>>>>>>>> you
>> >> >>>>>>>>>>> can
>> >> >>>>>>>>>>> use #shell_out! there.
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> using the lazy attribute evaluator and the Mixin ShellOut
>> >> >>>>>>>>>>> classes'
>> >> >>>>>>>>>>> shell_out!() functionality, something like this outta'
>> >> >>>>>>>>>>> work:
>> >> >>>>>>>>>>> https://gist.github.com/fujin/f0b7dd38b558001c5913
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> --aj
>> >> >>>>>>>>>>>
>> >> >>>>>>>>>>> On Thu, Jul 31, 2014 at 12:03 PM, Daniel DeLeo
>> >> >>>>>>>>>>> <
 >
>> >> >>>>>>>>>>> wrote:
>> >> >>>>>>>>>>>> On Wednesday, July 30, 2014 at 5:00 PM, Douglas Garstang
>> >> >>>>>>>>>>>> wrote:
>> >> >>>>>>>>>>>>> AJ,
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> Thanks. One more question... shelling out in chef and
>> >> >>>>>>>>>>>>> grabbing
>> >> >>>>>>>>>>>>> the
>> >> >>>>>>>>>>>>> output... has never been an easy task... best approach?
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> I was able to do this just now with the execute resource,
>> >> >>>>>>>>>>>>> I
>> >> >>>>>>>>>>>>> guess
>> >> >>>>>>>>>>>>> because that forks a new process too. It would be nice
>> >> >>>>>>>>>>>>> though
>> >> >>>>>>>>>>>>> to
>> >> >>>>>>>>>>>>> call getent
>> >> >>>>>>>>>>>>> in some fashion, and the parse the output, so that I
>> >> >>>>>>>>>>>>> could
>> >> >>>>>>>>>>>>> be
>> >> >>>>>>>>>>>>> sure
>> >> >>>>>>>>>>>>> that the
>> >> >>>>>>>>>>>>> group id I am assigning is the correct one for the user.
>> >> >>>>>>>>>>>>> At
>> >> >>>>>>>>>>>>> the
>> >> >>>>>>>>>>>>> moment it's
>> >> >>>>>>>>>>>>> coming out of a data bag.
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> Also kinda gross.
>> >> >>>>>>>>>>>>>
>> >> >>>>>>>>>>>>> Doug
>> >> >>>>>>>>>>>> Well, it’ll be a little gross, but I think you’ll have to
>> >> >>>>>>>>>>>> use
>> >> >>>>>>>>>>>> a
>> >> >>>>>>>>>>>> lazy
>> >> >>>>>>>>>>>> attribute
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> http://docs.getchef.com/resource_common.html#lazy-attribute-evaluation
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> In the code block you give to `lazy`, you can use
>> >> >>>>>>>>>>>> backticks
>> >> >>>>>>>>>>>> or
>> >> >>>>>>>>>>>> %x
>> >> >>>>>>>>>>>> to
>> >> >>>>>>>>>>>> run a command that returns the UID.
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>> --
>> >> >>>>>>>>>>>> Daniel DeLeo
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>> --
>> >> >>>>>>>>> Regards,
>> >> >>>>>>>>>
>> >> >>>>>>>>> Douglas Garstang
>> >> >>>>>>>>> http://www.linkedin.com/in/garstang
>> >> >>>>>>>>> Email: 
>> >> >>>>>>>>> 
 
>> >> >>>>>>>>> Cell: +1-805-340-5627
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>> --
>> >> >>>>>> Regards,
>> >> >>>>>>
>> >> >>>>>> Douglas Garstang
>> >> >>>>>> http://www.linkedin.com/in/garstang
>> >> >>>>>> Email: 
>> >> >>>>>> 
 
>> >> >>>>>> Cell: +1-805-340-5627
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> --
>> >> >>> Regards,
>> >> >>>
>> >> >>> Douglas Garstang
>> >> >>> http://www.linkedin.com/in/garstang
>> >> >>> Email: 
>> >> >>> 
 
>> >> >>> Cell: +1-805-340-5627
>> >> >>>
>> >> >>>
>> >>
>> >
>> >
>> >
>> > --
>> > Regards,
>> >
>> > Douglas Garstang
>> > http://www.linkedin.com/in/garstang
>> > Email: 
>> > 
 
>> > Cell: +1-805-340-5627
>
>