[chef] Re: Re: Re: Re: Re: Re: Re: Re: Re: Re: Bring up the discussion for knife-ec2 --node-name-prefix option

[Sascha Bates < >]

How are people working with these servers that have no useful labels?  I really want to know.  We ssh all over the place in our infrastructure and if I had to go out and look up a special snowflake ec2-blah-blah or IP every time I wanted to get to a server, I'd probably take a baseball bat to my infrastructure.

For example, we run a Jenkins setup with 20 slaves.  The slaves are rebuilt often using openstack and chef.  I'm currently writing a knife plugin to set up ssh aliases for nodes in a role that make it easy for us to do things like 'ssh slave20' as we are constantly helping people figure out stuff about their jobs on the slaves or diagnosing VM issues (I'd actually really like to take a baseball bat to our Openstack infra).

So I'm following this discussion and wondering how people are actually visiting their servers and how much time they waste looking up IPs for ssh?  We have a much smaller AWS installation and I do actually ec2-describe-instances|grep sometag and then ec2-describe-instances <instanceid> to get the IP address every time I want to go out there, which isn't often or I'd figure out a way to make it less painful.

How do people cope with connecting to nodes when they all have nonsense names that potentially change often?

Sascha Bates | "> | 612 850 0444 | | |

On 10/21/12 6:29 PM, Leo Dirac (SR) wrote:

" type="cite">Here's how I'd summarize the two sides of the issue:

Using manually-assigned names for nodes is not a best practice.  It does not scale up well, and encourages other problems like inefficient use of hardware.  It's better to use the searchable database to identify nodes.  Not allowing names encourages users to adopt best practices more quickly.

On the flip side, having simple names for nodes is convenient and simplifying when used appropriately.  It's probably fine for many small installations.  It's also particularly helpful for people who are still learning to use chef.  In either of these cases, a search query is likely to be more complex and harder to remember than a manually assigned name.

To me the question boils down to how we want to shape chef's learning curve.  Not allowing names makes it steeper at the beginning -- harder to learn, but by forcing new users to understand and adopt practices that scale well, they'll have fewer problems as they grow.

IMHO technologies are almost always more successful when they make themselves easier to use by a broader set of people.  I see it as better to have more users relying on and contributing to the technology even if some of them aren't using it perfectly than to turn people away who are considering it.  So I'd side with allowing names, but with the appropriate warnings about how they might be leading to bad habits.

 On Sun, Oct 21, 2012 at 3:05 PM, Jamie Winsor < " target="_blank"> > wrote:

You could think of AJ's approach of identifying systems as something similar to Ruby's approach of a type system. We don't have one - instead we ask an object how it behaves with "respond_to?".

AJ is asking you to think differently by dropping the need to name a node as something you'll keep in your head or a spreadsheet in favor of using node data to identify your machines.

You can ask a node if its your database master for production for a given application based on a set of attributes which make up its behavior (or role). You could even just simplify this by asking about a node's run_list.

Much like we do not need a static type system in Ruby - You simply do not need to name your nodes if you look at it this way.

@resetexistence

On Oct 21, 2012, at 12:06 PM, James Light < "> > wrote:

> Cool. Mostly inline responses as well.
>
> On Sun, Oct 21, 2012 at 2:37 PM, AJ Christensen < "> > wrote:
>> Reponses inline:
>>
>> On Oct 21, 2012 11:27 AM, "James Light" < "> > wrote:
>>>
>>> Okay. I may misunderstand the topic here. It sounds like you're saying
>>> that the i-xXxXxxwhatever-whatever-whocan-remember-any-of-this AWS EC2
>>> instance ID string should be good enough for everyone to use as a node
>>> name? Either that or they should settle for naming their nodes
>>> explicitly at creation time.
>>
>> That is exactly my point.
>
> No problem with that.
>
>
>>>
>>> And I thought what the pull request in the link was asking for was a
>>> way to have a prefix set so that one could launch a whole formation of
>>> nodes and have the node names all be able to be easily matched later
>>> by a trivial regular _expression_.
>>
>> That is fair, and may be useful for bulk regex operations.
>
> Well, yea, only thing is if I'm doing bulk operations on nodes
> regularly, I'm probably going to use knife exec scripts and then, once
> again, I don't need the info in the node name specifically and
> exclusively.
>
> [----snip----]
>>> but I'm a human being and sometime I'd also like to remember what the
>>> heck I just called one of my programmable resources so I can easily
>>> refer to it later.
>>>
>>> What am I missing on both sides of this please?
>>
>> You are saying you are a human and want to remember machine designations,
>> probably storing these in your head and a document (isn't that pretty?). At
>> what point does this stop being scalable? I consider that myopic.
>
> Well, whether or not it is scalable or not doesn't mean that its
> myopic. I'm open to both points... which is actually the precise
> opposite of myopic. ;)
>
>>
>> I am am engineer. I don't remember node names. When I need to locate one, I
>> have a full text index database of the node data available to me. I can
>> perform batch operations on a much more accurate scale thanks to the power
>> of Node Data.
>
> And you still need to name that node data in a way that is memorable
> to be able to do anything dynamic and spontaneous with it. You're just
> choosing to name other things memorable names instead of the entire
> node itself. This shows the approach of nodes are just resources that
> are used to accomplish something else and that seems to be very much
> in line with advanced chef usage. Kudos!
>
>>
>>>
>>> *) how is this ec2 specific?
>>
>> Definitely is not; this came up before we even had knife ec2
>>
>>> *) are there any situations where it is useful to have memorable and
>>> even groupable node namings or can we do all of that through other
>>> attributes and environments and roles?
>>
>> Potentially in infrastructures which do not have suitable systems for
>> machine identification / resolution.
>
> And *that* seems to be the real crux of the issue here.
>
> There need to be memorable names somewhere, be that in the entire
> domain of node data or some range of node data that may even be a
> subset of the original domain.
>
> To say that we are using the full power of node data seems to me to
> necessitate that we posses the ability and willingness to use all of
> the node data in flexible ways.
>
> The only difference seems to be that you remember the context and
> semantics of your node data and someone else wants to further abstract
> that so that this context and semantics is implicitly referred to by a
> specific node attribute, that of the node name.
>
> I really fail to see why it is "bad" to treat a node name like a
> snowflake when it is useful and why it is "good" to treat a node name
> like a sacred and immutable thing that is only must be exactly
> something that is so special that it isn't special at all.
> Just seems silly.
>
> It scales because I make it scale. That's my job. What I name things
> in the process of making things scale is flexible. When I am starting
> things out I have some names that make sense to me. When I dug a bit
> deeper it started to make more sense how to use node data in a logical
> way so that the names themselves don't need to be snowflakey anymore,
> BUT for someone else's needs they may still need the memorable names.
> I think that really scalability also applies to the concept of being
> able to do things quickly that have not been thought of before. A
> scalability of work flow so that when something unexpected comes up
> tomorrow (which it always does) that I can get it done quickly without
> being forced to do something in a way that is sideways towards my end
> goal.
>
> I agree that having snowflakey node names in your head to keep track
> of everything stops working at some point. I disagree that it never
> works at all and I think it is important for each Chef to be able to
> Choose Their Workflow.
>
> All this talk about snowflakes makes me want to go snowboarding.
> Opscode to Hood Mountain conference / outing someday??
>
> Thx for taking the time to express your opinions with only a subtle
> tinge of jade. ;)
>
>
>>
>> Cheers,
>>
>> AJ
>>
>>> *) any other details that I don't even know that I don't know yet?
>>>
>>> Thanks!
>>>
>>>
>>> On Sun, Oct 21, 2012 at 1:49 PM, AJ Christensen < "> >
>>> wrote:
>>>> The machines already come with extremely useful names (ec2). Node name
>>>> currently can be explicitly set or reflects the machine host name,
>>>> instance
>>>> ID. No?
>>>>
>>>> I believe the instance ID system currently in use may have been
>>>> developed by
>>>> some sort of computer scientist. Hard to say though.
>>>>
>>>> The negativity, and I'll thank you for asking, stems from the repeated
>>>> requests over time for pretty snowflake node names of varying forms. The
>>>> answer remains the same, so obviously one might become a little jaded.
>>>>
>>>> This doesn't need to be functionality baked into 'knife-ec2'. Not only
>>>> that,
>>>> but naming / host name management, machine DNS (etc) are solved problems
>>>> during convergence time.
>>>>
>>>> I hope this clarifies my original statement and I'd like to apologize
>>>> for
>>>> the negativity, it doesn't really help.
>>>>
>>>> I got 99 problems bit naming isn't one.
>>>>
>>>> Cheers,
>>>>
>>>> AJ
>>>>
>>>> On Oct 21, 2012 10:28 AM, "James Light" < "> >
>>>> wrote:
>>>>>
>>>>> Labels are for people who like naming things... like the whole field
>>>>> of computer scientists. All we do when it comes down to it is name
>>>>> things and refer to them by names that make sense to us to do things
>>>>> that are usually useful for some reason or another.
>>>>>
>>>>> Why the negativity towards someone else's perception of how to name
>>>>> things usefully?
>>>>>
>>>>> On Sun, Oct 21, 2012 at 12:53 PM, John Dewey < "> > wrote:
>>>>>> +1
>>>>>>
>>>>>> On Sunday, October 21, 2012 at 9:51 AM, AJ Christensen wrote:
>>>>>>
>>>>>> Labels are for jam jars, not machinery
>>>>>>
>>>>>> --AJ
>>>>>>
>>>>>> On Oct 21, 2012 11:21 PM, "Sachin Sagar Rai" < "> >
>>>>>> wrote:
>>>>>>
>>>>>> Ohai Chefs!
>>>>>>
>>>>>> Last week, there was a good discussion on whether to bring the
>>>>>> --elastic-ip
>>>>>> attachment into the knife-ec2 and I hope that it will settle down and
>>>>>> get
>>>>>> release on the upcoming version.
>>>>>>
>>>>>> But at this very same time, I'ld like to know and bring up the
>>>>>> discussion on
>>>>>> this pull req as well: https://github.com/opscode/knife-ec2/pull/61
>>>>>>
>>>>>> I'm really positive about this option as well.
>>>>>>
>>>>>> Want to hear about the community on this!
>>>>>>
>>>>>> -------------------------------------------
>>>>>> @millisami
>>>>>> ~ Sachin Sagar Rai
>>>>>> Ruby on Rails Developer
>>>>>> http://tfm.com.np
>>>>>> http://nepalonrails.tumblr.com
>>>>>> Sent with Sparrow
>>>>>>
>>>>>>