[chef-dev] Re: [chef] kitchen-ec2 0.9.0 released

[Tyler Ball < >]

Aaaaaand another one - I introduced a regression where (no matter what interface you specify) we try to connect to the private IP of the created instance.  Fixed by PR #151 and released as 0.9.5.

-T

On Jun 3, 2015, at 3:26 PM, Tyler Ball < " class=""> > wrote:

And another bug fix - a NoMethodError on the InstanceGenerator class.  PR #146 released as 0.9.4.

-T

On May 29, 2015, at 11:01 AM, Tyler Ball < " class=""> > wrote:

Another minor fix, just 1 bug this time - we need to check for existence before tagging an instance.  PR #140 merged to master and released as 0.9.3.

-T

On May 27, 2015, at 8:01 AM, Tyler Ball < " class=""> > wrote:

Another bug fix release!  I just released 0.9.2 - here are the CHANGELOG comments:

• Pull Request #131: Adding back support for a proxy via config http_proxy which defaults to ENV[‘HTTPS_PROXY’] || ENV['HTTP_PROXY']

• Pull Request #128:

• Fixes #121: Fixing error Network interfaces and an instance-level security groups may not be specified on the same request

• Fixes #127: User Data content should be base64 encoded when passed to aws sdk

-T

On May 21, 2015, at 1:11 PM, Luis Cosmes < " class=""> > wrote:

Thanks, Tyler.  I just tested it and I am able to assign a IAM profile via kitchen-ec2!

Luis

Luis Cosmes

DevOps Engineer

FinancialForce.com, Inc.

M: (510) 517 3400

E: " style="text-decoration:none" target="_blank" class="">

On Thu, May 21, 2015 at 11:50 AM, Tyler Ball < " target="_blank" class=""> > wrote:

I just released 0.9.1 which includes the two PRs referenced below.  The CHANGELOG has been copied below:

• Pull Request #124: AWS SDK V2 returns instance.public_dns_name as empty string instead of nil, and we were only checking for nil. Caused timeouts trying to connect. (@tyler-ball)

• Fixed regression: Adding back `interface` config value that I accidently removed, code is now in line with README.

• Pull Request #125: When specifying associate_public_ip we must send the subnet (if provided) in the network_interfaces section of the payload instead of the main section. (@tyler-ball)

• Fixed regression: Accidently renamed config `associate_public_ip` to `associate_public_ip_address`, reverting.

• Fixed regression: Accidently renamed config `iam_profile_name` to `iam_instance_profile`, reverting.

-T

On May 21, 2015, at 9:24 AM, Tyler Ball < " target="_blank" class=""> > wrote:

I’ve opened up 2 PRs (#125 and #124)  to address #1, #2 - there will be a patch fix incoming as soon as those are fixed.  I’ll look into the other issues you referenced.

You can include driver-specific information under the provisioning hash.  For example,

driver:

  subnet_id: subnet-12345

  ...

platforms:

  - ubuntu 14.04

    driver:

      security_group_ids: [“sg-abc”]

  - centos-6.4

    driver:

      security_group_ids: [“sg-def”]

I don’t think there is a regular chef-client provisioner included in base Test Kitchen, but I think someone may have written one at some point...

-T

On May 20, 2015, at 3:45 PM, Yoshi Spendiff < " target="_blank" class=""> > wrote:

Hi Tyler,

I'm having a few issues.

1. IAM profiles don't assign, there's another email thread about that

2. associate_public_ip_address doesn't work for a VPC instance, there's a github issue for that

3. I get asked for a password as soon as the instance is 'ready,' even if there's no ssh connection available (there's no public IP address and I have interface: public in the .yml file). I'm not sure why it's even asking for a password when I have ssh_key set under the transport section of the .yml file.

4. availability_zone is asking for a full path instead of a letter, but this is not needed anyway as the subnet should define that. There's another github issue for that

I just started trying this out as the inability to quickly run vagrant provision when using vagrant-ec2 is driving me nuts, so I have a couple of questions about usage too:

1. Is it possible to put some settings under the suites section of the .yml file instead of under the driver? Specifically I'm asking about security groups and an iam role but I think subnet-id could be useful too. I'm trying to use this to test a role cookbook with a couple of role recipes and in actuality these roles would have different security groups and iam roles (I know the best practice is one cookbook per role but that's another story).

2. Is there a chef-client provisioner or only chef-solo or chef-zero?

On Wed, May 20, 2015 at 8:22 AM, Luis Cosmes < " target="_blank" class=""> > wrote:

Thank you guys, I had two versions of test-kitchen installed and the latest one was 1.4.0, but looks like there was some conflict going on.  I just re-installed chefdk and things are working correctly now with kitchen-ec2 0.9.0!

Luis

Luis Cosmes

DevOps Engineer

FinancialForce.com, Inc.

M: (510) 517 3400

E: " style="text-decoration:none" target="_blank" class="">

On Wed, May 20, 2015 at 8:15 AM, Tyler Ball < " target="_blank" class=""> > wrote:

Yes, that `kitchen_driver_api_version` only works with Test Kitchen 1.4+.  @Luis what version of Test Kitchen do you have installed?  If you are using the ChefDK 0.5.0+ then it has the correct Test Kitchen version available.

-T

On May 19, 2015, at 9:02 PM, Galen Emery < " target="_blank" class=""> > wrote:

Luis,

Kitchen 1.4.0 has been out for a while, as part of chefdk 0.5.0, there were a significant amount of dependency bumps in that.  I'd try chefdk 0.5.0+ to see if that resolves your issue.

-Mobile Galen

---

From: " target="_blank" class="">Luis Cosmes
Sent: ‎5/‎19/‎2015 16:24
To: " target="_blank" class="">Tyler Ball
Cc: " target="_blank" class=""> ; " target="_blank" class="">
Subject: [chef-dev] Re: kitchen-ec2 0.9.0 released

Forgot to include the nested exception:

=========================================================

E, [2015-05-19T15:54:54.820521 #58612] ERROR -- Kitchen: ---Nested Exception---

E, [2015-05-19T15:54:54.820541 #58612] ERROR -- Kitchen: Class: NoMethodError

E, [2015-05-19T15:54:54.820551 #58612] ERROR -- Kitchen: Message: undefined method `kitchen_driver_api_version' for Kitchen::Driver::Ec2:Class

On Tue, May 19, 2015 at 4:01 PM, Luis Cosmes < " target="_blank" class=""> > wrote:

I just upgraded the kitchen-ec2 from 0.8.0 to 0.9.0:

sudo chef gem install kitchen-ec2

Using a previous .kitchen.yml file that worked with version 0.8.0 I started getting a "Could not load the 'ec2' driver from the load path" error.  As another test I decided to use the .kitchen.yml file described in the README file, and got the same results.   Maybe I am missing a new dependency?  (if I uninstall version 0.9.0 and re-install 0.8.0 the errors go away)

luis-cosmes-mbp:ffdc-common lcosmes$ kitchen list

>>>>>> ------Exception-------

>>>>>> Class: Kitchen::ClientError

>>>>>> Message: Could not load the 'ec2' driver from the load path. Please ensure that your driver is installed as a gem or included in your Gemfile if using Bundler.

>>>>>> ----------------------

>>>>>> Please see .kitchen/logs/kitchen.log for more details

>>>>>> Also try running `kitchen diagnose --all` for configuration

luis-cosmes-mbp:ffdc-common lcosmes$ kitchen diagnose --all

---

timestamp: 2015-05-19 22:57:00 UTC

kitchen_version: 1.3.1

loader:

  process_erb: true

  process_local: true

  process_global: true

  global_config: 

  project_config:

    filename: "/Users/lcosmes/branches/cookbooks/cookbooks/ffdc-common/.kitchen.yml"

    raw_data:

      driver:

        name: ec2

        aws_ssh_key_id: id_rsa-aws

        security_group_ids:

        - sg-1a2b3c4d

        region: us-east-1

        availability_zone: b

        require_chef_omnibus: true

        subnet_id: subnet-6d6...

        iam_profile_name: chef-client

        instance_type: t2.micro

        associate_public_ip: true

        private_ip_address: 10.0.0.27

        interface: dns

        block_device_mappings:

        - ebs_device_name: "/dev/sda0"

          ebs_volume_type: gp2

          ebs_virtual_name: test

          ebs_volume_size: 15

          ebs_delete_on_termination: true

      transport:

        ssh_key: "/path/to/id_rsa-aws"

        connection_timeout: 10

        connection_retries: 5

        username: ubuntu

      platforms:

      - name: ubuntu-12.04

        driver:

          image_id: ami-fd20ad94

          username: ubuntu

      - name: centos-6.3

        driver:

          image_id: ami-ef5ff086

          username: ec2-user

      suites:

      - name: default

        run_list:

        - recipe[ffdc-common::aws-scripts-mon]

        attributes: 

  local_config: 

  combined_config:

    filename: 

    raw_data:

      driver:

        name: ec2

        aws_ssh_key_id: id_rsa-aws

        security_group_ids:

        - sg-1a2b3c4d

        region: us-east-1

        availability_zone: b

        require_chef_omnibus: true

        subnet_id: subnet-6d6...

        iam_profile_name: chef-client

        instance_type: t2.micro

        associate_public_ip: true

        private_ip_address: 10.0.0.27

        interface: dns

        block_device_mappings:

        - ebs_device_name: "/dev/sda0"

          ebs_volume_type: gp2

          ebs_virtual_name: test

          ebs_volume_size: 15

          ebs_delete_on_termination: true

      transport:

        ssh_key: "/path/to/id_rsa-aws"

        connection_timeout: 10

        connection_retries: 5

        username: ubuntu

      platforms:

      - name: ubuntu-12.04

        driver:

          image_id: ami-fd20ad94

          username: ubuntu

      - name: centos-6.3

        driver:

          image_id: ami-ef5ff086

          username: ec2-user

      suites:

      - name: default

        run_list:

        - recipe[ffdc-common::aws-scripts-mon]

        attributes: 

instances:

  error:

    exception: "#<Kitchen::ClientError: Could not load the 'ec2' driver from the load

      path. Please ensure that your driver is installed as a gem or included in your

      Gemfile if using Bundler.>"

    message: Could not load the 'ec2' driver from the load path. Please ensure that

      your driver is installed as a gem or included in your Gemfile if using Bundler.

    backtrace:

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/driver.rb:50:in `rescue

      in for_plugin'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/driver.rb:40:in `for_plugin'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/config.rb:226:in `new_driver'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/config.rb:240:in `new_instance'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/config.rb:129:in `block

      in build_instances'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/config.rb:128:in `map'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/config.rb:128:in `with_index'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/config.rb:128:in `build_instances'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/config.rb:104:in `instances'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/command.rb:115:in `filtered_instances'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/command.rb:145:in `parse_subcommand'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/command/diagnose.rb:51:in

      `load_instances'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/command/diagnose.rb:35:in

      `block in call'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/command/diagnose.rb:75:in

      `record_failure'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/command/diagnose.rb:35:in

      `call'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/cli.rb:56:in `perform'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/cli.rb:116:in `diagnose'"

    - "/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/command.rb:27:in

      `run'"

    - "/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/invocation.rb:126:in

      `invoke_command'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/cli.rb:304:in `invoke_task'"

    - "/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor.rb:359:in

      `dispatch'"

    - "/opt/chefdk/embedded/lib/ruby/gems/2.1.0/gems/thor-0.19.1/lib/thor/base.rb:440:in

      `start'"

    - "/opt/chefdk/embedded/apps/test-kitchen/bin/kitchen:13:in `block in <top (required)>'"

    - "/opt/chefdk/embedded/apps/test-kitchen/lib/kitchen/errors.rb:154:in `with_friendly_errors'"

    - "/opt/chefdk/embedded/apps/test-kitchen/bin/kitchen:13:in `<top (required)>'"

    - "/usr/bin/kitchen:15:in `load'"

    - "/usr/bin/kitchen:15:in `<main>'"

On Tue, May 19, 2015 at 10:28 AM, Tyler Ball < " target="_blank" class=""> > wrote:

Greetings!

After a 15 month hiatius, I just released the latest version of kitchen-ec2.  Quite a lot has changed, so I’ll go over some of it here!

First off, all your existing tests and configuration should continue to work when moving from 0.8.0 to 0.9.0.  However, many configurations have been deprecated in preparation for a 1.0.0 release.  If you are using a deprecated config you should get a warning at the beginning of any kitchen command.  Check the README for deprecated configs and the recommended new value.

I also switched from using the Fog gem to using the AWS SDK gem for communication.  Again, this should not be a visible change for users, just a dependency change.  As part of this I deprecated the concept of storing the AWS secret key and token inside the .kitchen.yml.  Instead it is recommended to use environment variables, a ~/.aws/credentials file or an IAM instance profile if running Test Kitchen from within AWS.  Read more about it here.

This brings the credentials in line with how AWS recommends authenticating.

All deprecated logic will continue to be supported until version 1.0.0.  I’m planning on releasing a 0.10.0 version which adds Windows support, then quickly turning around and releasing 1.0.0 with the deprecated logic removed.  So if you are currently depending on the master branch of kitchen-ec2, please switch to a pinned gem dependency!

See the CHANGELOG for more detail around bug fixes and other small improvements.

Cheers!

-Tyler

Are you creating an ERP FrankenCloud? Watch 2:11 video

Are you creating an ERP FrankenCloud? Watch 2:11 video

--

Yoshi Spendiff

Ops Engineer

Indochino

Mobile: +1 778 952 2025

Email: " target="_blank" class="">

Are you creating an ERP FrankenCloud? Watch 2:11 video