[chef] RE: Re: RE: Re: RE: Re: RE: Re: RE: Re: Chef server update went wrong - how to repair?

[Kevin Keane Subscription < >]

Title: RE: [chef] Re: RE: Re: RE: Re: RE: Re: RE: Re: Chef server update went wrong - how to repair?

Thank you!

BTW, simply doing a chef-server-ctl start will start up all the services seemingly normally, but after I do that, I still get the "internal error" message from chef-client on all the clients.

I also checked to make sure that it is not caused by something as trivial as selinux (set the mode to permissive) or iptables (completely disabled it).

Kevin Keane

The NetTech

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html

-----Original message-----
From: Mark Mzyk < >
Sent: Tuesday 30th June 2015 14:52
To:
Subject: [chef] Re: RE: Re: RE: Re: RE: Re: RE: Re: Chef server update went wrong - how to repair?

I'm not sure Kevin. Let me put this thread in front of some of the other engineers and see if they know something around this that I don't.

- Mark

" type="cite">

" photoname="Kevin Keane Subscription" src="" name="compose-unknown-contact.jpg" height="25px" width="25px">

" style="color:#737F92 !important;padding-right:6px;font-weight:bold;text-decoration:none !important;" target="_blank" title="This external link will open in a new window">Kevin Keane Subscription

June 29, 2015 at 11:28 PM

No problem. Asking for logs is surely reflexive, because it usually really is a good idea!

Here is the output from chef-server-ctl upgrade (I didn't wait until the redis restart timed out, so the actual error message is missing at the end, but it is the restart of redis_lb that does not work.).

I notice that the schema updates do seem to take place well before the problem occurs.

Upgrading with options #<OpenStruct skip_confirmation=false, chef11_server_url="https://localhost", chef12_server_url="https://localhost", upload_threads=10, chef11_admin_client_name="admin", chef11_admin_client_key="/etc/chef-server/admin.pem">
Starting Chef Client, version 12.4.0.rc.2
resolving cookbooks for run list: ["private-chef::default"]
[2015-06-29T20:23:46-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:46-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:46-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:46-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:47-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:47-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:47-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:47-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:47-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:47-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:47-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:47-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:47-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:47-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:47-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:47-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:47-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:48-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:48-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:48-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:48-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:48-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:48-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:48-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:48-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:48-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:48-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:48-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:49-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:49-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:49-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:49-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:49-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:49-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:49-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
[2015-06-29T20:23:49-07:00] WARN: Cookbook 'local-mode-cache' is empty or entirely chefignored at /opt/opscode/embedded/cookbooks/local-mode-cache
Synchronizing Cookbooks:
  - private-chef
  - enterprise
  - apt
  - yum
  - runit
  - packagecloud
Compiling Cookbooks...
Recipe: private-chef::default
  * directory[/etc/opscode] action create (up to date)
  * directory[/etc/opscode/logrotate.d] action create (up to date)
[2015-06-29T20:23:50-07:00] WARN: 'path' attribute of 'execute' is not used by any provider in Chef 11 and Chef 12. Use 'environment' attribute to configure 'PATH'. This attribute will be removed in Chef 13.
[2015-06-29T20:23:51-07:00] WARN: Cloning resource attributes for runit_service[redis_lb] from prior resource (CHEF-3694)
[2015-06-29T20:23:51-07:00] WARN: Previous runit_service[redis_lb]: /opt/opscode/embedded/cookbooks/cache/cookbooks/enterprise/definitions/component_runit_service.rb:37:in `block in from_file'
[2015-06-29T20:23:51-07:00] WARN: Current  runit_service[redis_lb]: /opt/opscode/embedded/cookbooks/cache/cookbooks/private-chef/recipes/redis_lb.rb:83:in `from_file'
[2015-06-29T20:23:51-07:00] WARN: Cloning resource attributes for directory[/var/opt/opscode/nginx/etc/addon.d] from prior resource (CHEF-3694)
[2015-06-29T20:23:51-07:00] WARN: Previous directory[/var/opt/opscode/nginx/etc/addon.d]: /opt/opscode/embedded/cookbooks/cache/cookbooks/private-chef/recipes/oc_id.rb:173:in `from_file'
[2015-06-29T20:23:51-07:00] WARN: Current  directory[/var/opt/opscode/nginx/etc/addon.d]: /opt/opscode/embedded/cookbooks/cache/cookbooks/private-chef/recipes/nginx.rb:33:in `block in from_file'
  Converging 274 resources
  * link[/usr/bin/private-chef-ctl] action create (up to date)
  * link[/usr/bin/chef-server-ctl] action create (up to date)
  * directory[/etc/opscode] action nothing (skipped due to action :nothing)
  * directory[/etc/opscode/logrotate.d] action nothing (skipped due to action :nothing)
  * log[opscode_webui deprecation notice] action write (skipped due to only_if)
Recipe: private-chef::users
  * user[opscode] action create (up to date)
  * group[opscode] action create (up to date)
Recipe: private-chef::default
  * file[/etc/opscode/dark_launch_features.json] action create (up to date)
  * file[/etc/opscode/webui_pub.pem] action create (up to date)
  * file[/etc/opscode/webui_priv.pem] action create (up to date)
  * file[/etc/opscode/worker-public.pem] action create (up to date)
  * file[/etc/opscode/worker-private.pem] action create (up to date)
  * file[/etc/opscode/pivotal.pem] action create (up to date)
  * directory[/etc/chef] action create (up to date)
  * directory[/var/opt/opscode] action create (up to date)
  * directory[/var/log/opscode] action create (up to date)
  * private_chef_keepalived_safemode[warmfuzzy] action create (skipped due to only_if)
Recipe: enterprise::runit_upstart
  * execute[initctl stop opscode-runsvdir] action run (skipped due to only_if)
  * file[/etc/init/opscode-runsvdir.conf] action delete (up to date)
  * template[/etc/init/private-chef-runsvdir.conf] action create (up to date)
  * execute[initctl status private-chef-runsvdir] action run
    - execute initctl status private-chef-runsvdir
  * execute[initctl start private-chef-runsvdir] action run (skipped due to only_if)
Recipe: private-chef::sysctl-updates
  * execute[sysctl-reload] action nothing (skipped due to action :nothing)
  * bash[dual ip4/ip6 portbind] action run (skipped due to only_if)
Recipe: private-chef::plugins
  * chef_run[recipe[chef-ha-drbd::disable]] action run[2015-06-29T20:23:52-07:00] WARN: Class Chef::Provider::ChefRun does not declare 'provides :chef_run'.
[2015-06-29T20:23:52-07:00] WARN: This will no longer work in Chef 13: you must use 'provides' to provide DSL.
[2015-06-29T20:23:52-07:00] WARN: found a directory local-mode-cache in the cookbook path, but it contains no cookbook files. skipping.
Compiling Cookbooks...
Converging 0 resources

    - Running chef-run with run list recipe[chef-ha-drbd::disable]
Recipe: private-chef::rabbitmq
  * directory[/var/opt/opscode/rabbitmq] action create (up to date)
  * directory[/var/opt/opscode/rabbitmq/etc] action create (up to date)
  * directory[/var/opt/opscode/rabbitmq/db] action create (up to date)
  * directory[/var/log/opscode/rabbitmq] action create (up to date)
  * link[/var/opt/opscode/rabbitmq/db] action create (skipped due to not_if)
  * link[/opt/opscode/embedded/bin/rabbitmqctl] action create (up to date)
  * link[/opt/opscode/embedded/bin/rabbitmq-defaults] action create (up to date)
  * link[/opt/opscode/embedded/bin/rabbitmq-env] action create (up to date)
  * link[/opt/opscode/embedded/bin/rabbitmq-plugins] action create (up to date)
  * link[/opt/opscode/embedded/bin/rabbitmq-server] action create (up to date)
  * template[/opt/opscode/embedded/service/rabbitmq/sbin/rabbitmq-defaults] action create (up to date)
  * template[/var/opt/opscode/rabbitmq/etc/rabbitmq.conf] action create (up to date)
  * execute[restart_rabbitmq_log_service] action nothing (skipped due to action :nothing)
  * template[/var/log/opscode/rabbitmq/config] action create (up to date)
Recipe: <Dynamically Defined Resource>
  * service[rabbitmq] action nothing (skipped due to action :nothing)
Recipe: private-chef::rabbitmq
  * runit_service[rabbitmq] action enable
  Recipe: <Dynamically Defined Resource>
    * directory[/opt/opscode/sv/rabbitmq] action create (up to date)
    * template[/opt/opscode/sv/rabbitmq/run] action create (up to date)
    * directory[/opt/opscode/sv/rabbitmq/log] action create (up to date)
    * directory[/opt/opscode/sv/rabbitmq/log/main] action create (up to date)
    * template[/opt/opscode/sv/rabbitmq/log/run] action create (up to date)
    * template[/opt/opscode/sv/rabbitmq/log/config] action create (up to date)
    * link[/opt/opscode/init/rabbitmq] action create (up to date)
     (up to date)
Recipe: private-chef::rabbitmq
  * execute[/opt/opscode/bin/private-chef-ctl start rabbitmq] action run
    - execute /opt/opscode/bin/private-chef-ctl start rabbitmq
  * execute[/opt/opscode/embedded/bin/chpst -u opscode -U opscode /opt/opscode/embedded/bin/rabbitmqctl wait /var/opt/opscode/rabbitmq/db/ action run
    - execute /opt/opscode/embedded/bin/chpst -u opscode -U opscode /opt/opscode/embedded/bin/rabbitmqctl wait /var/opt/opscode/rabbitmq/db/
  * execute[/opt/opscode/embedded/bin/rabbitmqctl add_vhost /chef] action run (skipped due to not_if)
  * execute[/opt/opscode/embedded/bin/rabbitmqctl add_vhost /reindexer] action run (skipped due to not_if)
  * execute[/opt/opscode/embedded/bin/rabbitmqctl add_vhost /jobs] action run (skipped due to not_if)
  * execute[/opt/opscode/embedded/bin/rabbitmqctl add_vhost /analytics] action run (skipped due to not_if)
  * execute[/opt/opscode/embedded/bin/rabbitmqctl add_user chef ac6fc3163cf44cfdc681f57dc21a6c8fce7cded2693a776046c8796d8ab0d6b1647add2a36bdd65ed66a4a934d28cc73e43b] action run (skipped due to not_if)
  * execute[/opt/opscode/embedded/bin/rabbitmqctl add_user jobs a831b138caab7aabd0a0388955e73d83c6aa966fe602d0a3b7d1081cecbe47dd29d1cfef57ab184cf9ea60c2175e8023e047] action run (skipped due to not_if)
  * execute[/opt/opscode/embedded/bin/rabbitmqctl add_user actions 4910c8bd136fd60de76601d6233c7325467cf68ad4ae54ada2e4a73ac9daadc9e7c2324af050284a82314a7931b0ca00acf2] action run (skipped due to not_if)
  * execute[/opt/opscode/embedded/bin/rabbitmqctl set_permissions -p /chef chef ".*" ".*" ".*"] action run (skipped due to not_if)
  * execute[/opt/opscode/embedded/bin/rabbitmqctl set_permissions -p /reindexer chef ".*" ".*" ".*"] action run (skipped due to not_if)
  * execute[/opt/opscode/embedded/bin/rabbitmqctl set_permissions -p /jobs jobs ".*" ".*" ".*"] action run (skipped due to not_if)
  * execute[/opt/opscode/embedded/bin/rabbitmqctl set_permissions -p /analytics chef ".*" ".*" ".*"] action run (skipped due to not_if)
  * execute[/opt/opscode/embedded/bin/rabbitmqctl set_permissions -p /analytics actions ".*" ".*" ".*"] action run (skipped due to not_if)
Recipe: <Dynamically Defined Resource>
  * service[postgres] action nothing (skipped due to action :nothing)
Recipe: private-chef::old_postgres_cleanup
  * runit_service[postgres] action stop (up to date)
  * runit_service[postgres] action disable (up to date)
  * directory[/opt/opscode/sv/postgres] action delete (up to date)
Recipe: private-chef::postgresql
  * user[opscode-pgsql] action create (up to date)
  * directory[/var/opt/opscode/postgresql] action create (up to date)
  * file[/var/opt/opscode/postgresql/.profile] action create (up to date)
  * directory[/var/log/opscode/postgresql/9.2] action create (up to date)
  * directory[/var/opt/opscode/postgresql/9.2] action create (up to date)
  * service[procps] action nothing (skipped due to action :nothing)
  * template[/etc/sysctl.d/90-postgresql.conf] action create (up to date)
  * private_chef_pg_upgrade[upgrade_if_necessary] action upgrade
    - Database cluster is unchanged; nothing to upgrade
  * private_chef_pg_cluster[/var/opt/opscode/postgresql/9.2/data] action init
    * directory[/var/opt/opscode/postgresql/9.2/data] action create (up to date)
    * execute[initialize_cluster_/var/opt/opscode/postgresql/9.2/data] action run (skipped due to not_if)
    * template[/var/opt/opscode/postgresql/9.2/data/postgresql.conf] action create (up to date)
    * template[/var/opt/opscode/postgresql/9.2/data/pg_hba.conf] action create (up to date)
     (up to date)
  * link[/var/opt/opscode/postgresql/9.2/data] action create (skipped due to not_if)
  * execute[restart_postgresql_log_service] action nothing (skipped due to action :nothing)
  * template[/var/log/opscode/postgresql/9.2/config] action create (up to date)
Recipe: <Dynamically Defined Resource>
  * service[postgresql] action nothing (skipped due to action :nothing)
Recipe: private-chef::postgresql
  * runit_service[postgresql] action enable
  Recipe: <Dynamically Defined Resource>
    * directory[/opt/opscode/sv/postgresql] action create (up to date)
    * template[/opt/opscode/sv/postgresql/run] action create (up to date)
    * directory[/opt/opscode/sv/postgresql/log] action create (up to date)
    * directory[/opt/opscode/sv/postgresql/log/main] action create (up to date)
    * template[/opt/opscode/sv/postgresql/log/run] action create (up to date)
    * template[/opt/opscode/sv/postgresql/log/config] action create (up to date)
    * directory[/opt/opscode/sv/postgresql/control] action create (up to date)
    * template[/opt/opscode/sv/postgresql/control/t] action create (up to date)
    * link[/opt/opscode/init/postgresql] action create (up to date)
     (up to date)
Recipe: private-chef::postgresql
  * execute[/opt/opscode/bin/private-chef-ctl start postgresql] action run
    - execute /opt/opscode/bin/private-chef-ctl start postgresql
  * private_chef_pg_database[opscode-pgsql] action create
    * execute[create_database_opscode-pgsql] action run (skipped due to not_if)
     (up to date)
Recipe: private-chef::erchef_database
  * private_chef_pg_database[opscode_chef] action create
    * execute[create_database_opscode_chef] action run (skipped due to not_if)
     (up to date)
  * execute[chef-server-schema] action nothing (skipped due to action :nothing)
  * execute[enterprise-chef-server-schema] action nothing (skipped due to action :nothing)
  * private_chef_pg_user[opscode_chef] action create (up to date)
  * private_chef_pg_user_table_access[opscode_chef] action create
    - GRANT CONNECT, TEMPORARY ON DATABASE opscode_chef TO opscode_chef
    - ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT EXECUTE ON FUNCTIONS TO opscode_chef;
    - GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA public TO opscode_chef
    - ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, UPDATE ON SEQUENCES TO opscode_chef;
    - GRANT SELECT, UPDATE ON ALL SEQUENCES IN SCHEMA public TO opscode_chef
    - ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT INSERT, SELECT, UPDATE, DELETE ON TABLES TO opscode_chef;
    - GRANT INSERT, SELECT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO opscode_chef
  * private_chef_pg_user[opscode_chef_ro] action create (up to date)
  * private_chef_pg_user_table_access[opscode_chef_ro] action create
    - GRANT CONNECT ON DATABASE opscode_chef TO opscode_chef_ro
    - ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT EXECUTE ON FUNCTIONS TO opscode_chef_ro;
    - GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA public TO opscode_chef_ro
    - ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON SEQUENCES TO opscode_chef_ro;
    - GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO opscode_chef_ro
    - ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO opscode_chef_ro;
    - GRANT SELECT ON ALL TABLES IN SCHEMA public TO opscode_chef_ro
  * directory[/opt/opscode/embedded/service/enterprise-chef-server-schema] action delete (up to date)
Recipe: private-chef::bifrost_database
  * private_chef_pg_user[bifrost] action create (up to date)
  * private_chef_pg_user[bifrost_ro] action create (up to date)
  * private_chef_pg_database[bifrost] action create
    * execute[create_database_bifrost] action run (skipped due to not_if)
     (up to date)
  * private_chef_pg_user_table_access[bifrost] action create
    - GRANT CONNECT, TEMPORARY ON DATABASE bifrost TO bifrost
    - ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT EXECUTE ON FUNCTIONS TO bifrost;
    - GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA public TO bifrost
    - ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT, UPDATE ON SEQUENCES TO bifrost;
    - GRANT SELECT, UPDATE ON ALL SEQUENCES IN SCHEMA public TO bifrost
    - ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT INSERT, SELECT, UPDATE, DELETE ON TABLES TO bifrost;
    - GRANT INSERT, SELECT, UPDATE, DELETE ON ALL TABLES IN SCHEMA public TO bifrost
  * private_chef_pg_user_table_access[bifrost_ro] action create
    - GRANT CONNECT ON DATABASE bifrost TO bifrost_ro
    - ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT EXECUTE ON FUNCTIONS TO bifrost_ro;
    - GRANT EXECUTE ON ALL FUNCTIONS IN SCHEMA public TO bifrost_ro
    - ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON SEQUENCES TO bifrost_ro;
    - GRANT SELECT ON ALL SEQUENCES IN SCHEMA public TO bifrost_ro
    - ALTER DEFAULT PRIVILEGES IN SCHEMA public GRANT SELECT ON TABLES TO bifrost_ro;
    - GRANT SELECT ON ALL TABLES IN SCHEMA public TO bifrost_ro
  * execute[bifrost_schema] action run
    - execute sqitch --engine pg --db-name bifrost --top-dir /opt/opscode/embedded/service/oc_bifrost/db deploy --verify
  * execute[add_permissions_bifrost] action run
    - execute psql --dbname bifrost --single-transaction --set --set database_name=bifrost --file sql/permissions.sql
Recipe: private-chef::oc_id_database
  * private_chef_pg_user[oc_id] action create (up to date)
  * private_chef_pg_database[oc_id] action create
    * execute[create_database_oc_id] action run (skipped due to not_if)
     (up to date)
Recipe: private-chef::oc_bifrost
  * execute[/opt/opscode/bin/private-chef-ctl stop opscode-authz] action run
    - execute /opt/opscode/bin/private-chef-ctl stop opscode-authz
Recipe: <Dynamically Defined Resource>
  * service[opscode-authz] action nothing (skipped due to action :nothing)
Recipe: private-chef::oc_bifrost
  * runit_service[opscode-authz] action disable (up to date)
  * directory[/opt/opscode/sv/opscode-authz] action delete (up to date)
  * directory[/var/opt/opscode/oc_bifrost] action create (up to date)
  * directory[/var/opt/opscode/oc_bifrost/bin] action create (up to date)
  * directory[/var/opt/opscode/oc_bifrost/etc] action create (up to date)
  * directory[/var/log/opscode/oc_bifrost] action create (up to date)
  * directory[/var/log/opscode/oc_bifrost/sasl] action create (up to date)
  * link[/opt/opscode/embedded/service/oc_bifrost/log] action create (up to date)
  * template[/var/opt/opscode/oc_bifrost/sys.config] action create (up to date)
  * link[/opt/opscode/embedded/service/oc_bifrost/sys.config] action create (up to date)
  * execute[restart_oc_bifrost_log_service] action nothing (skipped due to action :nothing)
  * template[/var/log/opscode/oc_bifrost/config] action create (up to date)
Recipe: <Dynamically Defined Resource>
  * service[oc_bifrost] action nothing (skipped due to action :nothing)
Recipe: private-chef::oc_bifrost
  * runit_service[oc_bifrost] action enable
  Recipe: <Dynamically Defined Resource>
    * directory[/opt/opscode/sv/oc_bifrost] action create (up to date)
    * template[/opt/opscode/sv/oc_bifrost/run] action create (up to date)
    * directory[/opt/opscode/sv/oc_bifrost/log] action create (up to date)
    * directory[/opt/opscode/sv/oc_bifrost/log/main] action create (up to date)
    * template[/opt/opscode/sv/oc_bifrost/log/run] action create (up to date)
    * template[/opt/opscode/sv/oc_bifrost/log/config] action create (up to date)
    * link[/opt/opscode/init/oc_bifrost] action create (up to date)
     (up to date)
Recipe: private-chef::oc_id
  * directory[/var/opt/opscode/oc_id] action create (up to date)
  * directory[/var/opt/opscode/oc_id/config] action create (up to date)
  * directory[/var/opt/opscode/oc_id/tmp] action create (up to date)
  * directory[/var/log/opscode/oc_id] action create (up to date)
  * directory[/opt/opscode/embedded/service/oc_id/log] action delete (skipped due to not_if)
  * directory[/opt/opscode/embedded/service/oc_id/tmp] action delete (skipped due to not_if)
  * link[/opt/opscode/embedded/service/oc_id/log] action create (up to date)
  * link[/opt/opscode/embedded/service/oc_id/tmp] action create (up to date)
  * file[/var/opt/opscode/oc_id/config/production.yml] action create (up to date)
  * file[/opt/opscode/embedded/service/oc_id/config/settings/production.yml] action delete (skipped due to not_if)
  * link[/opt/opscode/embedded/service/oc_id/config/settings/production.yml] action create (up to date)
  * template[/var/opt/opscode/oc_id/config/secret_token.rb] action create (up to date)
  * file[/opt/opscode/embedded/service/oc_id/config/initializers/secret_token.rb] action delete (skipped due to not_if)
  * link[/opt/opscode/embedded/service/oc_id/config/initializers/secret_token.rb] action create (up to date)
  * template[/var/opt/opscode/oc_id/config/database.yml] action create (up to date)
  * file[/opt/opscode/embedded/service/oc_id/config/database.yml] action delete (skipped due to not_if)
  * link[/opt/opscode/embedded/service/oc_id/config/database.yml] action create (up to date)
  * execute[oc_id_schema] action run
    - execute bundle exec rake db:migrate
  * execute[restart_oc_id_log_service] action nothing (skipped due to action :nothing)
  * template[/var/log/opscode/oc_id/config] action create (up to date)
Recipe: <Dynamically Defined Resource>
  * service[oc_id] action nothing (skipped due to action :nothing)
Recipe: private-chef::oc_id
  * runit_service[oc_id] action enable
  Recipe: <Dynamically Defined Resource>
    * directory[/opt/opscode/sv/oc_id] action create (up to date)
    * template[/opt/opscode/sv/oc_id/run] action create (up to date)
    * directory[/opt/opscode/sv/oc_id/log] action create (up to date)
    * directory[/opt/opscode/sv/oc_id/log/main] action create (up to date)
    * template[/opt/opscode/sv/oc_id/log/run] action create (up to date)
    * template[/opt/opscode/sv/oc_id/log/config] action create (up to date)
    * link[/opt/opscode/init/oc_id] action create (up to date)
     (up to date)
Recipe: private-chef::oc_id
  * directory[/var/opt/opscode/nginx/etc/addon.d] action create (up to date)
  * template[/var/opt/opscode/nginx/etc/addon.d/40-oc_id_upstreams.conf] action create (up to date)
  * template[/var/opt/opscode/nginx/etc/addon.d/40-oc_id_external.conf] action create (up to date)
Recipe: private-chef::opscode-solr4
  * directory[/var/opt/opscode/opscode-solr4] action create (up to date)
  * directory[/var/opt/opscode/opscode-solr4/data] action create (up to date)
  * directory[/var/opt/opscode/opscode-solr4/home] action create (up to date)
  * directory[/var/opt/opscode/opscode-solr4/] action create (up to date)
  * directory[/var/log/opscode/opscode-solr4] action create (up to date)
  * directory[/var/opt/opscode/opscode-solr4/home/collection1] action create (up to date)
  * directory[/var/opt/opscode/opscode-solr4/home/collection1/conf] action create (up to date)
  * link[/var/opt/opscode/opscode-solr4/data] action create (skipped due to not_if)
  * cookbook_file[/var/opt/opscode/opscode-solr4/home/solr.xml] action create (up to date)
  * file[/var/opt/opscode/opscode-solr4/home/collection1/core.properties] action create (up to date)
  * template[/var/opt/opscode/opscode-solr4/home/collection1/conf/solrconfig.xml] action create (up to date)
  * cookbook_file[/var/opt/opscode/opscode-solr4/home/collection1/conf/schema.xml] action create (up to date)
  * template[/opt/opscode/embedded/service/opscode-solr4/jetty/etc/jetty.xml] action create (up to date)
  * template[/opt/opscode/embedded/service/opscode-solr4/jetty/contexts/solr-jetty-context.xml] action create (up to date)
  * execute[chown -R opscode /opt/opscode/embedded/service/opscode-solr4/jetty] action run
    - execute chown -R opscode /opt/opscode/embedded/service/opscode-solr4/jetty
  * execute[restart_opscode-solr4_log_service] action nothing (skipped due to action :nothing)
  * template[/var/log/opscode/opscode-solr4/config] action create (up to date)
Recipe: <Dynamically Defined Resource>
  * service[opscode-solr4] action nothing (skipped due to action :nothing)
Recipe: private-chef::opscode-solr4
  * runit_service[opscode-solr4] action enable
  Recipe: <Dynamically Defined Resource>
    * directory[/opt/opscode/sv/opscode-solr4] action create (up to date)
    * template[/opt/opscode/sv/opscode-solr4/run] action create (up to date)
    * directory[/opt/opscode/sv/opscode-solr4/log] action create (up to date)
    * directory[/opt/opscode/sv/opscode-solr4/log/main] action create (up to date)
    * template[/opt/opscode/sv/opscode-solr4/log/run] action create (up to date)
    * template[/opt/opscode/sv/opscode-solr4/log/config] action create (up to date)
    * link[/opt/opscode/init/opscode-solr4] action create (up to date)
     (up to date)
Recipe: private-chef::opscode-solr4
  * template[/etc/opscode/logrotate.d/opscode-solr4] action create (up to date)
Recipe: private-chef::opscode-expander
  * directory[/var/opt/opscode/opscode-expander] action create (up to date)
  * directory[/var/opt/opscode/opscode-expander/etc] action create (up to date)
  * directory[/var/log/opscode/opscode-expander] action create (up to date)
  * directory[/var/log/opscode/opscode-expander-reindexer] action create (up to date)
  * template[/var/opt/opscode/opscode-expander/etc/expander.rb] action create (up to date)
  * link[/opt/opscode/embedded/service/opscode-expander/conf/opscode-expander.rb] action create (up to date)
  * template[/var/opt/opscode/opscode-expander/etc/reindexer.rb] action create (up to date)
  * execute[restart_opscode-expander_log_service] action nothing (skipped due to action :nothing)
  * template[/var/log/opscode/opscode-expander/config] action create (up to date)
Recipe: <Dynamically Defined Resource>
  * service[opscode-expander] action nothing (skipped due to action :nothing)
Recipe: private-chef::opscode-expander
  * runit_service[opscode-expander] action enable
  Recipe: <Dynamically Defined Resource>
    * directory[/opt/opscode/sv/opscode-expander] action create (up to date)
    * template[/opt/opscode/sv/opscode-expander/run] action create (up to date)
    * directory[/opt/opscode/sv/opscode-expander/log] action create (up to date)
    * directory[/opt/opscode/sv/opscode-expander/log/main] action create (up to date)
    * template[/opt/opscode/sv/opscode-expander/log/run] action create (up to date)
    * template[/opt/opscode/sv/opscode-expander/log/config] action create (up to date)
    * link[/opt/opscode/init/opscode-expander] action create (up to date)
     (up to date)
Recipe: private-chef::opscode-expander
  * execute[restart_opscode-expander-reindexer_log_service] action nothing (skipped due to action :nothing)
  * template[/var/log/opscode/opscode-expander-reindexer/config] action create (up to date)
Recipe: <Dynamically Defined Resource>
  * service[opscode-expander-reindexer] action nothing (skipped due to action :nothing)
Recipe: private-chef::opscode-expander
  * runit_service[opscode-expander-reindexer] action enable
  Recipe: <Dynamically Defined Resource>
    * directory[/opt/opscode/sv/opscode-expander-reindexer] action create (up to date)
    * template[/opt/opscode/sv/opscode-expander-reindexer/run] action create (up to date)
    * directory[/opt/opscode/sv/opscode-expander-reindexer/log] action create (up to date)
    * directory[/opt/opscode/sv/opscode-expander-reindexer/log/main] action create (up to date)
    * template[/opt/opscode/sv/opscode-expander-reindexer/log/run] action create (up to date)
    * template[/opt/opscode/sv/opscode-expander-reindexer/log/config] action create (up to date)
    * link[/opt/opscode/init/opscode-expander-reindexer] action create (up to date)
     (up to date)
Recipe: private-chef::bookshelf
  * template[/opt/opscode/embedded/bin/cookbook_migration.sh] action create (up to date)
  * directory[/var/opt/opscode/bookshelf] action create (up to date)
  * directory[/var/opt/opscode/bookshelf/etc] action create (up to date)
  * directory[/var/log/opscode/bookshelf] action create (up to date)
  * directory[/var/log/opscode/bookshelf/sasl] action create (up to date)
  * directory[/var/opt/opscode/bookshelf/data] action create (up to date)
  * execute[cookbook migration] action run (skipped due to not_if)
  * link[/opt/opscode/embedded/service/bookshelf/log] action create (up to date)
  * template[/var/opt/opscode/bookshelf/sys.config] action create (up to date)
  * link[/opt/opscode/embedded/service/bookshelf/sys.config] action create (up to date)
  * execute[restart_bookshelf_log_service] action nothing (skipped due to action :nothing)
  * template[/var/log/opscode/bookshelf/config] action create (up to date)
Recipe: <Dynamically Defined Resource>
  * service[bookshelf] action nothing (skipped due to action :nothing)
Recipe: private-chef::bookshelf
  * runit_service[bookshelf] action enable
  Recipe: <Dynamically Defined Resource>
    * directory[/opt/opscode/sv/bookshelf] action create (up to date)
    * template[/opt/opscode/sv/bookshelf/run] action create (up to date)
    * directory[/opt/opscode/sv/bookshelf/log] action create (up to date)
    * directory[/opt/opscode/sv/bookshelf/log/main] action create (up to date)
    * template[/opt/opscode/sv/bookshelf/log/run] action create (up to date)
    * template[/opt/opscode/sv/bookshelf/log/config] action create (up to date)
    * link[/opt/opscode/init/bookshelf] action create (up to date)
     (up to date)
Recipe: private-chef::opscode-erchef
  * directory[/var/opt/opscode/opscode-erchef] action create (up to date)
  * directory[/var/opt/opscode/opscode-erchef/etc] action create (up to date)
  * directory[/var/log/opscode/opscode-erchef] action create (up to date)
  * directory[/var/log/opscode/opscode-erchef/sasl] action create (up to date)
  * link[/opt/opscode/embedded/service/opscode-erchef/log] action create (up to date)
  * template[/var/opt/opscode/opscode-erchef/sys.config] action create (up to date)
  * execute[remove_erchef_siz_files] action nothing (skipped due to action :nothing)
  * link[/opt/opscode/embedded/service/opscode-erchef/sys.config] action create (up to date)
  * execute[restart_opscode-erchef_log_service] action nothing (skipped due to action :nothing)
  * template[/var/log/opscode/opscode-erchef/config] action create (up to date)
Recipe: <Dynamically Defined Resource>
  * service[opscode-erchef] action nothing (skipped due to action :nothing)
Recipe: private-chef::opscode-erchef
  * runit_service[opscode-erchef] action enable
  Recipe: <Dynamically Defined Resource>
    * directory[/opt/opscode/sv/opscode-erchef] action create (up to date)
    * template[/opt/opscode/sv/opscode-erchef/run] action create (up to date)
    * directory[/opt/opscode/sv/opscode-erchef/log] action create (up to date)
    * directory[/opt/opscode/sv/opscode-erchef/log/main] action create (up to date)
    * template[/opt/opscode/sv/opscode-erchef/log/run] action create (up to date)
    * template[/opt/opscode/sv/opscode-erchef/log/config] action create (up to date)
    * link[/opt/opscode/init/opscode-erchef] action create (up to date)
     (up to date)
Recipe: private-chef::opscode-chef-mover
  * directory[/var/opt/opscode/opscode-chef-mover] action create (up to date)
  * directory[/var/opt/opscode/opscode-chef-mover/etc] action create (up to date)
  * directory[/var/opt/opscode/opscode-chef-mover/data] action create (up to date)
  * directory[/var/log/opscode/opscode-chef-mover] action create (up to date)
  * directory[/var/log/opscode/opscode-chef-mover/sasl] action create (up to date)
  * link[/opt/opscode/embedded/service/opscode-chef-mover/log] action create (up to date)
  * template[/var/opt/opscode/opscode-chef-mover/sys.config] action create (up to date)
  * link[/opt/opscode/embedded/service/opscode-chef-mover/sys.config] action create (up to date)
  * execute[restart_opscode-chef-mover_log_service] action nothing (skipped due to action :nothing)
  * template[/var/log/opscode/opscode-chef-mover/config] action create (up to date)
Recipe: <Dynamically Defined Resource>
  * service[opscode-chef-mover] action nothing (skipped due to action :nothing)
Recipe: private-chef::opscode-chef-mover
  * runit_service[opscode-chef-mover] action enable
  Recipe: <Dynamically Defined Resource>
    * directory[/opt/opscode/sv/opscode-chef-mover] action create (up to date)
    * template[/opt/opscode/sv/opscode-chef-mover/run] action create (up to date)
    * directory[/opt/opscode/sv/opscode-chef-mover/log] action create (up to date)
    * directory[/opt/opscode/sv/opscode-chef-mover/log/main] action create (up to date)
    * template[/opt/opscode/sv/opscode-chef-mover/log/run] action create (up to date)
    * template[/opt/opscode/sv/opscode-chef-mover/log/config] action create (up to date)
    * link[/opt/opscode/init/opscode-chef-mover] action create (up to date)
     (up to date)
Recipe: private-chef::opscode-chef-mover
  * log[stop runit_service[opscode-chef-mover]] action write
 
  * runit_service[opscode-chef-mover] action down
    - send down to runit_service[opscode-chef-mover]
Recipe: private-chef::redis_lb
  * directory[/var/opt/opscode/redis_lb] action create (up to date)
  * directory[/var/opt/opscode/redis_lb/etc] action create (up to date)
  * directory[/var/opt/opscode/redis_lb/data] action create (up to date)
  * directory[/var/log/opscode/redis_lb] action create (up to date)
  * link[/var/opt/opscode/redis_lb/data] action create (skipped due to not_if)
  * template[/var/opt/opscode/redis_lb/etc/redis.conf] action create (up to date)
  * execute[restart_redis_lb_log_service] action nothing (skipped due to action :nothing)
  * template[/var/log/opscode/redis_lb/config] action create (up to date)
Recipe: <Dynamically Defined Resource>
  * service[redis_lb] action nothing (skipped due to action :nothing)
Recipe: private-chef::redis_lb
  * runit_service[redis_lb] action enable
  Recipe: <Dynamically Defined Resource>
    * directory[/opt/opscode/sv/redis_lb] action create (up to date)
    * template[/opt/opscode/sv/redis_lb/run] action create (up to date)
    * directory[/opt/opscode/sv/redis_lb/log] action create (up to date)
    * directory[/opt/opscode/sv/redis_lb/log/main] action create (up to date)
    * template[/opt/opscode/sv/redis_lb/log/run] action create (up to date)
    * template[/opt/opscode/sv/redis_lb/log/config] action create (up to date)
    * link[/opt/opscode/init/redis_lb] action create (up to date)
     (up to date)
  * service[redis_lb] action nothing (skipped due to action :nothing)
  * service[redis_lb] action nothing (skipped due to action :nothing)
Recipe: private-chef::redis_lb
  * runit_service[redis_lb] action restart

Kevin Keane

The NetTech

http://www.4nettech.com

Our values: Privacy, Liberty, Justice

See https://www.4nettech.com/corp/the-nettech-values.html