Merge pull request #64 from zl4bv/source

Add LWRPs for source config files

.kitchen.yml

@@ -69,3 +69,19 @@ suites:
       - centos-7.1
       - debian-8.1
       - ubuntu-14.04
+  - name: source-resource
+    run_list:
+      - recipe[source-resource]
+      - recipe[source-resource::docker_create]
+      - recipe[source-resource::local_file_create]
+      - recipe[source-resource::local_win_event_log_create]
+      - recipe[source-resource::remote_file_create]
+      - recipe[source-resource::remote_win_event_log_create]
+      - recipe[source-resource::script_create]
+      - recipe[source-resource::syslog_create]
+    attributes:
+      SUMO_ACCESS_ID: <%= ENV['SUMO_ACCESS_ID'] %>
+      SUMO_ACCESS_KEY: <%= ENV['SUMO_ACCESS_KEY'] %>
+    excludes:
+      - windows-2008r2
+      - windows-2012r2

Berksfile

@@ -5,4 +5,5 @@ metadata
 group :integration do
   cookbook 'data-bag-faker', path: 'test/fixtures/cookbooks/data-bag-faker'
   cookbook 'installer-resource', path: 'test/fixtures/cookbooks/installer-resource'
+  cookbook 'source-resource', path: 'test/fixtures/cookbooks/source-resource'
 end

README.md

@@ -24,20 +24,24 @@ Note
 Starting from 19.107, there are 2 major extensions to SumoLogic collectors:
 * You can configure a collector's parameters from a set of json files under a common folder. Each of the json file will represent a source on that collector. Updates made to a json file will then be reflected on its corresponding source. Note that the format of this kind of file is **slightly different** from that of the traditional single json file (sumo.json) and they are **not** compatible. You also need to use the parameter `syncSources` instead of `sources` inside `sumo.conf`. See more details [here](https://service.sumologic.com/help/Default.htm#Using_sumo.conf.htm).
 * You can change a collector's existing parameters through local configuration json file(s) continuously. Before this, using collector API was the only option. More information about this is [here](https://service.sumologic.com/help/Default.htm#Using_Local_Configuration_File_Management.htm)
- 
+
 Installation
 ------------
 1. Create an [Access Key](http://help.sumologic.com/i19.69v2/Default.htm#Generating_Collector_Installation_API_Keys.htm)
 2. Install the cookbook in your Chef repo (your knife version should be at least 11.10.4 and you should have the [knife github plugin](https://github.com/websterclay/knife-github-cookbooks) installed):
-```knife cookbook github install SumoLogic/sumologic-collector-chef-cookbook``` 
+```
+knife cookbook github install SumoLogic/sumologic-collector-chef-cookbook
+```
 3. Specify data bag and item with your access credentials.  The data item should
 contain attributes `accessID` and `accessKey`.  Note that attribute names are case sensitive.  If the cases mismatch, the values will not appear when chef-client runs.  The default data bag/item is
 `['sumo-creds']['api-creds']`
-4. (Optional) Decide if you want to use the Local Configuration Management feature by setting the attribute `default['sumologic']['local_management']` properly. By default this feature is on, to leverage the power of Chef. 
+4. (Optional) Decide if you want to use the Local Configuration Management feature by setting the attribute `default['sumologic']['local_management']` properly. By default this feature is on, to leverage the power of Chef.
 5. (Optional) Select the json configuration option (i.e. through a single file or a folder) by setting the attribute `default['sumologic']['use_json_path_dir']` appropriately. By default a single json file is used.
-6. (Optional) Check if the path to the json file or the json folder is set correctly in the attribute `default['sumologic']['sumo_json_path']`. By default this is the path to the json file at `/etc/sumo.json` on Linux or `c:\sumo\sumo.json` on Windows. 
+6. (Optional) Check if the path to the json file or the json folder is set correctly in the attribute `default['sumologic']['sumo_json_path']`. By default this is the path to the json file at `/etc/sumo.json` on Linux or `c:\sumo\sumo.json` on Windows.
 7. Upload the cookbook to your Chef Server:
-```knife cookbook upload sumologic-collector```
+```
+knife cookbook upload sumologic-collector
+```
 8. Add the `sumologic-collector` receipe to your node run lists.  This step depends
 on your node configuration, so specifics will not be described in this README.md.
 
@@ -78,6 +82,319 @@ Attributes
   </tr>
 </table>
 
+Resource/Provider
+-----------------
+
+sumologic_collector_installer
+---------
+
+Allows for additional customisation of the Sumo Logic Collector installer
+
+### Actions
+`default` = `:install`
+
+- `:install` - installs the Sumo Logic Collector if it is not already installed
+
+### Attribute Parameters
+
+See the [Sumo Logic documentation](https://service.sumologic.com/help/Default.htm#Using_Quiet_Mode_to_Install_a_Collector.htm)
+for more information about these attributes.
+
+- `dir` - Directory where the Collector will be installed
+- `source` - URL where installer will be downloaded from
+- `collector_name`
+- `collector_url`
+- `sumo_email`
+- `sumo_password`
+- `sumo_token_and_url`
+- `sumo_access_id`
+- `sumo_access_key`
+- `proxy_host`
+- `proxy_port`
+- `proxy_user`
+- `proxy_password`
+- `proxy_ntlmdomain`
+- `sources`
+- `sync_sources`
+- `ephemeral`
+- `clobber`
+- `runas_username`
+- `winrunas_password`
+- `skip_registration`
+
+### Examples
+
+```ruby
+# Installs the Collector on Windows and skips registration
+sumologic_collector_installer 'c:\sumo' do
+  source 'https://collectors.sumologic.com/rest/download/win64'
+  sumo_access_id node['SUMO_ACCESS_ID']
+  sumo_access_key node['SUMO_ACCESS_KEY']
+  skip_registration true
+end
+```
+
+Collector Sources
+---------
+
+### Attribute Parameters
+
+The following attributes are common to all of the sources listed below.
+
+See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
+for more information about these attributes.
+
+- `owner` - owner of the JSON Source configuration file
+- `group` - group of the JSON Source configuration file
+- `mode` - file mode of the JSON Source configuration file
+- `source_name` - name of the source. **required**
+- `source_json_directory` - directory where JSON Source configuration file will be stored. **required**
+- `description`
+- `category`
+- `host_name`
+- `time_zone`
+- `automatic_date_parsing`
+- `multiline_processing_enabled`
+- `use_autoline_matching`
+- `manual_prefix_regexp`
+- `force_time_zone`
+- `default_date_format`
+- `filters`
+- `alive`
+
+sumo_source_docker
+---------
+
+### Actions
+`default` = `:create`
+
+- `:create` - creates a JSON Source configuration
+
+### Attribute Parameters
+
+See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
+for more information about these attributes.
+
+The following attribute parameters are in addition to the generic parameters
+listed above.
+
+- `uri`
+- `specified_containers`
+- `all_containers`
+- `cert_path`
+- `source_type` - one of `:docker_stats`, `:docker_log`. **required**
+- `collected_events`
+
+### Examples
+
+```ruby
+sumo_source_docker 'docker_stats' do
+  source_json_directory node['sumologic']['sumo_json_path']
+  source_type :docker_stats
+  uri 'https://127.0.0.1:2376'
+  all_containers true
+end
+
+sumo_source_docker 'docker_log' do
+  source_json_directory node['sumologic']['sumo_json_path']
+  source_type :docker_log
+  uri 'https://127.0.0.1:2376'
+  all_containers true
+end
+```
+
+sumo_source_local_file
+---------
+
+### Actions
+`default` = `:create`
+
+- `:create` - creates a JSON Source configuration
+
+### Attribute Parameters
+
+See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
+for more information about these attributes.
+
+The following attribute parameters are in addition to the generic parameters
+listed above.
+
+- `path_expression` - **required**
+- `blacklist`
+- `encoding`
+
+### Examples
+
+```ruby
+sumo_source_local_file 'local_file' do
+  source_json_directory node['sumologic']['sumo_json_path']
+  path_expression '/tmp/example'
+end
+```
+
+sumo_source_local_windows_event_log
+---------
+
+### Actions
+`default` = `:create`
+
+- `:create` - creates a JSON Source configuration
+
+### Attribute Parameters
+
+See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
+for more information about these attributes.
+
+The following attribute parameters are in addition to the generic parameters
+listed above.
+
+- `log_names` - **required**
+
+### Examples
+
+```ruby
+sumo_source_local_windows_event_log 'local_win_event_log' do
+  source_json_directory node['sumologic']['sumo_json_path']
+  log_names ['security', 'application']
+end
+```
+
+sumo_source_remote_file
+---------
+
+### Actions
+`default` = `:create`
+
+- `:create` - creates a JSON Source configuration
+
+### Attribute Parameters
+
+See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
+for more information about these attributes.
+
+The following attribute parameters are in addition to the generic parameters
+listed above.
+
+- `remote_hosts` - **required**
+- `remote_port` - **required**
+- `remote_user` - **required**
+- `remote_password` - **required**
+- `key_path` - **required**
+- `key_password`
+- `path_expression` - **required**
+- `auth_method` - one of `key` or `password`.
+- `blacklist`
+
+### Examples
+
+```ruby
+sumo_source_remote_file 'remote_file' do
+  source_json_directory node['sumologic']['sumo_json_path']
+  remote_hosts ['127.0.0.1']
+  remote_port 22
+  remote_user 'user'
+  remote_password 'password'
+  key_path ''
+  path_expression '/tmp/example'
+  auth_method 'password'
+end
+```
+
+sumo_source_remote_windows_event_log
+---------
+
+### Actions
+`default` = `:create`
+
+- `:create` - creates a JSON Source configuration
+
+### Attribute Parameters
+
+See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
+for more information about these attributes.
+
+The following attribute parameters are in addition to the generic parameters
+listed above.
+
+- `domain` - **required**
+- `username` - **required**
+- `password` - **required**
+- `hosts` - **required**
+- `log_names` - **required**
+
+### Examples
+
+```ruby
+sumo_source_remote_windows_event_log 'remote_win_event_log' do
+  source_json_directory node['sumologic']['sumo_json_path']
+  domain 'mydomain'
+  username 'user'
+  password 'password'
+  hosts ['myremotehost1']
+  log_names ['security', 'application']
+end
+```
+
+sumo_source_script
+---------
+
+### Actions
+`default` = `:create`
+
+- `:create` - creates a JSON Source configuration
+
+### Attribute Parameters
+
+See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
+for more information about these attributes.
+
+The following attribute parameters are in addition to the generic parameters
+listed above.
+
+- `commands` - **required**
+- `file`
+- `working_dir`
+- `timeout`
+- `script`
+- `cron_expression`
+
+### Examples
+
+```ruby
+sumo_source_script 'script' do
+  source_json_directory node['sumologic']['sumo_json_path']
+  commands ['/bin/bash']
+  cron_expression '0 * * * *'
+end
+```
+
+sumo_source_syslog
+---------
+
+### Actions
+`default` = `:create`
+
+- `:create` - creates a JSON Source configuration
+
+### Attribute Parameters
+
+See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
+for more information about these attributes.
+
+The following attribute parameters are in addition to the generic parameters
+listed above.
+
+- `protocol`
+- `port`
+
+### Examples
+
+```ruby
+sumo_source_syslog 'syslog' do
+  source_json_directory node['sumologic']['sumo_json_path']
+end
+```
+
 Contributing
 ------------
 This cookbook is meant to help customers use Chef to install Sumo Logic

libraries/provider_docker_source.rb

+require 'chef/provider/lwrp_base'
+require_relative 'provider_source'
+
+class Chef
+  class Provider
+    class SumoSourceDocker < Chef::Provider::SumoSource
+      provides :sumo_source_docker
+
+      def config_hash
+        hash = super
+        hash['source']['uri'] = new_resource.uri
+        hash['source']['specifiedContainers'] = new_resource.specified_containers unless new_resource.specified_containers.nil?
+        hash['source']['allContainers'] = new_resource.all_containers
+        hash['source']['certPath'] = new_resource.cert_path unless new_resource.cert_path.nil?
+        hash['source']['sourceType'] = source_type_map[new_resource.source_type]
+        hash['source']['collectEvents'] = new_resource.collect_events unless new_resource.collect_events.nil?
+        hash
+      end
+    end
+  end
+end

libraries/provider_local_file_source.rb

+require 'chef/provider/lwrp_base'
+require_relative 'provider_source'
+
+class Chef
+  class Provider
+    class SumoSourceLocalFile < Chef::Provider::SumoSource
+      provides :sumo_source_local_file
+
+      def config_hash
+        hash = super
+        hash['source']['pathExpression'] = new_resource.path_expression unless new_resource.path_expression.nil?
+        hash['source']['blacklist'] = new_resource.blacklist unless new_resource.blacklist.nil?
+        hash['source']['encoding'] = new_resource.encoding unless new_resource.encoding.nil?
+        hash
+      end
+    end
+  end
+end

libraries/provider_local_win_event_log_source.rb

+require 'chef/provider/lwrp_base'
+require_relative 'provider_source'
+
+class Chef
+  class Provider
+    class SumoSourceLocalWindowsEventLog < Chef::Provider::SumoSource
+      provides :sumo_source_local_windows_event_log
+
+      def config_hash
+        hash = super
+        hash['source']['logNames'] = new_resource.log_names
+        hash
+      end
+    end
+  end
+end

libraries/provider_remote_file_source.rb

+require 'chef/provider/lwrp_base'
+require_relative 'provider_source'
+
+class Chef
+  class Provider
+    class SumoSourceRemoteFile < Chef::Provider::SumoSource
+      provides :sumo_source_remote_file
+
+      def config_hash
+        hash = super
+        hash['source']['remoteHosts'] = new_resource.remote_hosts
+        hash['source']['remotePort'] = new_resource.remote_port
+        hash['source']['remoteUser'] = new_resource.remote_user
+        hash['source']['remotePassword'] = new_resource.remote_password
+        hash['source']['keyPath'] = new_resource.key_path
+        hash['source']['keyPassword'] = new_resource.key_password
+        hash['source']['pathExpression'] = new_resource.path_expression unless new_resource.path_expression.nil?
+        hash['source']['authMethod'] = new_resource.auth_method
+        hash['source']['blacklist'] = new_resource.blacklist unless new_resource.blacklist.nil?
+        hash
+      end
+    end
+  end
+end

libraries/provider_remote_win_event_log_source.rb

+require 'chef/provider/lwrp_base'
+require_relative 'provider_source'
+
+class Chef
+  class Provider
+    class SumoSourceRemoteWindowsEventLog < Chef::Provider::SumoSource
+      provides :sumo_source_remote_windows_event_log
+
+      def config_hash
+        hash = super
+        hash['source']['domain'] = new_resource.domain
+        hash['source']['username'] = new_resource.username
+        hash['source']['password'] = new_resource.password
+        hash['source']['hosts'] = new_resource.hosts
+        hash['source']['logNames'] = new_resource.log_names
+        hash
+      end
+    end
+  end
+end

libraries/provider_script_source.rb

+require 'chef/provider/lwrp_base'
+require_relative 'provider_source'
+
+class Chef
+  class Provider
+    class SumoSourceScript < Chef::Provider::SumoSource
+      provides :sumo_source_script
+
+      def config_hash
+        hash = super
+        hash['source']['commands'] = new_resource.commands
+        hash['source']['file'] = new_resource.file unless new_resource.file.nil?
+        hash['source']['workingDir'] = new_resource.working_dir unless new_resource.working_dir.nil?
+        hash['source']['timeout'] = new_resource.timeout unless new_resource.timeout.nil?
+        hash['source']['script'] = new_resource.script unless new_resource.script.nil?
+        hash['source']['cronExpression'] = new_resource.cron_expression
+        hash
+      end
+    end
+  end
+end

libraries/provider_source.rb

+require 'chef/platform/query_helpers'
+require 'chef/provider/lwrp_base'
+
+class Chef
+  class Provider
+    class SumoSource < Chef::Provider::LWRPBase
+      provides :sumo_source
+
+      use_inline_resources if defined?(use_inline_resources)
+
+      def whyrun_supported?
+        false
+      end
+
+      def load_current_resource
+        @current_resource = Chef::Resource::SumoSource.new(new_resource.name)
+      end
+
+      action :create do
+        converge_by("Create #{source_json_path}") do
+          file source_json_path do
+            content config_json
+            owner new_resource.owner
+            group new_resource.group
+            mode new_resource.mode
+            checksum new_resource.checksum
+            backup new_resource.backup
+            if Platform.windows?
+              inherits new_resource.inherits
+              rights new_resource.rights
+            end
+            sensitive !!(config_json.match(/password/i))
+          end
+        end
+      end
+
+      def api_version
+        'v1'
+      end
+
+      def config_hash
+        hash = {}
+        hash['api.version'] = api_version
+        hash['source'] = {}
+        hash['source']['sourceType'] = source_type_map[new_resource.source_type]
+        hash['source']['name'] = new_resource.name
+        hash['source']['description'] = new_resource.description unless new_resource.description.nil?
+        hash['source']['category'] = new_resource.category unless new_resource.category.nil?
+        hash['source']['hostName'] = new_resource.host_name unless new_resource.host_name.nil?
+        hash['source']['timeZone'] = new_resource.time_zone unless new_resource.time_zone.nil?
+        hash['source']['automaticDateParsing'] = new_resource.automatic_date_parsing unless new_resource.automatic_date_parsing.nil?
+        hash['source']['multilineProcessingEnabled'] = new_resource.multiline_processing_enabled unless new_resource.multiline_processing_enabled.nil?
+        hash['source']['useAutolineMatching'] = new_resource.use_autoline_matching unless new_resource.use_autoline_matching.nil?
+        hash['source']['manualPrefixRegexp'] = new_resource.manual_prefix_regexp unless new_resource.manual_prefix_regexp.nil?
+        hash['source']['forceTimeSoze'] = new_resource.force_time_zone unless new_resource.force_time_zone.nil?
+        hash['source']['defaultDateFormat'] = new_resource.default_date_format unless new_resource.default_date_format.nil?
+        hash['source']['filters'] = new_resource.filters unless new_resource.filters.nil?
+        hash['source']['alive'] = new_resource.alive unless new_resource.alive.nil?
+        hash
+      end
+
+      def config_json
+        JSON.pretty_generate(config_hash)
+      end
+
+      def source_json_path
+        "#{new_resource.source_json_directory}/#{new_resource.source_name}.json"
+      end
+
+      def source_type_map
+        {
+          :local_file => 'LocalFile',
+          :remote_file => 'RemoteFile',
+          :local_windows_event_log => 'LocalWindowsEventLog',
+          :remote_windows_event_log => 'RemoteWindowsEventLog',
+          :syslog => 'Syslog',
+          :script => 'Script',
+          :docker_stats => 'DockerStats',
+          :docker_log => 'DockerLog'
+        }
+      end
+    end
+  end
+end

libraries/provider_syslog_source.rb

+require 'chef/provider/lwrp_base'
+require_relative 'provider_source'
+
+class Chef
+  class Provider
+    class SumoSourceSyslog < Chef::Provider::SumoSource
+      provides :sumo_source_syslog
+
+      def config_hash
+        hash = super
+        hash['source']['protocol'] = new_resource.protocol unless new_resource.protocol.nil?
+        hash['source']['port'] = new_resource.port
+        hash
+      end
+    end
+  end
+end

libraries/resource_docker_source.rb

+require 'chef/resource/lwrp_base'
+require_relative 'resource_source'
+
+class Chef
+  class Resource
+    class SumoSourceDocker < Chef::Resource::SumoSource
+      attribute :source_type, kind_of: Symbol, required: true, equal_to: [:docker_stats, :docker_log]
+      attribute :uri, kind_of: String, required: true
+      attribute :specified_containers, kind_of: Array
+      attribute :all_containers, kind_of: [TrueClass, FalseClass], required: true
+      attribute :cert_path, kind_of: String
+      attribute :collect_events, kind_of: String
+    end
+  end
+end

libraries/resource_local_file_source.rb

+require 'chef/resource/lwrp_base'
+require_relative 'resource_source'
+
+class Chef