Commit 1b4ba504 authored by Duc Ha's avatar Duc Ha
Browse files

Merge pull request #64 from zl4bv/source

Add LWRPs for source config files
parents e6c362ae 03a6dea7
......@@ -69,3 +69,19 @@ suites:
- centos-7.1
- debian-8.1
- ubuntu-14.04
- name: source-resource
run_list:
- recipe[source-resource]
- recipe[source-resource::docker_create]
- recipe[source-resource::local_file_create]
- recipe[source-resource::local_win_event_log_create]
- recipe[source-resource::remote_file_create]
- recipe[source-resource::remote_win_event_log_create]
- recipe[source-resource::script_create]
- recipe[source-resource::syslog_create]
attributes:
SUMO_ACCESS_ID: <%= ENV['SUMO_ACCESS_ID'] %>
SUMO_ACCESS_KEY: <%= ENV['SUMO_ACCESS_KEY'] %>
excludes:
- windows-2008r2
- windows-2012r2
......@@ -5,4 +5,5 @@ metadata
group :integration do
cookbook 'data-bag-faker', path: 'test/fixtures/cookbooks/data-bag-faker'
cookbook 'installer-resource', path: 'test/fixtures/cookbooks/installer-resource'
cookbook 'source-resource', path: 'test/fixtures/cookbooks/source-resource'
end
......@@ -24,20 +24,24 @@ Note
Starting from 19.107, there are 2 major extensions to SumoLogic collectors:
* You can configure a collector's parameters from a set of json files under a common folder. Each of the json file will represent a source on that collector. Updates made to a json file will then be reflected on its corresponding source. Note that the format of this kind of file is **slightly different** from that of the traditional single json file (sumo.json) and they are **not** compatible. You also need to use the parameter `syncSources` instead of `sources` inside `sumo.conf`. See more details [here](https://service.sumologic.com/help/Default.htm#Using_sumo.conf.htm).
* You can change a collector's existing parameters through local configuration json file(s) continuously. Before this, using collector API was the only option. More information about this is [here](https://service.sumologic.com/help/Default.htm#Using_Local_Configuration_File_Management.htm)
Installation
------------
1. Create an [Access Key](http://help.sumologic.com/i19.69v2/Default.htm#Generating_Collector_Installation_API_Keys.htm)
2. Install the cookbook in your Chef repo (your knife version should be at least 11.10.4 and you should have the [knife github plugin](https://github.com/websterclay/knife-github-cookbooks) installed):
```knife cookbook github install SumoLogic/sumologic-collector-chef-cookbook```
```
knife cookbook github install SumoLogic/sumologic-collector-chef-cookbook
```
3. Specify data bag and item with your access credentials. The data item should
contain attributes `accessID` and `accessKey`. Note that attribute names are case sensitive. If the cases mismatch, the values will not appear when chef-client runs. The default data bag/item is
`['sumo-creds']['api-creds']`
4. (Optional) Decide if you want to use the Local Configuration Management feature by setting the attribute `default['sumologic']['local_management']` properly. By default this feature is on, to leverage the power of Chef.
4. (Optional) Decide if you want to use the Local Configuration Management feature by setting the attribute `default['sumologic']['local_management']` properly. By default this feature is on, to leverage the power of Chef.
5. (Optional) Select the json configuration option (i.e. through a single file or a folder) by setting the attribute `default['sumologic']['use_json_path_dir']` appropriately. By default a single json file is used.
6. (Optional) Check if the path to the json file or the json folder is set correctly in the attribute `default['sumologic']['sumo_json_path']`. By default this is the path to the json file at `/etc/sumo.json` on Linux or `c:\sumo\sumo.json` on Windows.
6. (Optional) Check if the path to the json file or the json folder is set correctly in the attribute `default['sumologic']['sumo_json_path']`. By default this is the path to the json file at `/etc/sumo.json` on Linux or `c:\sumo\sumo.json` on Windows.
7. Upload the cookbook to your Chef Server:
```knife cookbook upload sumologic-collector```
```
knife cookbook upload sumologic-collector
```
8. Add the `sumologic-collector` receipe to your node run lists. This step depends
on your node configuration, so specifics will not be described in this README.md.
......@@ -78,6 +82,319 @@ Attributes
</tr>
</table>
Resource/Provider
-----------------
sumologic_collector_installer
---------
Allows for additional customisation of the Sumo Logic Collector installer
### Actions
`default` = `:install`
- `:install` - installs the Sumo Logic Collector if it is not already installed
### Attribute Parameters
See the [Sumo Logic documentation](https://service.sumologic.com/help/Default.htm#Using_Quiet_Mode_to_Install_a_Collector.htm)
for more information about these attributes.
- `dir` - Directory where the Collector will be installed
- `source` - URL where installer will be downloaded from
- `collector_name`
- `collector_url`
- `sumo_email`
- `sumo_password`
- `sumo_token_and_url`
- `sumo_access_id`
- `sumo_access_key`
- `proxy_host`
- `proxy_port`
- `proxy_user`
- `proxy_password`
- `proxy_ntlmdomain`
- `sources`
- `sync_sources`
- `ephemeral`
- `clobber`
- `runas_username`
- `winrunas_password`
- `skip_registration`
### Examples
```ruby
# Installs the Collector on Windows and skips registration
sumologic_collector_installer 'c:\sumo' do
source 'https://collectors.sumologic.com/rest/download/win64'
sumo_access_id node['SUMO_ACCESS_ID']
sumo_access_key node['SUMO_ACCESS_KEY']
skip_registration true
end
```
Collector Sources
---------
### Attribute Parameters
The following attributes are common to all of the sources listed below.
See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
for more information about these attributes.
- `owner` - owner of the JSON Source configuration file
- `group` - group of the JSON Source configuration file
- `mode` - file mode of the JSON Source configuration file
- `source_name` - name of the source. **required**
- `source_json_directory` - directory where JSON Source configuration file will be stored. **required**
- `description`
- `category`
- `host_name`
- `time_zone`
- `automatic_date_parsing`
- `multiline_processing_enabled`
- `use_autoline_matching`
- `manual_prefix_regexp`
- `force_time_zone`
- `default_date_format`
- `filters`
- `alive`
sumo_source_docker
---------
### Actions
`default` = `:create`
- `:create` - creates a JSON Source configuration
### Attribute Parameters
See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
for more information about these attributes.
The following attribute parameters are in addition to the generic parameters
listed above.
- `uri`
- `specified_containers`
- `all_containers`
- `cert_path`
- `source_type` - one of `:docker_stats`, `:docker_log`. **required**
- `collected_events`
### Examples
```ruby
sumo_source_docker 'docker_stats' do
source_json_directory node['sumologic']['sumo_json_path']
source_type :docker_stats
uri 'https://127.0.0.1:2376'
all_containers true
end
sumo_source_docker 'docker_log' do
source_json_directory node['sumologic']['sumo_json_path']
source_type :docker_log
uri 'https://127.0.0.1:2376'
all_containers true
end
```
sumo_source_local_file
---------
### Actions
`default` = `:create`
- `:create` - creates a JSON Source configuration
### Attribute Parameters
See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
for more information about these attributes.
The following attribute parameters are in addition to the generic parameters
listed above.
- `path_expression` - **required**
- `blacklist`
- `encoding`
### Examples
```ruby
sumo_source_local_file 'local_file' do
source_json_directory node['sumologic']['sumo_json_path']
path_expression '/tmp/example'
end
```
sumo_source_local_windows_event_log
---------
### Actions
`default` = `:create`
- `:create` - creates a JSON Source configuration
### Attribute Parameters
See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
for more information about these attributes.
The following attribute parameters are in addition to the generic parameters
listed above.
- `log_names` - **required**
### Examples
```ruby
sumo_source_local_windows_event_log 'local_win_event_log' do
source_json_directory node['sumologic']['sumo_json_path']
log_names ['security', 'application']
end
```
sumo_source_remote_file
---------
### Actions
`default` = `:create`
- `:create` - creates a JSON Source configuration
### Attribute Parameters
See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
for more information about these attributes.
The following attribute parameters are in addition to the generic parameters
listed above.
- `remote_hosts` - **required**
- `remote_port` - **required**
- `remote_user` - **required**
- `remote_password` - **required**
- `key_path` - **required**
- `key_password`
- `path_expression` - **required**
- `auth_method` - one of `key` or `password`.
- `blacklist`
### Examples
```ruby
sumo_source_remote_file 'remote_file' do
source_json_directory node['sumologic']['sumo_json_path']
remote_hosts ['127.0.0.1']
remote_port 22
remote_user 'user'
remote_password 'password'
key_path ''
path_expression '/tmp/example'
auth_method 'password'
end
```
sumo_source_remote_windows_event_log
---------
### Actions
`default` = `:create`
- `:create` - creates a JSON Source configuration
### Attribute Parameters
See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
for more information about these attributes.
The following attribute parameters are in addition to the generic parameters
listed above.
- `domain` - **required**
- `username` - **required**
- `password` - **required**
- `hosts` - **required**
- `log_names` - **required**
### Examples
```ruby
sumo_source_remote_windows_event_log 'remote_win_event_log' do
source_json_directory node['sumologic']['sumo_json_path']
domain 'mydomain'
username 'user'
password 'password'
hosts ['myremotehost1']
log_names ['security', 'application']
end
```
sumo_source_script
---------
### Actions
`default` = `:create`
- `:create` - creates a JSON Source configuration
### Attribute Parameters
See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
for more information about these attributes.
The following attribute parameters are in addition to the generic parameters
listed above.
- `commands` - **required**
- `file`
- `working_dir`
- `timeout`
- `script`
- `cron_expression`
### Examples
```ruby
sumo_source_script 'script' do
source_json_directory node['sumologic']['sumo_json_path']
commands ['/bin/bash']
cron_expression '0 * * * *'
end
```
sumo_source_syslog
---------
### Actions
`default` = `:create`
- `:create` - creates a JSON Source configuration
### Attribute Parameters
See the [Sumo Logic documentation](https://service.sumologic.com/help/#Using_JSON_to_configure_Sources.htm)
for more information about these attributes.
The following attribute parameters are in addition to the generic parameters
listed above.
- `protocol`
- `port`
### Examples
```ruby
sumo_source_syslog 'syslog' do
source_json_directory node['sumologic']['sumo_json_path']
end
```
Contributing
------------
This cookbook is meant to help customers use Chef to install Sumo Logic
......
require 'chef/provider/lwrp_base'
require_relative 'provider_source'
class Chef
class Provider
class SumoSourceDocker < Chef::Provider::SumoSource
provides :sumo_source_docker
def config_hash
hash = super
hash['source']['uri'] = new_resource.uri
hash['source']['specifiedContainers'] = new_resource.specified_containers unless new_resource.specified_containers.nil?
hash['source']['allContainers'] = new_resource.all_containers
hash['source']['certPath'] = new_resource.cert_path unless new_resource.cert_path.nil?
hash['source']['sourceType'] = source_type_map[new_resource.source_type]
hash['source']['collectEvents'] = new_resource.collect_events unless new_resource.collect_events.nil?
hash
end
end
end
end
require 'chef/provider/lwrp_base'
require_relative 'provider_source'
class Chef
class Provider
class SumoSourceLocalFile < Chef::Provider::SumoSource
provides :sumo_source_local_file
def config_hash
hash = super
hash['source']['pathExpression'] = new_resource.path_expression unless new_resource.path_expression.nil?
hash['source']['blacklist'] = new_resource.blacklist unless new_resource.blacklist.nil?
hash['source']['encoding'] = new_resource.encoding unless new_resource.encoding.nil?
hash
end
end
end
end
require 'chef/provider/lwrp_base'
require_relative 'provider_source'
class Chef
class Provider
class SumoSourceLocalWindowsEventLog < Chef::Provider::SumoSource
provides :sumo_source_local_windows_event_log
def config_hash
hash = super
hash['source']['logNames'] = new_resource.log_names
hash
end
end
end
end
require 'chef/provider/lwrp_base'
require_relative 'provider_source'
class Chef
class Provider
class SumoSourceRemoteFile < Chef::Provider::SumoSource
provides :sumo_source_remote_file
def config_hash
hash = super
hash['source']['remoteHosts'] = new_resource.remote_hosts
hash['source']['remotePort'] = new_resource.remote_port
hash['source']['remoteUser'] = new_resource.remote_user
hash['source']['remotePassword'] = new_resource.remote_password
hash['source']['keyPath'] = new_resource.key_path
hash['source']['keyPassword'] = new_resource.key_password
hash['source']['pathExpression'] = new_resource.path_expression unless new_resource.path_expression.nil?
hash['source']['authMethod'] = new_resource.auth_method
hash['source']['blacklist'] = new_resource.blacklist unless new_resource.blacklist.nil?
hash
end
end
end
end
require 'chef/provider/lwrp_base'
require_relative 'provider_source'
class Chef
class Provider
class SumoSourceRemoteWindowsEventLog < Chef::Provider::SumoSource
provides :sumo_source_remote_windows_event_log
def config_hash
hash = super
hash['source']['domain'] = new_resource.domain
hash['source']['username'] = new_resource.username
hash['source']['password'] = new_resource.password
hash['source']['hosts'] = new_resource.hosts
hash['source']['logNames'] = new_resource.log_names
hash
end
end
end
end
require 'chef/provider/lwrp_base'
require_relative 'provider_source'
class Chef
class Provider
class SumoSourceScript < Chef::Provider::SumoSource
provides :sumo_source_script
def config_hash
hash = super
hash['source']['commands'] = new_resource.commands
hash['source']['file'] = new_resource.file unless new_resource.file.nil?
hash['source']['workingDir'] = new_resource.working_dir unless new_resource.working_dir.nil?
hash['source']['timeout'] = new_resource.timeout unless new_resource.timeout.nil?
hash['source']['script'] = new_resource.script unless new_resource.script.nil?
hash['source']['cronExpression'] = new_resource.cron_expression
hash
end
end
end
end
require 'chef/platform/query_helpers'
require 'chef/provider/lwrp_base'
class Chef
class Provider
class SumoSource < Chef::Provider::LWRPBase
provides :sumo_source
use_inline_resources if defined?(use_inline_resources)
def whyrun_supported?
false
end
def load_current_resource
@current_resource = Chef::Resource::SumoSource.new(new_resource.name)
end
action :create do
converge_by("Create #{source_json_path}") do
file source_json_path do
content config_json
owner new_resource.owner
group new_resource.group
mode new_resource.mode
checksum new_resource.checksum
backup new_resource.backup
if Platform.windows?
inherits new_resource.inherits
rights new_resource.rights
end
sensitive !!(config_json.match(/password/i))
end
end
end
def api_version
'v1'
end
def config_hash
hash = {}
hash['api.version'] = api_version
hash['source'] = {}
hash['source']['sourceType'] = source_type_map[new_resource.source_type]
hash['source']['name'] = new_resource.name
hash['source']['description'] = new_resource.description unless new_resource.description.nil?
hash['source']['category'] = new_resource.category unless new_resource.category.nil?
hash['source']['hostName'] = new_resource.host_name unless new_resource.host_name.nil?
hash['source']['timeZone'] = new_resource.time_zone unless new_resource.time_zone.nil?
hash['source']['automaticDateParsing'] = new_resource.automatic_date_parsing unless new_resource.automatic_date_parsing.nil?
hash['source']['multilineProcessingEnabled'] = new_resource.multiline_processing_enabled unless new_resource.multiline_processing_enabled.nil?
hash['source']['useAutolineMatching'] = new_resource.use_autoline_matching unless new_resource.use_autoline_matching.nil?
hash['source']['manualPrefixRegexp'] = new_resource.manual_prefix_regexp unless new_resource.manual_prefix_regexp.nil?
hash['source']['forceTimeSoze'] = new_resource.force_time_zone unless new_resource.force_time_zone.nil?
hash['source']['defaultDateFormat'] = new_resource.default_date_format unless new_resource.default_date_format.nil?
hash['source']['filters'] = new_resource.filters unless new_resource.filters.nil?
hash['source']['alive'] = new_resource.alive unless new_resource.alive.nil?
hash
end
def config_json
JSON.pretty_generate(config_hash)
end
def source_json_path
"#{new_resource.source_json_directory}/#{new_resource.source_name}.json"
end
def source_type_map
{
:local_file => 'LocalFile',
:remote_file => 'RemoteFile',
:local_windows_event_log => 'LocalWindowsEventLog',
:remote_windows_event_log => 'RemoteWindowsEventLog',
:syslog => 'Syslog',
:script => 'Script',
:docker_stats => 'DockerStats',
:docker_log => 'DockerLog'
}
end
end
end
end
require 'chef/provider/lwrp_base'
require_relative 'provider_source'
class Chef
class Provider
class SumoSourceSyslog < Chef::Provider::SumoSource
provides :sumo_source_syslog
def config_hash
hash = super
hash['source']['protocol'] = new_resource.protocol unless new_resource.protocol.nil?
hash['source']['port'] = new_resource.port
hash
end
end
end
end
require 'chef/resource/lwrp_base'
require_relative 'resource_source'
class Chef
class Resource
class SumoSourceDocker < Chef::Resource::SumoSource
attribute :source_type, kind_of: Symbol, required: true, equal_to: [:docker_stats, :docker_log]
attribute :uri, kind_of: String, required: true
attribute :specified_containers, kind_of: Array
attribute :all_containers, kind_of: [TrueClass, FalseClass], required: true
attribute :cert_path, kind_of: String
attribute :collect_events, kind_of: String
end
end
end
require 'chef/resource/lwrp_base'
require_relative 'resource_source'
class Chef