I'm working with a Splunk Enterprise cluster deployed with the splunk-enterprise Helm Chart. I'm trying to install Amazon's CloudWatch Agent onto my Splunk pods, to send Splunk application logs to CloudWatch.
I decided to try to do this by defining Ansible pre tasks and setting them in my Helm values.yaml, for example:
clusterManager:
defaults:
ansible_pre_tasks:
- 'file:///mnt/playbooks/install_cloudwatch_agent.yml'
I got my pre tasks working, but they're failing.
At first I tried to install CloudWatch Agent from yum, but this failed because the Python dnf module was missing. Actually, it looks like yum and dnf aren't installed at all in the splunk Docker image.
Then I tried to just download the RPM and install that, but this failed because I didn't have permission to get a transaction lock with rpm. I tried to solve the permissions issue by setting become_user "{{ privileged_user }}" on my task, but this didn't work either, nor could I become root.
Are splunk-ansible pre tasks and post tasks an appropriate way to install additional supporting services onto the Splunk Enterprise pods like this? If so, are there any examples showing how to do it? If not, is there some other approach that would be a better fit?