Attack Lab Automation with Terraform, Ansible, and vSphere
UPDATE (3/31/2023) - There is an updated post you can read here which goes over this in much more detail, with published Ansible, Terraform, and Packer code for reference.
Introduction
Vagrant, Ansible, Terraform, Packer, PowerCLI, API’s. The list of tools go on and on in regards to automating infrastructure locally, in the cloud, on ESXi/vSphere hosts, and more.
My labs typically live on ESXi hosts in my home lab and are managed with vSphere, but there are some fringe use cases I put in the cloud (typically AWS or DigitalOcean) or even a mix thereof. Making a template or an export of a virtual machine (VM) in vSphere is cool, but my requirements are constantly changing. Therefore a combination of VM templates and configuration management makes the most sense.
I’m content in regards to my automation approach in the cloud (DigitalOcean, AWS) with the use of Terraform and now leveraging Ansible for post-deployment tasks. Now, I need to shift that same approach to my home lab environment, which is way overdue.
But First - What’s in the Attack Lab?
I use the attack lab for a few different scenarios. The primary one is testing different reconnaissance and attack tools, payloads, and C2 frameworks. In some cases, I use the lab to familiarize myself with a piece of software or technology used in the enterprise where I can obtain a trial of that software and set it up to better understand it… or how to abuse it.
But, the attack labs aren’t only for offensive security practice and testing, I also collect logs of my activity so I can understand what noise or events I’m generating with each activity I perform, tool I use, or payload I execute to see how something would look from a SOC/Blue Team perspective. Essentially, looking for ways to reduce noise or find other ways to blend in; maybe LOLBAS?
For the most part, the lab is usually comprised of the following components:
- Active Directory server
- One or more Windows Servers (general purpose; file server, application server)
- One or more Windows 10 Endpoints
- Elastic Stack (SIEM to monitor Windows Security, Sysmon, and PowerShell logs and network traffic using Beats)
- Kali Linux
In general, I’m just trying to lay out a generic Windows-based network akin to a corporate environment.
DetectionLab by Chris Long
It’s also worth mentioning a project called Detection Lab created by Chris Long which uses a combination of Terraform, Ansible, Packer, and Vagrant to create, well, a detection lab to test attacks and defenses. It can be deployed locally using ESXi, VirtualBox, VMware Workstation/Fusion, Hyper-V and it can also be deployed in AWS or Azure. It’s very extensive and comes packed with a lot of detection features as well, such as Suricata, Zeek, and Splunk.
It’s a little more than what I currently need for my purposes, which is why I’m taking this milder approach that I can build off of over time. Nonetheless, you should check out Detection Lab!
Besides, I want to purpose build for my specific use cases as they grow and evolve over time. Also, learning by doing is always more fun and rewarding in my opinion.
Terraform + vSphere Provider = Infrastructure
Enter the vSphere Terraform Provider; while it’s not a provisioner, that’s totally fine! As I mentioned earlier, Ansible will be taking the role of post-deployment tasks. Leveraging templates within vSphere and deploying as-needed is the approach I’m after for the sake of repeatable lab infrastructure and the ability to programmatically develop different lab scenarios using Terraform.
Building the initial templates was straight-forward: I stood up virtual machines for Ubuntu Server 20.04, Windows Server 2016, and Windows 10 Pro. I ensured all of them were fully up-to-date and installed VMware Tools. Additionally, I disabled the firewalls, ensured that RDP was enabled on the Windows VM’s and that SSH was enabled on the Ubuntu VM.
For the Ubuntu VM, I added an SSH key specifically to allow authentication from my Ansible server so it can do it’s thing via SSH. Below is an example on generating a dedicated SSH key for this purpose and copying it to the Ubuntu Server VM.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
[itadmin@ansible ~]$ ssh-keygen -t rsa -b 4096
Generating public/private rsa key pair.
Enter file in which to save the key (/home/itadmin/.ssh/id_rsa): /home/itadmin/.ssh/ansible
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/itadmin/.ssh/ansible.
Your public key has been saved in /home/itadmin/.ssh/ansible.pub.
The key fingerprint is:
SHA256:kk4X/PFOIAno18ABaqS3TIPrJWZTQAKNcNmebdHQOaY [email protected]
The key's randomart image is:
+---[RSA 4096]----+
|B=oo.++= . |
|o=+.o +o*. |
|o *+ o *=.o |
| *.o+ E..+ + |
|.=o. o+ S . o |
|+ + o o o |
| . . . |
| |
| |
+----[SHA256]-----+
[itadmin@ansible ~]$ ssh-copy-id -i ~/.ssh/ansible [email protected]
For the Windows VM’s, I configured WinRM for Ansible using the ConfigureRemotingForAnsible PowerShell script in conjunction with the Terraform scripts for the Windows-related infrastructure by using the run_once_command_list option when I deploy the VM itself.
Remember, it’s important to note that using the example ConfigureRemotingForAnsible.ps1 script is a quick and easy way to get WinRM configured for use with Ansible and shouldn’t be used in production as it only implements Basic authentication and lacks any HTTPS encryption. Given this is a lab environment, it’s fine.
Once complete, I shut down the VM’s and convert them into templates using vSphere.
Below is an example Terraform script for deploying a Windows Server 2016 VM from a template.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
provider "vsphere" {
user = var.vsphere_user
password = var.vsphere_password
vsphere_server = var.vsphere_server
allow_unverified_ssl = true
}
data "vsphere_datacenter" "dc" {
name = "MGIOR Home"
}
data "vsphere_datastore" "datastore" {
name = "MGIORVMH02-Datastore"
datacenter_id = data.vsphere_datacenter.dc.id
}
data "vsphere_network" "network" {
name = "Lab Network"
datacenter_id = data.vsphere_datacenter.dc.id
}
data "vsphere_compute_cluster" "cluster" {
name = "CL-1"
datacenter_id = data.vsphere_datacenter.dc.id
}
data "vsphere_virtual_machine" "template" {
name = "Windows Server 2016"
datacenter_id = data.vsphere_datacenter.dc.id
}
resource "vsphere_virtual_machine" "lab_dc" {
name = var.vm_name
datastore_id = data.vsphere_datastore.datastore.id
resource_pool_id = data.vsphere_compute_cluster.cluster.resource_pool_id
num_cpus = 2
memory = 4096
guest_id = data.vsphere_virtual_machine.template.guest_id
scsi_type = data.vsphere_virtual_machine.template.scsi_type
firmware = "efi"
network_interface {
network_id = data.vsphere_network.network.id
adapter_type = data.vsphere_virtual_machine.template.network_interface_types[0]
}
disk {
label = "disk0"
size = data.vsphere_virtual_machine.template.disks.0.size
eagerly_scrub = data.vsphere_virtual_machine.template.disks.0.eagerly_scrub
thin_provisioned = data.vsphere_virtual_machine.template.disks.0.thin_provisioned
}
clone {
template_uuid = data.vsphere_virtual_machine.template.id
customize {
windows_options {
computer_name = "LABDC1"
admin_password = var.admin_password
auto_logon = true
auto_logon_count = 2
time_zone = 035
run_once_command_list = [
"powershell.exe [Net.ServicePointManager]::SecurityProtocol=[Net.SecurityProtocolType]::Tls12; Invoke-WebRequest -Uri https://raw.githubusercontent.com/ansible/ansible/devel/examples/scripts/ConfigureRemotingForAnsible.ps1 -Outfile C:\\WinRM_Ansible.ps1",
"powershell.exe -ExecutionPolicy Bypass -File C:\\WinRM_Ansible.ps1"
]
}
network_interface {
ipv4_address = var.windc_ipv4_addr
ipv4_netmask = 24
}
ipv4_gateway = "192.168.50.1"
dns_server_list = ["127.0.0.1","192.168.50.1"]
}
}
}
Given the example above, let’s go ahead and deploy a Windows Server 2016 VM using Terraform!
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
gio@slipgate:~/Devel/AttackLab-Lite/terraform$ terraform apply -var='vm_name=labdc1' -var='vsphere_server=<VSPHERE_IP_OR_FQDN>' -var='vsphere_user=<VSPHERE_USERNAME>' -var='vsphere_password=<VSPHERE_PASSWORD>' -var='windc_ipv4_addr=192.168.50.200'
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
+ create
Terraform will perform the following actions:
# vsphere_virtual_machine.lab_dc will be created
+ resource "vsphere_virtual_machine" "lab_dc" {
+ boot_retry_delay = 10000
+ change_version = (known after apply)
+ cpu_limit = -1
+ cpu_share_count = (known after apply)
+ cpu_share_level = "normal"
+ datastore_id = "datastore-10"
+ default_ip_address = (known after apply)
+ ept_rvi_mode = "automatic"
+ firmware = "efi"
+ force_power_off = true
+ guest_id = "windows9Server64Guest"
+ guest_ip_addresses = (known after apply)
+ hardware_version = (known after apply)
+ host_system_id = (known after apply)
+ hv_mode = "hvAuto"
+ id = (known after apply)
+ ide_controller_count = 2
+ imported = (known after apply)
+ latency_sensitivity = "normal"
+ memory = 4096
+ memory_limit = -1
+ memory_share_count = (known after apply)
+ memory_share_level = "normal"
+ migrate_wait_timeout = 30
+ moid = (known after apply)
+ name = "labdc1"
+ num_cores_per_socket = 1
+ num_cpus = 2
+ poweron_timeout = 300
+ reboot_required = (known after apply)
+ resource_pool_id = "resgroup-104"
+ run_tools_scripts_after_power_on = true
+ run_tools_scripts_after_resume = true
+ run_tools_scripts_before_guest_shutdown = true
+ run_tools_scripts_before_guest_standby = true
+ sata_controller_count = 0
+ scsi_bus_sharing = "noSharing"
+ scsi_controller_count = 1
+ scsi_type = "lsilogic-sas"
+ shutdown_wait_timeout = 3
+ storage_policy_id = (known after apply)
+ swap_placement_policy = "inherit"
+ uuid = (known after apply)
+ vapp_transport = (known after apply)
+ vmware_tools_status = (known after apply)
+ vmx_path = (known after apply)
+ wait_for_guest_ip_timeout = 0
+ wait_for_guest_net_routable = true
+ wait_for_guest_net_timeout = 5
+ clone {
+ template_uuid = "42275bb3-1a58-0f17-df9d-16877e869c8e"
+ timeout = 30
+ customize {
+ dns_server_list = [
+ "127.0.0.1",
+ "192.168.50.1",
]
+ ipv4_gateway = "192.168.50.1"
+ timeout = 10
+ network_interface {
+ ipv4_address = "192.168.50.200"
+ ipv4_netmask = 24
}
+ windows_options {
+ admin_password = (sensitive value)
+ auto_logon = true
+ auto_logon_count = 2
+ computer_name = "LABDC1"
+ full_name = "Administrator"
+ organization_name = "Managed by Terraform"
+ run_once_command_list = [
+ "powershell.exe [Net.ServicePointManager]::SecurityProtocol=[Net.SecurityProtocolType]::Tls12; Invoke-WebRequest -Uri https://raw.githubusercontent.com/ansible/ansible/devel/examples/scripts/ConfigureRemotingForAnsible.ps1 -Outfile C:\\WinRM_Ansible.ps1",
+ "powershell.exe -ExecutionPolicy Bypass -File C:\\WinRM_Ansible.ps1",
]
+ time_zone = 35
}
}
}
+ disk {
+ attach = false
+ controller_type = "scsi"
+ datastore_id = "<computed>"
+ device_address = (known after apply)
+ disk_mode = "persistent"
+ disk_sharing = "sharingNone"
+ eagerly_scrub = false
+ io_limit = -1
+ io_reservation = 0
+ io_share_count = 0
+ io_share_level = "normal"
+ keep_on_remove = false
+ key = 0
+ label = "disk0"
+ path = (known after apply)
+ size = 40
+ storage_policy_id = (known after apply)
+ thin_provisioned = false
+ unit_number = 0
+ uuid = (known after apply)
+ write_through = false
}
+ network_interface {
+ adapter_type = "e1000e"
+ bandwidth_limit = -1
+ bandwidth_reservation = 0
+ bandwidth_share_count = (known after apply)
+ bandwidth_share_level = "normal"
+ device_address = (known after apply)
+ key = (known after apply)
+ mac_address = (known after apply)
+ network_id = "network-12"
}
}
Plan: 1 to add, 0 to change, 0 to destroy.
Do you want to perform these actions?
Terraform will perform the actions described above.
Only 'yes' will be accepted to approve.
Enter a value: yes
vsphere_virtual_machine.lab_dc: Creating...
vsphere_virtual_machine.lab_dc: Still creating... [10s elapsed]
vsphere_virtual_machine.lab_dc: Still creating... [20s elapsed]
vsphere_virtual_machine.lab_dc: Still creating... [30s elapsed]
...
...
...
vsphere_virtual_machine.lab_dc: Still creating... [9m50s elapsed]
vsphere_virtual_machine.lab_dc: Creation complete after 9m52s [id=4227ff27-0f25-e429-7f2a-402eb9d7894d]
Apply complete! Resources: 1 added, 0 changed, 0 destroyed.
Thanks, Terraform!
The Terraform script used above, as well as the others for Ubuntu Server and Windows 10, are available in my GitHub repository as reference to the attack lab being described in this post. Feel free to use it and/or modify to your needs, or to better understand what the hell is going on here!
Leveraging Ansible for Post-Deployment Tasks
Now that the infrastructure I want is up and running thanks to Terraform, I’ll leverage Ansible to perform some post-deployment tasks. I understand I can use the run_once_command_list option with Terraform to download and execute different PowerShell scripts or commands, but the point is to build and maintain playbooks using Ansible for different lab scenarios, so I’m trying to keep the Terraform aspect as light as possible; just executing the PowerShell script for prepping WinRM in this case.
For the Windows VM’s, I’ve configured their playbooks to install Chocolatey. Unlike Linux, Windows doesn’t have a built-in package manager per se, and to install software such as the different Beats agents or any generic software I want to install on the Windows-based VM’s, this makes that much easier. Take a look at the library of packages Chocolatey offers. Always use caution if you’re looking to leverage Chocolatey in production. Some packages offered are from the software vendors themselves, whereas others may be community-provided. Each package does show AV scan results, so there’s that. Just be careful!
So, what am I doing playbook-wise anyways?
Whatever you say, Gio.
Windows Serverr 2016 (Active Directory)
- Configure as a Domain Controller
- Install and configure Winlogbeat, Filebeat, Packetbeat
- Install and configure Sysmon
- Create GPO’s for PowerShell/audit logging, Sysmon deployment
- Leverage Vulnerable-AD by WazeHell to generate users, SPN’s, and prepare AD for different attack scenarios
Windows 10 (Endpoint)
- Join to attack lab domain
- Install and configure Winlogbeat, Filebeat, Packetbeat
- Use Chocolatey to install additional software
- Ensure Sysmon is installed and active
Ubuntu Server (Elastic Stack)
- Automating the installation and configuration of Elastic Stack components (Elasticsearch, Logstash, Kibana)
- Deploying Logstash configuration files
- Configuring and enabling X-Pack (Elastic Stack security)
I’ve posted the Ansible playbooks for these basic post-deployment activities outlined above are available in my GitHub repository for reference - feel free to use it and/or modify to your needs.
Side Note for using Ansible with Windows (WinRM)
Below is a high-level overview on how I install Ansible on a CentOS 7 VM. There are different approaches to install Ansible such as using pip or Miniconda (“virtual” environment) - however, I just went with the standard yum installation approach:
1
2
3
4
5
6
7
8
9
[itadmin@ansible ~]$ sudo yum install epel-release
[itadmin@ansible ~]$ sudo yum install ansible
[itadmin@ansible ~]$ ansible-playbook --version
ansible-playbook 2.9.25
config file = /etc/ansible/ansible.cfg
configured module search path = [u'/home/itadmin/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules']
ansible python module location = /usr/lib/python2.7/site-packages/ansible
executable location = /usr/bin/ansible-playbook
python version = 2.7.5 (default, Nov 16 2020, 22:23:17) [GCC 4.8.5 20150623 (Red Hat 4.8.5-44)]
With a default Ansible installation on a CentOS 7 server, it’s important to note there are a few dependencies required to enable Ansible to speak the WinRM dialect in order to communicate with the Windows hosts. Note above how the Python version being leveraged by Ansible is part of the 2.x branch. We’ll need to install pip2 to get the pywinrm library and relevant dependencies installed.
1
2
3
4
5
6
7
8
9
10
11
12
13
[itadmin@ansible ~]$ ansible windows -m win_ping
192.168.1.200 | FAILED! => {
"msg": "winrm or requests is not installed: No module named winrm"
}
[itadmin@ansible ~]$ sudo yum install python-pip2
[itadmin@ansible ~]$ pip -V
pip 8.1.2 from /usr/lib/python2.7/site-packages (python 2.7)
[itadmin@ansible ~]$ sudo pip install pywinrm
[itadmin@ansible ~]$ ansible windows -m win_ping
192.168.1.200 | SUCCESS => {
"changed": false,
"ping": "pong"
}
With WinRM and Ansible now talking to each other, and now that I have this Windows Server 2016 VM sitting around waiting, I’m going to leverage an Ansible playbook to configure it as a Domain Controller. This playbook will also install and configure the other items mentioned above for my Domain Controller VM setup.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
[itadmin@ansible01 lab-dc]$ ansible-playbook labdc-playbook.yml
PLAY [windows-lab-dc] *************************************************************************************************************************************************************
TASK [Gathering Facts] *****************************************************************************************************************************************************
ok: [192.168.50.200]
TASK [Install Chocolatey for Package Management] ***************************************************************************************************************************
[WARNING]: Chocolatey was missing from this system, so it was installed during this task run.
changed: [192.168.50.200]
TASK [Disable IPv6] ********************************************************************************************************************************************************
changed: [192.168.50.200]
TASK [Install bginfo] ******************************************************************************************************************************************************
changed: [192.168.50.200]
TASK [Install Beats (Winlogbeat, Filebeat, Packetbeat - 7.15.0)] ***********************************************************************************************************
changed: [192.168.50.200]
TASK [Copy Winlogbeat Config File] *****************************************************************************************************************************************
changed: [192.168.50.200]
TASK [Install Sysmon] ******************************************************************************************************************************************************
changed: [192.168.50.200]
TASK [Install Active Directory Domain Services (ADDS)] *********************************************************************************************************************
changed: [192.168.50.200]
TASK [Create new Domain in a new Forest (AD Lab)] **************************************************************************************************************************
changed: [192.168.50.200]
TASK [Restart Server] ******************************************************************************************************************************************************
changed: [192.168.50.200]
TASK [Waiting for reconnect...] ********************************************************************************************************************************************
ok: [192.168.50.200]
TASK [Create Temp folder in C:\] *******************************************************************************************************************************************
changed: [192.168.50.200]
TASK [Copy VulnAD Script] **************************************************************************************************************************************************
changed: [192.168.50.200]
TASK [Run Invoke-VulnAD (100 Users)] ***************************************************************************************************************************************
changed: [192.168.50.200]
TASK [Setup Winlogbeat Kibana Dashboards] **********************************************************************************************************************************
changed: [192.168.50.200]
PLAY RECAP *****************************************************************************************************************************************************************
192.168.50.201 : ok=14 changed=13 unreachable=0 failed=0 skipped=0 rescued=0 ignored=0
The same, for the most part applies to the other VM’s in this attack lab, but in this post I just went over the deployment of the Windows Server 2016 VM for use as a Domain Controller.
Retrospective & ToDo’s
As with anything, I like to do a retrospective to look for things I can improve upon or change entirely. Right now, these VM’s are connected to a “Lab Network” vSwitch which is connected to a separate physical network used solely for lab work, meaning, it’s largely isolated from my internal and home server networks.
Typically, I like to use pfSense to make a more isolated network and leverage a “jump host” to access that it.
With that, here’s the retrospective & TODO’s:
- Modify Elastic Stack Ansible playbook and Terraform code to deploy and configure a multi-node cluster with X-Pack configured for HTTPS communication to take advantage of Fleet Server as well as Elastic Agent’s rather than individual Beats agents
- Terraform a pfSense deployment, jump host VM, and place attack lab infrastructure behind pfSense
- Supplemental blog posts on AD enumeration/attacks, lateral movement, and use of relevant tools for these tasks Attacking the Attack Lab - Part 1
- Explore Packer more in-depth to see if that can save more time with the initial template creation; i.e. “Gold Image” using infrastructure as code