Setting up ProxMOX

Hardware:

  • Set your device to auto power on in BIOS
  • Set Intel® Virtualization Technology for Directed I/O (VT-d) to Enabled if allowed
  • Set other applicable BIOS options for Fan, Cooling, Performance
  • Set BIOS to UEFI
  • Don’t setup any RAID

Proxmox USB Install:

  • If one disk set RAID 0, ZFS, Compression On
  • If two disks set RAID 1, ZFS, Compression On
  • Leave all the other disks alone
  • Be sure to check your Hostname as this is not easy to change later
  • Set your IP Address to something static.

Initial Configuration – Setting Timesync, Setting APT Sources and Update Packages

  • Navigate to your ProxMOX Ipaddress – https://ipaddressofproxmox:8006
  • Logon as ROOT
  • Click on your ProxMOX Node Name
  • Click Shell
  • At the prompt type nano /etc/systemd/timesyncd.conf
  • In the editor under [time]
  • Remove the # next to NTP= and then set your NTP server eg NTP=au.pool.ntp.org
  • Hit CTRL-O to Save then CTRL-X to Exit Editor
  • Now, type nano /etc/apt/sources.list.d/pve-enterprise.list
  • Add a # in front of the deb to cancel out the Enterprise Update Source
  • CTRL-O to Save then CTRL-X to Exit Editor
  • Now, type nano /etc/apt/sources.list
  • On a new line at the bottom add the “No Subscription” Update Source

    deb http://download.proxmox.com/debian/pve buster pve-no-subscription
  • CTRL-O to Save then CTRL-X to Exit Editor
  • Issue the reboot command and wait to restart.

Perform all updates

  • Navigate back to the ProxMox Web Portal
  • Click on Updates, then the Refresh Button
  • Wait a few seconds, then close the web dialog box.
  • Now Click the Upgrade Button, this will take a bit, but once done, reboot the box again. You are now ready to do some other stuff.

Configure Networking

By Default Proxmox creates a Network Bridge of vmbr0 which is untagged to your main Network Interface (eno1). You may want to create other bridges that are tagged to VLANs. Especially if you are creating experimental systems, servers etc

To create another bridge do the following:

  • Before we get started, you will have to install a special script called ifupdown2 without it, high level configuration in the ProxMOX Management Portal is much more difficult.
    Go back to the Shell for your Node and enter the following command:
    apt install ifupdown2
    Follow the prompts, then…
  • Under System for your Node, Click Networking
  • Click Create and Select Linux Bridge
  • No need to set an IP Address on it, unless you want stuff on your experiemental network to access the ProxMox Management Portal
  • In Bridge Ports field, enter our network interface followed by a ‘.’ and a vlan number.. so if you interface is eno1 and you want your experimental networking to be on VLAN 3, then enter eno1.3

    The secret here, is to do the same thing on all the ProxMOX nodes you are setting up, so they can talk to each other across your switch.

    Also, make sure your switches can carry VLAN Traffic. Nearly all of them do these days so probably nothing to worry about.

    Don’t worry about VLAN Aware.. There are many ways to offer solutions, but this creates a very simple bridge where you don’t have to then create special VLAN settings on your Guest VM’s.
  • Click Apply Configuration. The ifupdown2 script will now run and make the changes to your system

Make your host Power Efficient

By default, ProxMOX, will run your processor TDP at full power to get maximum performance. Most of time your VM’s will sit idle, so you don’t want the constant power draw. Instead you can use use cpufreq to change your CPU to powersave. This will lower the processor power, but when there is demand increase speed to deal with compute load.

At the shell for your node, type crontab –e
Pick Nano as your preferred Editor if asked

Add the following line to the file:
@reboot echo “powersave” | tee /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor

Save, then reboot.

Pass PCI and PCIe Devices directly to your Guest VMs

This allows your to access your hardware directly by the VM. Useful if you access things like Wireless Cards for Virtual Routers etc. Remember, usually, you have to keep your VM local to the ProxMOX node for it to work. Some nodes with exact identical hardware may be the exception to this rule.

There is a guide here, but it can be confusing: PCI(e) Passthrough – Proxmox VE

Instead, I give you the step by step here. This is to use Intel VT-d which is the more popular option these days for most home/small business setups.

  • Goto the shell for your ProxMox node
  • Run the following command nano /etc/modules
  • Add the following lines to the file:

 vfio
 vfio_iommu_type1
 vfio_pci
 vfio_virqfd
  • Save the file and exit
  • Now we refresh the initramfs to ensure these modules are loaded at boot.. run the command update-initramfs -u -k all
  • Next we have to update the Kernel Commandline to enable Intel IOMMU
  • If you setup ProxMOX as described above, that is UEFI Boot with a ZFS file system, then your ProxMOX will be using SYSTEMD to Boot.
  • type nano /etc/kernel/cmdline
  • add the following to the end of the line intel_iommu=on iommu=pt
    Example:
    root=ZFS=rpool/ROOT/pve-1 boot=zfs intel_iommu=on iommu=pt
  • Save the file and exit
  • Now type the following: proxmox-boot-tool refresh this will update the Kernel Command line.
  • Reboot