Most people seem to use qemu together with libvirt. I saw two drawbacks for my limited use-case: a) on Gentoo, libvirt rakes in tons of dependencies when emerged, b) I would like to have a low-level understanding of how to configure qemu. So I went ahead and the result is this:
#!/bin/bash
qemu-system-x86_64 \
-m 32g \
-machine type=q35 \
-display vnc=:1 \
-accel kvm \
-object iothread,id=iothread0 \
-device ich9-ahci,id=ahci \
-device virtio-blk-pci,iothread=iothread0,drive=virtblk1 \
-device virtio-blk-pci,iothread=iothread0,drive=virtblk2 \
-blockdev driver=qcow2,node-name=virtblk1,file.driver=file,file.filename=qemu3-disk1.qcow2 \
-blockdev driver=qcow2,node-name=virtblk2,file.driver=file,file.filename=qemu3-disk2.qcow2 \
-drive if=pflash,format=raw,readonly=on,file=OVMF_CODE.fd \
-drive if=pflash,format=raw,readonly=on,file=OVMF_VARS.fd \
-cpu host,vmx=off \
-smp 6,sockets=2,cores=3,threads=1,maxcpus=6 \
-object memory-backend-ram,size=16g,id=m0 \
-object memory-backend-ram,size=16g,id=m1 \
-numa node,nodeid=0,memdev=m0 \
-numa node,nodeid=1,memdev=m1 \
-numa cpu,node-id=0,socket-id=0 \
-numa cpu,node-id=1,socket-id=1 \
-netdev tap,id=net0,vhost=on,ifname=tap0,script=../tap-up.sh,downscript=../tap-down.sh \
-device virtio-net-pci,packed=on,netdev=net0,mac=08:00:27:65:98:33 \
-monitor tcp:127.0.0.1:55555,server,nowait
I use "modern" options instead of "older", possibly simpler, but deprecated options as widely as possible. My host machine is a NUMA server with two CPU nodes with 4 cores each and memory allocated to each of them. I mirror this setup on the guest side, however only with 3 cores per node. For the block devices, I use virtio-blk. The guest has an EFI bios and is connected via a bridged interface and virtio-net. I use "vhost" and "packed" acceleration. The tap-up.sh and tap-down.sh scripts are as follows:
#!/bin/bash
# tap-up.sh
[ $# -eq 1 ] || exit 1
ip link set ${1} master br0
ip link set dev ${1} up
#!/bin/bash
# tap-down.sh
[ $# -eq 1 ] || exit 1
ip link set dev ${1} down
ip link set ${1} nomaster
To bind the guest nodes to host nodes in a sensible (but admittedly, not very generic) way, I use:
#!/bin/bash
j=1
for i in $( \
echo "info cpus" | \
socat - tcp4:127.0.0.1:55555 | \
sed -e'/^. CPU/!d' -e's/^.*thread_id=//' -e's/\r$//'
)
do
if (( $j <= 3 )); then k='0-3'; else k='4-7'; fi
taskset -c -p $k ${i}
j=$(( j+1 ))
done
