I often run demos on my laptop with the help of libvirt. Managing 20+ machines that way is annoying when you have no DNS resolution for those. Luckily, with libvirt and NetworkManager, that can be easily solved.
Imagine you want to test something in a demo setup with 5 machines. You create the necessary VMs in your local KVM/libvirt environment – but you cannot address them properly by name. With 5 machines you also need to write down the appropriate IP addresses – that’s hardly practical.
It is possible to create static entries in the libvirt network configuration – however, that is still very inflexible, difficult to automate and only works for name resolution inside the libvirt environment. When you want to ssh into a running VM from the host, you again have to look up the IP.
Name resolution in the host network would be possible by adding each entry to
/etc/hosts additionally. But that would require the management of two lists at the same time. Not automated, far from dynamic, and very ponderous.
Luckily, there is an elegant solution: libvirt comes with its own in-build DNS server, dnsmasq. Configured properly, that can be used to serve DHCP and DNS to servers respecting a previous defined domain. Additionally, NetworkManager can be configured to use its own dnsmasq instance to resolve DNS entries – forwarding requests to the libvirt instance if needed.
That way, the only thing which has to be done is setting a proper host name inside the VMs. Everything else just works out of the box (with a recently Linux, see below).
The solution presented here is based on great post from Dominic Cleal.
First of all, libvirt needs to be configured. Given that the network “default” is assigned to the relevant VMs, the configuration should look like this:
$ sudo virsh net-dumpxml default <network connections='1'> <name>default</name> <uuid>158880c3-9adb-4a44-ab51-d0bc1c18cddc</uuid> <forward mode='nat'> <nat> <port start='1024' end='65535'/> </nat> </forward> <bridge name='virbr0' stp='on' delay='0'/> <mac address='52:54:00:fa:cb:e5'/> <domain name='qxyz.de' localOnly='yes'/> <ip address='192.168.122.1' netmask='255.255.255.0'> <dhcp> <range start='192.168.122.128' end='192.168.122.254'/> </dhcp> </ip> </network>
You can modify the network for example with the command
virsh net-edit default. The interesting part is below the mac address: a local domain is defined and marked as
localOnly. That domain will be the authoritative domain for the relevant VMs, and libvirt will configure dnsmasq to act as a resolver for that domain. The attribute makes sure that DNS requests regarding that domain will never be forwarded upstream. This is important to avoid loop holes.
Note, however: as mentioned in the comment by taurus, your domain should not be named “local” because this might cause trouble in relation to mDNS.
Configuring the VM guests
When the domain is set, the guests inside the VMs need to be defined. With recent Linux releases this is as simple as setting the host name:
$ sudo hostnamectl set-hostname neon.qxyz.de
There is no need to enter the host name anywhere else: the command above takes care of that. And the default configuration of DHCP clients of recent Linux releases sends this hostname together with the DHCP request – dnsmasq picks the host name automatically up if the domain matches.
If you are on a Linux where the
hostnamectl command does not work, or where the DHCP client does not send the host name with the request – switch to a recent version of Fedora or RHEL 😉
Because with such systems the host name must be set manually. To do so follow the documentation of your OS. Just ensure that the resolution of the name works locally. Additionally, besides the hostname itself the DHCP configuration must be altered to send along the hostname. For example, in older RHEL and Fedora versions the option
has to be added to
At this point automatic name resolution between VMs should already work after a restart of libvirt.
The last missing piece is the configuration of the actual KVM/libvirt host, so that the local domain, here
qxyz.de, is properly resolved. Adding another name server to
/etc/resolv.conf might work for a workstation with a fixed network connection, but certainly does not work for laptops which have changing network connections and DNS servers all the time. In such cases, the NetworkManager is often used anyway so we take advantage of its capabilities.
First of all, NetworkManager needs to start its own version of dnsmasq. That can be achieved with a simple configuration option:
$ cat /etc/NetworkManager/conf.d/localdns.conf [main] dns=dnsmasq
This second dnsmasq instance just works out of the box. All DNS requests will automatically be forwarded to DNS servers acquired by NetworkManager via DHCP, for example. The only notable difference is that the entry in
/etc/resolv.conf is different:
# Generated by NetworkManager search whatever nameserver 127.0.0.1
Now as a second step the second dnsmasq instance needs to know that for all requests regarding
qxyz.de the libvirt dnsmasq instance has to be queried. This can be achieved with another rather simple configuration option, given the domain and the IP from the libvirt network configuration at the top of this blog post:
$ cat /etc/NetworkManager/dnsmasq.d/libvirt_dnsmasq.conf server=/qxyz.de/192.168.122.1
And that’s it, already. Restart NetworkManager and everything should be working fine.
As a side node: if the attribute
localOnly would not have been set in the libvirt network configuration, queries for unknown
qxyz.de entries would be forwarded from the libvirt dnsmasq to the NetworkManager dnsmasq – which would again forward them to the libvirt dnsmasq, and so on. That would quickly overload your dnsmasq servers, resulting in error messages:
dnsmasq: Maximum number of concurrent DNS queries reached (max: 150)
With these rather few and simple changes a local domain is established for both guest and host, making it easy to resolve their names everywhere. There is no need to maintain one or even two lists of static IP entries, everything is done automatically.
For me this is a huge relief, making it much easier in the future to set up demo and test environments. Also, it looks much nicer during a demo if you have FQDNs and not IP addresses. I can only recommend this setup to everyone who often uses libvirt/KVM on a local machine for test/demo environments.
26 thoughts on “[Howto] Automated DNS resolution for KVM/libvirt guests with a local domain [Update]”
I want my physical network to be able to resolve both A and PTR records in the virtual domain. A records can be solved mostly following your lead – my hypervisor’s resolver library doesn’t point to its own dnsmasq, rather to my routers, which will forward queries for the virt domain to dnsmasq on the hypervisor – but PTR records will always result in the query storm you described, as libvirt provides no sane way to add the *.in-addr.arpa domains marked as localOnly. Any thoughts?
Hm, that’s a good question. Haven’t tried that so far. There were some PTR patches for libvirt in 2016, but I haven’t tested them…
I ran into this same problem. Check out the ‘localPtr’ attribute for the ip tag (https://libvirt.org/formatnetwork.html). With this set & the reverse zone set to forward on the hypervisor host (see below), I can do reverse DNS queries (on the hypervisor) without hitting the ‘concurrent dns queries’ issue
server=/122.168.192.in-addr.arpa/192.168.122.1I ran into this same problem. Check out the ‘localPtr’ attribute for the ip tag (https://libvirt.org/formatnetwork.html). With this set & the reverse zone set to forward on the hypervisor host (see below), I can do reverse DNS queries (on the hypervisor) without hitting the ‘concurrent dns queries’ issue
@pjt, thanks for the detailed discussion of the localPtr attribute, that one was new to me!
Thanks for the article.
When running “sudo virsh net-dumpxml default” my domain is NOT showing localOnly=’yes’. How can I enable this option via the CLI? The In the KVM-GUI I did not find any option to enable this.
Found the solution:
virsh net-dumpxml default > default.xml (add missing entry to xml)
virsh net-destroy default
virsh net-define default.xml
Thank you very much.
You can edit the default network with
virsh net-edit default
The link referenced in the article ( https://m0dlx.com/blog/Automatic_DNS_updates_from_libvirt_guests.html ) also instructs how to make the changes take effect: virsh net-destroy default; virsh net-start default; This will make your VMs unreachable and they should be rebooted.
Thanks. Good instruction. But a couple of very important notes. I spend six hours before found why it’s not worked for me.
1. dnsmasqd must be installed but must be disabled.
2. It’s very bad idea to use local as domain name. https://superuser.com/questions/355625/unable-to-resolve-foo-local-domain-names
Please add this notes to your instruction. It really pitfalls.
Hi taurus, thanks for the comment. I might have to redo this entire setup soon anyway and then will update the article and cover your suggestion as well.
Let’s say I have a public IP for the host, a bridge set up for a guest with its own public IP, and another guest using NAT. Further, I have a FQDN of example.com, and some subdomains: 1.example.com, 2.example.com.
With this method work if I assign a guest example.com or 1.example.com, or assign both example.com and 1.example.com to a guest?
Unfortunately I have never tested this setup with a bridged setup. So I am not sure if this will work. Also, I have yet to find out how to assign full domains instead of hostname+domain.
I’d be happy to update the article though if you run tests and share the results with me.
Hi. Thanks for the tip.
Is there a way to dynamically “update the DNS”? If I change the name of a host I must reboot it before I can ping it again.
(NB : I’m not really modifying my hostnames all the time, but first time I boot a new VM from a base image that serves as a template a hostname is automatically assigned )
You should be able to restart networking with ‘systemctl restart network’ and that will cause the dhclient to broadcast the hostname on the network.
@Mike Yes it works, Thanks.
I also manage to do the randomize sooner in the init system before network steps (seems obvious but I’m not very good with service configuration ^_^) which solve my problem, but still good to know it can be done on the fly.
@Mike, thanks for answering this even faster than I could do. 🙂
@Jmeb, glad that this was of help to you!
In case you are using a newer distribution with systemd-resolv activated: this howto will not work anymore. systemd-resolv simply has no idea that there might be domain names in NetworkManager.
These two commands solve the problem temporarily:
❯ resolvectl dns virbr0 192.168.122.1
❯ resolvectl domain virbr0 lxyz.de
But they are lost after a reboot – once I have enough time to look into this I will post an update on this here.
Many hours were spent trying to solve this issue. Happy to report this post (https://libvirt.org/nss.html) was the solution.