Traditional zone files are still quite a popular way of keeping DNS zone data, especially for Internet Service Providers for whom managing DNS is not the core business and the demand for changing data quickly is not high. The most basic solution is to edit a zone file directly on the primary authoritative DNS server and to use standard DNS zone transfer (AXFR/IXFR) to sync zone contents from primary to secondary servers. In such a setup, care must be taken not to introduce syntax errors into the zone files and to increase the zone serial number after every change. Otherwise the zone will not transfer to the secondary servers and a situation called split-brain will occur.

This is the main reason why hidden primary DNS servers are deployed. A hidden primary server holds the zone data but is not listed as the name server in the parent zone. Its sole purpose is to transfer zone data to the public-facing secondary servers using zone transfer. This way, zone data is guaranteed to be error-free and synchronous amongst all public facing servers. In case of DNSSEC deployment, it's also practical to keep private DNSSEC key material out of the public-facing servers.

It's always useful to track the history of zone files. Using comments directly in the zone files doesn't scale very well and is superseded by modern revision control systems such as Git. Committing every single change to a Git repository, on the other hand, adds to the work overhead of the operator in charge. Even though all sub-tasks are rather simple, doing them manually is prone to errors. Fortunately, Git supports custom- made hooks that can be set up to intervene with various stages of its proceedings, so most sub-tasks, as well as error-checking, can be automated. In the end, the system could look like this:

Figure 1: Custom made hooks can be set up in Git to automate sub-tasks and error checking

Most components of this system are readily available as open-source software from multiple vendors. The only exception is the integration of the Git repository and the DNS server. I found only one project, named GitZone, which combines DNS server integration and Git repository management in one Perl script. I wanted something more lightweight that could be used with existing and well tested Git repository managers like Gitolite. After some research, I decided to create my own lightweight and universal solution.

Introducing dzonegit

Project dzonegit (pronounced as d-zone-git) is a set of Git hooks to automate management of DNS zones. It's written in Python 3.5 and depends only on named-compilezone binary, that is part of the BIND name server.

In the pre-commit hook, it tries to compile all changed zone files. If they don't compile, the commit is refused. If they compile, their serial numbers are compared against previous versions. If serial numbers haven't been increased, the commit is refused as well.

Since all Git hooks are local to the repository instance, they cannot be enforced on a foreign repository. Therefore, similar checks have to be conducted on the DNS-server repository upon receiving commits. This is done by either apre-receive or update hook provided by dzonegit. That ensures that all zone files committed to the repository are valid DNS zone files and that their serial number increases every time they are changed.

Another hook provided by dzonegit is post-receive . It is run after every update of the repository. First, it checks out the most recent revision of the repository to get zone files accessible for the DNS server. Then it generates configuration file snippets to be included into the DNS server configuration. For instance, if using the BIND server, each hosted zone requires a configuration snippet like this:

zone "example.com" { type master; file "/path/to/example.com.zone"; };

In order to be a universal solution, these snippets are generated from a JSON template. For the example above, the template could look like this:

{
  "header": "# Autogenerated by dzonegit on $datetime. Do not edit.\n",
  "item": "zone \"$zonename\" { type master; file \"$zonefile\"; };"
}

The last step in the post-receive hook is to let the DNS server know that some zone files are changed. Two commands can be defined for that:

• zonereloadcmd, which is run for each changed zone and has the zone name appended as the last command line argument, and
• reconfigcmd, which is run whenever some zones are added or withdrawn.

All configurable parameters of dzonegit are stored as Git configuration options under the dzonegit namespace. Using the standard git config utility, they can be set either per repository, per user, or system-wide.

DNSSEC signing

In the early days of DNS Security Extensions (DNSSEC) technology, the only signing solution available was dnssec-signzone utility from BIND. It reads a zone file, signs it and outputs another zone file. Such a utility could be used manually before a zone file is committed to the Git repository. However, such a setup would require manual periodic signature refreshing.

Today, there are a few good implementations of automatic DNSSEC signing and key management like BIND, OpenDNSSEC, PowerDNS or Knot DNS.

In order to make the system as modular as possible, it's a good idea to separate DNSSEC signing and key management to a dedicated functional block, in a setup known as bump-in-the-wire signing. That means unsigned zones are transferred to the signer, signed there and then transferred out to the public-facing secondary servers.

I have chosen Knot DNS as the signer. The reasons behind this are very similar to those explained by the RIPE NCC a while ago. During testing, I discovered a few bugs in the implementation of the key management. They were promptly fixed by the developers, but those fixes are available only in the newest version of Knot. I therefore strongly recommend to use packages provided directly by developers, instead of possibly outdated distribution packages.

The setup of Knot DNS is pretty easy. In the main configuration file, we set up a zone template and signing policy:

template:
  - id: default
    storage: "/var/lib/knot"
    zonefile-load: none   # do not use zone files at all
    zonefile-sync: -1
    journal-content: all  # keep whole zone contents in the journal
    master: primary       # reference to the server serving unsigned zone
    acl: acl_primary      # allow NOTIFY from the primary
    acl: acl_secondary    # allow AXFR from secondary servers
    notify: secondary     # send NOTIFY to secondary servers
    dnssec-signing: on
    dnssec-policy: ecdsa

policy:
  - id: ecdsa
    algorithm: ecdsap256sha256
    zsk-lifetime: 30d
    rrsig-lifetime: 30d
    rrsig-refresh: 15d
    nsec3: on

Then we include a config snippet, generated again by the post-receive hook of dzonegit – it is capable of generating more than one config snippet. In this case, we can make use of per-zone variables to use different signing policies for different zones, so we can, for instance, sign only selected zones or adjust signing policy per zone.

To share or not to share

In a web hosting environment, at least in the Czech Republic, it's quite common to use one set of DNSSEC keys for all hosted zones. For a large number of hosted zones, this dramatically simplifies operational workload. Also FRED, the open-source domain registry developed and operated by CZ.NIC, makes sharing of DNSSEC keys between domains easy by using so-called Keyset objects. These objects hold a public DNSSEC key, which can be associated with any number of domain objects. This is pretty different to the commonly adopted solution defined in RFC 5910 where the DS records are submitted as attributes of each domain object. Since a DS record is made by hashing a public key and domain name, it cannot be shared between zones.

Last year, CZ.NIC deployed automated keyset management, so the secure delegation from the .cz registry can be established and kept in sync by in-band signalling defined in RFC 7344. This reduces the operational workload to keep secure delegations in sync to zero, no matter whether keys are shared or not.

From the perspective of key management, keeping a separate set of keys for each zone is easier. Unless you manage thousands of zones, it's probably better to keep keys separate.

Automate all the things!

Automated management of secure delegation makes running DNSSEC easy. After proper initial setup, there is no periodic task to be taken care of. Unfortunately, there are only a few parent zones supporting automated secure delegation – .cz, .cr, .ch and .li. For other parent zones, Key Signing Key (KSK) rollover still requires manual intervention.

Ignoring the problem is obviously the easiest solution. With state-of-the-art DNSSEC based on the ECDSA algorithm with 256-bit key, the keys are so strong they can be left unchanged for years.

Another solution is to use CDS/CDNSKEY records produced by the DNSSEC signing software as a universal signalling solution and implement automatic management for other registries on your own. Securing reverse records delegated from the RIPE NCC looks like the lowest-hanging fruit, as there is a pretty simple and well-documented REST API to read and update the RIPE Database, which holds the secure delegation data. A similar API is available for the ARIN Database. For APNIC, any changes to DS records have to be submitted via the MyAPNIC portal, which is quite problematic to automate.

I've created a proof of concept code that compares DS record data stored in the RIPE Database with DS record data obtained by a DNSSEC-validated query for DS record of the zone itself. If new record data are found, the database gets updated. During RIPE 77, this got some attention in the DNS Working Group, so maybe in the future similar software might be provided by the RIPE NCC. A similar approach can be used with other domain registries, provided that you find a registrar supporting DNSSEC and having an API to allow DS record updates.

The biggest challenge is how to secure delegations from zones hosted at your own platform. This would require either some sort of automatic commits into the Git repository whenever DS records have to be changed, or some hidden intermediate step of adding DS records after checking out the Git repository, before the zone is loaded into the DNS server – perhaps some sort of smudge filter.

Editing DNS records now easier

After a few months of testing, we have switched the DNS provisioning system of all domains hosted by CESNET (the Czech NREN) to the presented solution at the beginning of September 2018, after a few months of testing. The transition to Knot DNS allowed us to migrate from shared RSA keys to independent ECDSA keys. The reduced workload and a safety belt made editing DNS records a much easier task for human operators.

There is still some work to be done. Everything is open-sourced, contributions are very welcome. Also, if you think automated updates of secure delegations are a clever idea, speak to your registry, registrar or raise your voice in the RIPE community's DNS Working Group.