Run your own DNS-over-HTTPS (DoH) Server with NGINX

I run an DNS-Server (dnsmasq) on my local network to block certain Domains so that they don't deliver ads to me. I simply use dnsmasq with a list of additional hosts that are mapped to "0.0.0.0". This is very much like Pihole, but DIY.

For the fun of it, I wanted to provide the DNS service not only via TCP/UDP 53, but also via the all new and fancy HTTP protocol.

I decided to stay with dnsmasq and let the HTTPS part be handled by nginx. Why:

• I tried some DNS-Servers with included DoH support (bind, unbound and DNSdist). They were all working but could not handle the blocklist of one million domains as quickly as dnsmasq does
• My server already has NGINX running - with a standalone DoH-Server I would have to use another port or another ip address for DoH

So we let nginx handle the TLS termination and the proxying to the dns server. Caching, domain blocking and request forwarding is handled by dnsmasq. Every tool does what it is primarily designed for. Sounds good!

But we can't simple proxy the HTTP protocol to dnsmasq. We have to get the HTTP-Body (if DoH-Post) or the dns query parameter, base64-decoded (if DoH-Get). A DoH guide from Nginx suggests to use the nginx JavaScript Module "NJS" and the "nginx-dns" scripts from TuxInvader.

I have tried that - but it wasn't working very well. After patching the script things were working. However, having DNS-Requests parsed and mangled by JavaScript within a webserver process seems a little odd to me. So I decided to strip the script down to the bare essentials of getting the DNS payload.

Settings things up

We need NJS support in nginx. For debian there is the package libnginx-mod-stream-js

apt install nginx libnginx-mod-stream-js

Create the NJS-Script - vi /etc/nginx/doh.js:

function handle_request(s) {
	s.on("upstream", function(data,flags) {
		if ( data.length == 0 ) {
			return;
		}
		var bytes;
		if (data.toString('utf8', 0, 3) == "GET") {
			const path = data.toString('utf8', 4, data.indexOf(' ', 4));
			const params = path.split("?")[1];
			if (!params) return;
			const qs = params.split("&");
			qs.some(param => {
				if (param.startsWith("dns=") ) {
					bytes = Buffer.from(param.slice(4), "base64url");
				}
			});
		}

		if(data.toString('utf8', 0, 4) == "POST") {
			bytes = data.slice(data.indexOf('\r\n\r\n') + 4);
		}

		if (bytes) {
			s.send( to_bytes(bytes.length) );
			s.send( bytes, {flush: true} );
		} else {
			s.send("");
		}
	});

	s.on("downstream", function(data, flags) {
		if ( data.length == 0 ) {
			return;
		}
		// Drop the TCP length field
		data = data.slice(2);

		s.send("HTTP/1.1 200\r\nConnection: Keep-Alive\r\nKeep-Alive: timeout=60, max=1000\r\nContent-Type: application/dns-message\r\nContent-Length:" + data.length + "\r\n");
		s.send("\r\n");
		s.send(data, {flush: true});
	});
}

function to_bytes(number) {
	return Buffer.from([((number>>8) & 0xff), (number & 0xff)]);
}

The script above gets the DNS binary payload and is invoked by an nginx stream service:

Add this to nginx-config - vi /etc/nginx/nginx.conf:

stream {
	js_import /etc/nginx/doh.js;

	upstream dnsmasq {
		zone dns 64k;
		server 127.0.0.1:53;
		server 8.8.8.8:53 backup;
	}
	server {
		listen 127.0.0.1:8053;
		js_filter doh.handle_request;
		proxy_pass dnsmasq;
	}
}

This is our middleware. It gets the DoH-Request from the vhost, extracts the binary DNS query and passes it to a local DNS server. A fallback DNS server (line 7) can also be defined.

Last step: Setup the DoH vhost endpoint - vi /etc/nginx/sites-enabled/default:

upstream dohloop {
	zone dohloop 64k;
	server 127.0.0.1:8053;
	keepalive_timeout 60s;
	keepalive_requests 100;
	keepalive 10;
}
server {
	listen 443 ssl http2;
	listen [::]:443 ssl http2;

	server_name xxx;
	ssl_certificate xxx;
	ssl_certificate_key xxx;

	location /dns-query {
		proxy_http_version 1.1;
		proxy_set_header Connection "";

		proxy_pass http://dohloop;
	}
}

This is the regular vhost that receives the DoH request on /dns-query and passes it to the middleware.

Test it

Dig can be used to test DoH:

dig +https @your-server.com A example.org
dig +https-get @your-server.com A example.org