Automated LetsEncrypt DNS-Based Challenges

JohnEDee

Is an automatic DNS challenge via DNS provider API anywhere on the radar? The current manual DNS challenge option is appreciated, but it's still a regular manual intervention.
DNS challenges are the only way to do wildcard domain certs with LetsEncrypt, and for a site like mine that has a ton of subdomain sites all with the same domain, it would save a tremendous amount of time.
It would be awesome to have the Abyss ACME client be able to interact with the larger DNS providers (AWS Route 53, MS Azure DNS, GoDaddy, etc) and be able to automatically create/change the DNS TXT record when necessary. I think that process/capability is described here, and many ACME clients can apparently do that, so the code should already be written and available.
I think the way it would work is that for a given LetsEncrypt cert, I'd give the Abyss ACME-bot the credentials to be able to make changes to the "_acme-challenge" TXT record in my DNS, and as long as I'm using a capable provider, it would do the TXT record thing itself whenever the cert needs to be renewed.
I understand that it may not be a huge priority, but hopefully it can be worked in at some point.
Thanks!

admin

JohnEDee,
We have considered this but the complexity and the number of different APIs each DNS provider uses have made us really afraid of chasing a moving target.
Please check the number of scripts in https://github.com/acmesh-official/acme.sh/tree/master/dnsapi where each provider has a different method for updating its DNS records.
This is a daunting task and we'd have to test against each provider which is almost impossible for us to do. We may restrict to the few most used ones but not to the full list.

JohnEDee

Thanks for the reply, and I absolutely agree: limiting it to the most-used DNS providers would be great. Anyone who wants to use LE DNS challenges would need to switch to one of them....it's easy to do. :-)

admin wrote
JohnEDee,
We have considered this but the complexity and the number of different APIs each DNS provider uses have made us really afraid of chasing a moving target.
Please check the number of scripts in https://github.com/acmesh-official/acme.sh/tree/master/dnsapi where each provider has a different method for updating its DNS records.
This is a daunting task and we'd have to test against each provider which is almost impossible for us to do. We may restrict to the few most used ones but not to the full list.

JohnEDee

Just to give you the scope of why I'd like the DNS challenge to be available, even if it's with a limited number of DNS providers....I currently have 67 websites running in Abyss, all with different hostnames in one domain.
With my previous commercial certificates, I just did a wildcard cert for the domain and set them all to that. With LetsEncrypt HTTP challenges on individual hostnames, I would have to create 67 separate entries...and also likely will run afoul of LE's limits, one of which is 100 certs per domain (because I'm using LE for other non-Abyss-hosted hostnames in our domain, and that will take us over the limit). After that, they say (rightly) that you should be using a wildcard cert.
(BTW, it's not that I'm too cheap to buy commercial wildcard certs, but with the industry now forcing annual renewals, I'm done with that nonsense...I was ok at 10 years, then 5 years, then 3 years...but every year, nope. LE's hands-free automatic renewal is the only way to go now.)
But wildcard certs can only be done via DNS challenge, hence why I really want it to be available in the product, even if it's only for a few of the bigger DNS providers.
In the meantime, I'm going to have to do a wildcard cert and manually make the DNS changes every 90 days.
FWIW, CertifyTheWeb's awesome (but sadly Windows-only) open source project does an amazing job, including DNS challenges with a bunch of providers, so the DNS challenge source code is there to refer to if needed. They based their project (including the DNS challenge code) on a previous open source project called PoshACME, so there are references to that in there.

admin

JohnEDee wrote
FWIW, CertifyTheWeb's awesome (but sadly Windows-only) open source project does an amazing job, including DNS challenges with a bunch of providers, so the DNS challenge source code is there to refer to if needed. They based their project (including the DNS challenge code) on a previous open source project called PoshACME, so there are references to that in there.

Thank you for the pointers. Most of the DNS plugins in these projects are relatively clean PowerShell scripts so they can simplify the work.

JohnEDee

admin wrote
Thank you for the pointers. Most of the DNS plugins in these projects are relatively clean PowerShell scripts so they can simplify the work.

FWIW, it looks like all C# code to me in the CTW code, probably more transportable than PowerShell.

admin

JohnEDee wrote

admin wrote
Thank you for the pointers. Most of the DNS plugins in these projects are relatively clean PowerShell scripts so they can simplify the work.
FWIW, it looks like all C# code to me in the CTW code, probably more transportable than PowerShell.

Sure. We missed the code in https://github.com/webprofusion/certify/tree/development/src/Certify.Providers/DNS which is cleaner than PowerShell's scripts.

JohnEDee

It has been a couple of years, so just wanted to check in to see if this feature request might get any love in the near future. :-) I'm still using my workaround to replace the wildcard cert every three months, but it requires some downtime and mass-editing the abyss.conf is always a hold-my-breath experience, so I wondered if we might get the domain-based auth soon.

admin

JohnEDee wrote
It has been a couple of years, so just wanted to check in to see if this feature request might get any love in the near future. :-) I'm still using my workaround to replace the wildcard cert every three months, but it requires some downtime and mass-editing the abyss.conf is always a hold-my-breath experience, so I wondered if we might get the domain-based auth soon.

Unfortunately, this great feature did not make it in Abyss Web Server until now because of its complexity.
Creating a specific "recipe" for each DNS provider is very time consuming and very complex to test.
So far, we're working on an alternative idea which leverages an established tool such the one you're using. Abyss Web Server will run that external tool to provision the certificates and will handle all the work of installing the certificate and so on. This won't be perfect but this may save us a lot of work and uncertainty.
What do you think of the idea?

Horizon

Hello,

admin wrote
So far, we're working on an alternative idea which leverages an established tool such the one you're using. Abyss Web Server will run that external tool to provision the certificates and will handle all the work of installing the certificate and so on. This won't be perfect but this may save us a lot of work and uncertainty.
What do you think of the idea?

That's exactly what I wanted for DNS-based challenges.
Basically Abyss talks to the ACME servers, gets the challenge data, provides it to an external tool, then Abyss could itself poll the challenge DNS record to verify that the external tool added it correctly.
Then it tells the ACME server to check it, and it handles certificate installs & renewals.
We could call this feature 'DNS challenge interpreters'.
This way Abyss does the ACME communication & certificate configuration.
Then external tools will do the specific per-service DNS communication.

JohnEDee

admin wrote

JohnEDee wrote
It has been a couple of years, so just wanted to check in to see if this feature request might get any love in the near future. :-) I'm still using my workaround to replace the wildcard cert every three months, but it requires some downtime and mass-editing the abyss.conf is always a hold-my-breath experience, so I wondered if we might get the domain-based auth soon.

Unfortunately, this great feature did not make it in Abyss Web Server until now because of its complexity.
Creating a specific "recipe" for each DNS provider is very time consuming and very complex to test.
So far, we're working on an alternative idea which leverages an established tool such the one you're using. Abyss Web Server will run that external tool to provision the certificates and will handle all the work of installing the certificate and so on. This won't be perfect but this may save us a lot of work and uncertainty.
What do you think of the idea?

I'm a little confused about your stated need to create a specific "recipe" for each DNS provider, because the open-source code that we had discussed in this thread back in 2021 handles all those "recipes": https://github.com/webprofusion/certify/tree/development/src/Certify.Providers/DNS
But whatever gets the job done is fine with me! Whatever this established tool is that Abyss can interact with, it will be a great near-term feature. And if necessary for some reason, you could always replace later with more integrated functionality.
.

admin

JohnEDee wrote
I'm a little confused about your stated need to create a specific "recipe" for each DNS provider, because the open-source code that we had discussed in this thread back in 2021 handles all those "recipes": https://github.com/webprofusion/certify/tree/development/src/Certify.Providers/DNS

The code is there but we cannot use it verbatim for many reasons. The most evident is that it requires a scripting language that needs to be installed on many platforms that Abyss Web Server targets. It may even not install on some. So a lot has to be rewritten from scratch.
Second, most of these recipes are contributions from developers and not from the companies behind these services. We see in the code a lot of guess work and some edge cases are not handled. It would become a support hell for us if we shipped that code in the wild without verifying and rewriting it.

JohnEDee

admin wrote
The code is there but we cannot use it verbatim for many reasons. The most evident is that it requires a scripting language that needs to be installed on many platforms that Abyss Web Server targets. It may even not install on some. So a lot has to be rewritten from scratch.
Second, most of these recipes are contributions from developers and not from the companies behind these services. We see in the code a lot of guess work and some edge cases are not handled. It would become a support hell for us if we shipped that code in the wild without verifying and rewriting it.

Ah that makes sense, thanks for taking the time to explain it...just another of the many reasons why Aprelium/Abyss is so awesome!
And thanks for the alternate method of accomplishing this, I'm looking forward to giving it a try! :-)

JohnEDee

Coming up on a year later...any joy in the land of LetsEncrypt automated domain/DNS-01 functionality?

admin

JohnEDee wrote
Coming up on a year later...any joy in the land of LetsEncrypt automated domain/DNS-01 functionality?

This is still on our to do list. A proof of concept has been done for pre-setup and post-setup ACME actions. But it has yet to find its way into the main version...

JohnEDee

Hi, just wanted to check in on this functionality to see if it's still on the radar somewhere for a future version. :-) Thanks!

admin

JohnEDee wrote
Hi, just wanted to check in on this functionality to see if it's still on the radar somewhere for a future version. :-) Thanks!

Please be sure it is on our radar as promised. :)
We'll probably switch to basing the solution on https://github.com/dns-lexicon/dns-lexicon which is Python-based and could work seamlessly on all platforms Abyss Web Server supports.

JohnEDee

Just wanted to check in on the progress of this feature. Thanks!
BTW, love the new forum platform!

aprelium-support

JohnEDee It's nice to see you posting on the new forum. 🙂

Unfortunately, we're probably going to not honor our promise exactly : instead of DNS challenges and their toll of complex scripts, we're going to implement straightaway the newly announced DNS-PERSIST01 challenge.

You can read about it in https://letsencrypt.org/2026/02/18/dns-persist-01 . It's modern, solves the issue radically without any fiddling with APIs. Basically, you can prove once with DNS records that you're allowing certificate issuance for a domain or wildcard domain for an indefinite duration...

JohnEDee

Oh that's not unfortunate, that's wonderful, even better! And thanks for the heads-up on DNS-PERSIST01, that's a much superior mechanism. Since it's much simpler, do you think it might make it into Abyss sooner than later, to coincide with the LE rollout of the feature?

BTW, the Insert emoji feature isn't working, it just comes up with regional_indicator_a through regional_indicator_g. Hence the giant icon image lol.

aprelium-support

JohnEDee Since it's much simpler, do you think it might make it into Abyss sooner than later, to coincide with the LE rollout of the feature?

That's the plan if there are no issues with the staging platform from Let's Encrypt.

aprelium-support

JohnEDee BTW, the Insert emoji feature isn't working, it just comes up with regional_indicator_a through regional_indicator_g. Hence the giant icon image lol.

When the drop-down is shown, start writing "hand" or "wink" and the list will change to include the actual emojis based on the typed text.