Maybe this is more of a home lab question, but I’m utterly clueless regarding PKI and HTTPS certs, despite taking more than one class that goes into some detail about how the system works. I’ve tried finding guides on how to set up your own CA, but my eyes glaze over after the third or fourth certificate you have to generate.

Anyway, I know you need a public DNS record for HTTPS to work, and it struck me recently that I do in fact own a domain name that I currently use as my DNS suffix on my LAN. Is there a way I can get Let’s Encrypt to dole out a wildcard certificate I can use on the hosts in my LAN so I don’t have to fiddle with every machine that uses every service I’m hosting? If so, is there a guide for the brain dead one could point me to? Maybe doing this will help me grock the whole PKI thing.

UPDATE:

Here’s what I ended up doing:

  1. set up cloudflare as the DNS provider for my domain
  2. use certbot plus the cloudflare DNS plugin to create a wildcard cert. Because I want to use wildcard certs and because the web servers are on a NATed private LAN, HTTP-01 challenge cannot be used. Wildcard certs use a DNS challenge. From what I understand of the certbot docs, the HTTP challenge makes a certain HTTP resource available on the web server, then requests that resource, presumably via an external client, to verify that you own the domain. the DNS challenge works by temporarily placing a TXT record in your DNS server. This method requires your DNS provider to have an accessible API that allows the modification of resource records.
  3. Once the cert and key are generated, I place them on the servers I want to to make use of them and set up the web server accordingly.
  4. Visit the websites and confirm that HTTPS works.

There are some other hiccups that I’m guessing aren’t related to HTTPS. Per My earlier question about self hosting, I’m experimenting with NodeBB. I cannot get the two test instances to federate, which I initially assumed was an issue with HTTPS. That’s a question best asked elsewhere, though I thought it relevant to note because it was my initial purpose for setting up HTTPS.

  • reluctant_squidd@lemmy.ca
    link
    fedilink
    English
    arrow-up
    4
    arrow-down
    1
    ·
    edit-2
    13 hours ago

    Not sure if anyone else mentioned this, but you can just redirect traffic on your local LAN with an ad blocker like pihole ( I currently use adguardhome podman instance )

    Basically, it rewrites any calls to your outside domain from within your local network, back to your local web server. As long as the site is setup with the certificate there, you’re good.

    Then setup a nginx reverse proxy and you’re golden. Regular site outside LAN, internal site inside LAN.

    Edit: spelling

  • A Mouse@midwest.social
    link
    fedilink
    English
    arrow-up
    98
    arrow-down
    1
    ·
    edit-2
    2 days ago

    I use Caddy for this. I’ll leave links to the documentation as well as a few examples.

    Here’s the documentation for wildcard certs. https://caddyserver.com/docs/automatic-https#wildcard-certificates

    Here’s how you add DNS providers to Caddy without Docker. https://caddy.community/t/how-to-use-dns-provider-modules-in-caddy-2/8148

    Here’s how you do it with Docker. https://github.com/docker-library/docs/tree/master/caddy#adding-custom-caddy-modules

    Look for the DNS provider in this repository first. https://github.com/caddy-dns

    Here’s documentation about using environment variables. https://caddyserver.com/docs/caddyfile/concepts#environment-variables

    Docker

    A few examples of Dockerfiles. These will build Caddy with DNS support.

    DuckDNS

    FROM caddy:2-builder AS builder
    RUN xcaddy build --with github.com/caddy-dns/duckdns
    
    FROM caddy:2
    COPY --from=builder /usr/bin/caddy /usr/bin/caddy
    

    Cloudflare

    FROM caddy:2-builder AS builder
    RUN xcaddy build --with github.com/caddy-dns/cloudflare
    
    FROM caddy:2
    COPY --from=builder /usr/bin/caddy /usr/bin/caddy
    

    Porkbun

    FROM caddy:2-builder AS builder
    RUN xcaddy build --with github.com/caddy-dns/porkbun
    
    FROM caddy:2
    COPY --from=builder /usr/bin/caddy /usr/bin/caddy
    

    Configure DNS provider

    This is what to add the the Caddyfile, I’ve used these in the examples that follow this section. You can look at the repository for the DNS provider to see how to configure it for example.

    DuckDNS

    https://github.com/caddy-dns/cloudflare?tab=readme-ov-file#caddyfile-examples

    tls {
    	dns duckdns {env.DUCKDNS_API_TOKEN}
    }
    

    CloudFlare

    https://github.com/caddy-dns/cloudflare?tab=readme-ov-file#caddyfile-examples Dual-key

    tls {
    	dns cloudflare {
    		zone_token {env.CF_ZONE_TOKEN}
    		api_token {env.CF_API_TOKEN}
    	}
    }
    

    Single-key

    tls {
    	dns cloudflare {env.CF_API_TOKEN}
    }
    

    PorkBun

    https://github.com/caddy-dns/porkbun?tab=readme-ov-file#config-examples Global

    {
            acme_dns porkbun {
                    api_key {env.PORKBUN_API_KEY}
                    api_secret_key {env.PORKBUN_API_SECRET_KEY}
            }
    }
    

    or per site

    tls {
    	dns porkbun {
    			api_key {env.PORKBUN_API_KEY}
    			api_secret_key {env.PORKBUN_API_SECRET_KEY}
    	}
    }
    

    Caddyfile

    And finally the Caddyfile examples.

    DuckDNS

    Here’s how you do it with DuckDNS.

    *.example.org {
            tls {
                    dns duckdns {$DUCKDNS_TOKEN}
            }
    
            @hass host home-assistant.example.org
            handle @hass {
                    reverse_proxy home-assistant:8123
            }
    }
    

    Also you can use environment variables like this.

    *.{$DOMAIN} {
            tls {
                    dns duckdns {$DUCKDNS_TOKEN}
            }
    
            @hass host home-assistant.{$DOMAIN}
            handle @hass {
                    reverse_proxy home-assistant:8123
            }
    }
    

    CloudFlare

    *.{$DOMAIN} {
            tls {
    	        dns cloudflare {env.CF_API_TOKEN}
            }
    
            @hass host home-assistant.{$DOMAIN}
            handle @hass {
                    reverse_proxy home-assistant:8123
            }
    }
    

    Porkbun

    *.{$DOMAIN} {
            tls {
    	        dns porkbun {
    			api_key {env.PORKBUN_API_KEY}
    			api_secret_key {env.PORKBUN_API_SECRET_KEY}
    	        }
            }
    
            @hass host home-assistant.{$DOMAIN}
            handle @hass {
                    reverse_proxy home-assistant:8123
            }
    }
    
    • sugar_in_your_tea@sh.itjust.works
      link
      fedilink
      English
      arrow-up
      2
      ·
      10 hours ago

      I did basically this w/ Cloudflare, and it worked perfectly. I used to do ACME requests, but this is simpler and doesn’t require me to route traffic into my LAN. I now expose a handful of services, but I used to have to expose all services for TLS cert renewal to work.

    • theparadox@lemmy.world
      link
      fedilink
      English
      arrow-up
      12
      ·
      2 days ago

      Thanks for being so detailed!

      I use caddy for straightforward https, but every time I try to use it for a service that isn’t just a reverse_proxy entry, I really struggle to find resources I understand… and most of the time the “solutions” I find are outdated and don’t seem to work. The most recent example of this for me would be Baikal.

      Do you have any recommendations for where I might get good examples and learn more about how do troubleshoot and improve my Caddyfile entries?

      Thanks!

    • Monument@lemmy.sdf.org
      link
      fedilink
      English
      arrow-up
      2
      ·
      1 day ago

      The advice I needed and have not been able to find. I could kiss you. Or at least give you a fond nod.

    • conrad82@lemmy.world
      link
      fedilink
      English
      arrow-up
      5
      ·
      2 days ago

      I do the same!

      I have a provider that is not supported by caddy, but I can still use it via duckdns delegation!

      https://github.com/caddy-dns/duckdns?tab=readme-ov-file#challenge-delegation

      Challenge delegation

      To obtain a certificate using ACME DNS challenges, you’d use this module as described above. But, if you have a different domain (say, my.example.com) CNAME’d to your Duck DNS domain, you have two options:

      1. Not use this module: Use a module matching the DNS provider for my.example.com.
      2. Delegate the challenge to Duck DNS.
  • False@lemmy.world
    link
    fedilink
    English
    arrow-up
    8
    ·
    1 day ago

    You don’t need a public DNS record for https to work. You can just use public external certs as long as it’s for a domain you own. You don’t need to setup the same domains externally.

    If you want certs for a domain you own, then yeah you’re looking at self signed.

  • xrun_detected@programming.dev
    link
    fedilink
    English
    arrow-up
    33
    ·
    2 days ago

    +1 for the letsencrypt wildcard with DNS verification, been using this for years. with dehydrated (https://github.com/dehydrated-io/dehydrated) you can automate renewing the certs, pretty convenient.

    One thing i didn’t see mentioned yet - you can also easily create a wildcard for a subdomain of your domain, e.g. *.local.example.com. Most DNS providers let you define something like _acme-challenge.local IN TXT ... so you don’t even need to define an extra zone for local.example.com. Probably makes no big difference, but i like it ^^

    • 4am@lemm.ee
      link
      fedilink
      English
      arrow-up
      9
      ·
      2 days ago

      If you are really looking for hassle-free this is it. LetsEncrypt root certificates are already trusted by most devices so when your friends come over and wanna control the media library or whatever you don’t need to install your locally hosted CA’s self-signed certificates on their phone.

      Also certbot and a cron or systemd timer is all you need; people have rolled all these fancy solutions but I say keep it simple.

      • sem@lemmy.blahaj.zone
        link
        fedilink
        English
        arrow-up
        1
        ·
        8 hours ago

        Adding to this, the eff certbot website has really great noob-friendly instructions which really helped me get set up.

  • IanTwenty@lemmy.world
    link
    fedilink
    English
    arrow-up
    13
    arrow-down
    1
    ·
    2 days ago

    I’ll mention this as no one has yet but you can be your own CA. Tools like mkcert make it easy

    https://github.com/FiloSottile/mkcert

    This is potentially more hassle (than using public DNS) as you have to get your CA certs onto every device. However it may be suitable depending on the situation.

    • False@lemmy.world
      link
      fedilink
      English
      arrow-up
      3
      ·
      1 day ago

      Running your own CA is essentially still a form of self signed. Though it will work better for some use cases (at the cost of more complexity)

      • IanTwenty@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        1 day ago

        I know what you mean but using real self-signed certificates (i.e. no CA at all) with modern browsers causes so many issues I find them unusable.

          • WhyJiffie@sh.itjust.works
            link
            fedilink
            English
            arrow-up
            1
            ·
            edit-2
            1 day ago

            for every single subdomain, on desktop. firefox mobile does not even remember the decision. HA Android straight out refuses it, and thats not a local problem but a relatively known one in the community

            • N0x0n@lemmy.ml
              link
              fedilink
              English
              arrow-up
              1
              ·
              16 hours ago

              Just create a wildcard domain certificate !

              I access all my services in my lan through https://servicename.home.lab/ I just had to add the rootCA certificat (actually the intermediate certificate) into my trust store on every device. That’s what they actually do, just in automated way !

              Never had an issue to access my services with my self-signed certs, neither on Android, iOS, windows, linux ! Everything served from my server via my reverse proxy of choice (Treafik).

              However I do remember that there was something of importance to make my Android device accept the certificate (something in certificate itself and the extension).

              If you’re interested I can send you the snipped of a book to fully host your own CA :). It’s a great read and easy to follow !

              • WhyJiffie@sh.itjust.works
                link
                fedilink
                English
                arrow-up
                2
                ·
                8 hours ago

                Just create a wildcard domain certificate !

                that’s what I do already, but yeah I haven’t added it to the trust store so far, only on linux for git and curl

                If you’re interested I can send you the snipped of a book to fully host your own CA :). It’s a great read and easy to follow !

                that would be interesting, thanks for the offer. but according to plan I don’t want to host a full-on CA, just make the CA cert, store them at a restricted place, and build other certs on top of it for use by nginx

            • False@lemmy.world
              link
              fedilink
              English
              arrow-up
              2
              ·
              edit-2
              1 day ago

              Import it into the trust store in the browser/OS. It should be the same (or very similar) operation for a self-signed cert and a CA that isn’t subordinate to the standard internet root CAs.

              If you can’t import your own root CA cert then you’re probably screwed on both fronts and are going to have to use certs issued by a public CA that’s subordinate to a commonly trusted root CA.

              My point here is that there’s little distinguishing a self-signed cert and a cert issued by your own private CA for most people that are self-hosting.

  • TedZanzibar@feddit.uk
    link
    fedilink
    English
    arrow-up
    14
    ·
    2 days ago

    If you own a domain, which you do, you can get wildcard certs from Let’s Encrypt using a DNS challenge. Most (all?) popular reverse proxies can do this either natively or via an addon/module, you just need to use a supported DNS provider.

  • catloaf@lemm.ee
    link
    fedilink
    English
    arrow-up
    13
    ·
    2 days ago

    You don’t need public DNS. You can use whatever domain you want if you use your own DNS server (though you should use one you own, or something under the .internal TLD).

    Likewise, you can issue whatever certs you want if you trust the CA.

    But LE does support wildcard certs. You can get them with certbot or other tools.

    Personally I use traefik, which has LE support built in. It automatically gets an individual cert for each service. If you use caddy, I’m sure it has something similar.

  • JakenVeina@lemm.ee
    link
    fedilink
    English
    arrow-up
    6
    arrow-down
    1
    ·
    2 days ago

    The most straightforward thing to do, on a private LAN, is to make all your own certs, from a custom root cert, and then manually install that cert as “trusted” on each machine. If none of the machines on this network need to accessed from outside the LAN, then you’re golden.

  • douglasg14b@lemmy.world
    link
    fedilink
    English
    arrow-up
    4
    ·
    edit-2
    14 hours ago

    I just:

    1. Have my router setup with DNS for domains I want to direct locally, and point them to:
    2. Have a reverse proxy that has auto- certbot behavior (caddy) connected to the cloud flair API. Anytime I add a new domain or subdomain for reverse proxine to a particular device on my network a valid certificate is automatically generated for me. They are also automatically renewed
    3. Navigation I do within my local network to these domains gives me real certificates, my traffic never goes to the internet.
    • LovableSidekick@lemmy.world
      link
      fedilink
      English
      arrow-up
      5
      ·
      edit-2
      1 day ago

      When somebody says they “just” reverse the polarity of the navigational deflector array and channel power directly from the warp core.

      I can’t even get host mapping to work on my Centurylink router - the name is defined for the IP address but nothing else on my network can browse to it by name, only by IP. - software dev who has never understood networking.

      • douglasg14b@lemmy.world
        link
        fedilink
        English
        arrow-up
        1
        ·
        14 hours ago

        In this case I run pfSense instead of my ISP provided router. This allows me to have my own DNS resolver, which I can then resolve various domains to internal addresses.

        All devices on my network point to my router for DNS allowing them to resolve internal addresses from all of these.

        • LovableSidekick@lemmy.world
          link
          fedilink
          English
          arrow-up
          1
          ·
          10 hours ago

          Thanks, I’ll lookup pfSense. But straightforward host mapping has worked for me in the past with this router and others. It worked great on my old Cisco DSL router 25 years ago. So simple and straightforward, it should just freaking work. sigh

    • Celestus@lemm.ee
      link
      fedilink
      English
      arrow-up
      3
      ·
      2 days ago

      FYI, all the certs you generate are public record, so it might be a good idea to use a wildcard route in Caddy. That will make it only generates one cert, so no one can find your internal domain names. Especially if your Caddy instance is accessible from the Internet, and you’re expecting external connections not to be able to access domains with only internal DNS records

      • douglasg14b@lemmy.world
        link
        fedilink
        English
        arrow-up
        2
        ·
        edit-2
        14 hours ago

        That’s a good call out.

        There are a few things I do right now:

        1. All of my public DNS entries for the certs point at cloudflare, not my IP.
        2. My internal Network DNS resolver will resolve those domains to an internal address. I don’t rely on nat reflection.
        3. I drop all connections to those domains in cloudflare with rules
        4. In caddy, I drop all connections that come from a non-internal IP range for all internal services. Additionally I drop all connections from subnet that should not be allowed to access those services (network is segmented into VLANs)
        5. I use tailscale to avoid having to have routes from the Internet into my internal services for when I’m not at home.
        6. For externally accessible routes, I have entirely separate configurations that proxy access to them. And external DNS still points to cloudflare, which has very restrictive rules on allowable connections.

        Hopefully this information helps someone else that’s also trying to do this.

  • offspec@lemmy.world
    link
    fedilink
    English
    arrow-up
    3
    ·
    2 days ago

    With certbot there’s probably a plugin to do it automatically, but if you just want to get something working right now you can run the following to manually run a dns challenge against your chosen domain names and get a cert for any specified. This will expire in ~3 months and you’ll need to do it again, so I’d recommend throwing it in a cron job and finding the applicable certbot-dns-dnsprovider plugin that will make it run without your input. Once you have it working you can extract the certs from /etc/letsencrypt/live on most systems. Just be aware that the files there are going to be symlinks so you’ll want to copy them before tarballing them to move other machines.

    certbot --preferred-challenges dns --manual certonly -d *.mydomain.tld -d mydomain.tld -d *.local.mydomain.tld

  • suicidaleggroll@lemm.ee
    link
    fedilink
    English
    arrow-up
    4
    ·
    2 days ago

    Reverse proxy + DNS-challenge wildcard cert for your domain. The end. Super easy to set up and zero maintenance. Adding a new service is just a couple clicks in your reverse proxy and you’re done.