ramblings of a geek

The trouble with wildcard certs; Distributing them. Here’s how!

When Let’s Encrypt started issuing wildcard certs (i.e. *.your.domain.com) I was so very happy. In ‘production’ senses, I really dislike them. But for ‘home’ use?

Eh, they’re ok.

They really simplify the management of being able to use them all over the place. No longer do I need a different cert for my SEIM, wiki, ldap hosts, www hosts, software repos, etc. Just deal with the cert.

When the cert (via certbot) is renewed, however, it’s only updated on the host that contacts the Lets Encrypt servers. What I’ve found is that because of the short expiration of the certs, every three months all of my hosts start complaining because of expired certs.

That’s an issue. I also don’t want to bundle the private key in my generic configuration management system. Checking in any private keys into any source code repo just gives me the shivers. Ew.

What I did then was realized, I already had a vault store that I I could store them in.

Since’s its been awhile since I really wrote any code, I figured I would go ahead and write some python that would store the certs/chains/privkeys from my certbot host into my Vault instance.

Then, each host that needs the cert, I can then just query the vault (weekly, for example) and update the certificate from the vault.

All the code is in github. Would love to have folks let me know what they think.

How I Started Using Vault

Vault. It’s the “new” thing (it’s actually not that new).

Secrets management. I’m sure you already know what it is if you’re here. What I did to get using it:

First, I had to find a host that would run it well enough. I knew I wanted to make my life simple and run it in a docker image. I have a relatively small home network setup, a few subnets, that will need access. I tried to run it on an old RaspberryPi model B+ (one of the original 32bit ones) that I had laying around. Docker on the RPi sucks.

Next, I thought about just running it on an old MacMini 2010 (Dual-Core 4GB Ram). Yeah – no dice there either as Docker CE won’t run on that Mac without virtualization extensions in the CPU. Which I thought was weird, since I use it to run several VirtualBox hosts.

Fine – I’ll run it on one of the generic servers I have.

For my (at least initial) use cases, I’m not going to get fancy with a highly available backend. One of these days, I may actually look into it with an S3 backend too.


But not today.


Next, on the host where we are running the vault container, create a directory to store the vault files. This will be local configuration of vault, the secrets file that will be stored on disk, and other stuff.

mkdir -p /srv/vault

Then I created a local.json file I stole from the interwebs and tweaked it a little bit. Eventually, I’ll get around to configuring TLS. But not today. I (mostly) trust my home network.


Then we run the docker, binding it local filesystem into the docker instnace


Once the docker has been instantiated, go ahead and connected to it and get a shell:


Let’s make sure the /vault/data directory is owned by the correct users:


And then we go ahead an initialize the vault database:


Please store your keys appropriately. If you trust your family, give one to your partner/spouse. Give one to each kid. Etc. 🙂


Go ahead and unseal the vault using 3 of the 5 tokens created from the initialize above.

For me, I want to store my AWS ACCESS_KEY/SECRET_KEY in vault:


Let’s see if we can re-read them from the vault:

Final thoughts — Since we just about done there are some final things you should check on (not a complete list):

  • Check the file permissions on the host OS
  • How do we make sure vault is always running
  • Do we keep the vault unsealed for our automated systems accessing it
  • Create authentication tokens
    • Don’t have everything ‘login’ to vault as root. Grant restrictive permissions to what needs access to what.
  • You’ll probably want to enable port 8200 available to more than just the loopback address
    • Probably will want to enable TLS before doing so
  • You can unseal the vault via curl:
    • curl -H “Content-Type: application/json” -X PUT http://${VAULT_IP}:8200/v1/sys/unseal -d ‘{“key”: “${VAULT_UNSEAL_KEY}”}’


Not dead; Just Busy

There hasn’t been a post in awhile, but that doesn’t mean I’ve not been busy.

I had been thinking of participating in the Hacky Easter event, but I don’t think I’ll even be able to register, much less participate.

Oh well.

I typically hate the “oh yeah, I’ve been doing so many cool things stay tuned” posts, but here I am writing one.

I’ve been in the process behind the curtains moving sites around and building new infrastructure. Stay tuned for some info on how I’m managing this site (along with several others) using Terraform and Ansible, specifically how I do them “securely”.

Buffer Exploits

I happen to come across an on-line exploit development class recently. It is/was a class hosted by the Community City College of San Francisco, but due to some logistical issues the professor is hosting as a ‘pirate class’ — available to everyone for free.


After the first couple weeks, I’m having a blast in it. It covers some basic ASM (for 32bit Intel processors) and basic C and teaches the GDB 101 stuff.


I’m starting to dust off many old cobwebs from an ex-developer from decades ago.


Wow, has it really been that long?


It took me longer than I wanted, but I was eventually able to remember how to read stack memory.


For example, when debugging, it’s helpful to know the memory address where the stack starts, ends and the memory address of the next instruction (the eip register).

Above, you’ll see the output from GDB when you ask it to print the register information. You’ll notice things like eax, ecx, edx and ebx, all general purpose registers that are being used.

For this post, however, take a look at esp and ebp. Esp is the pointer to the current stack frame. And esb is the base pointer. The way I think of it (and I hope I’m not wrong about it) is that the stack memory begins at esp and ends at ebp.

To display the memory of the stack, we can tell gdb to to print (arbitrarily) thirty hexadecimal 32-bit WORDs — starting at at the start of the frame:

What we see above is that the memory is printed started from 0xffffd5f0, which happens be the same as $esp when we looked at ‘info reg’. To make a Each line starts another ‘bank’ (in my head) of memory. The first row lists the four memory address: 0xffffd5f0, 0xffffd5f4, 0xffffd5f8 and 0xffffd5fc.

Following along to $ebp, the memory referenced at location 0xffffd638 would be “0xffffd648”.




Long time Pi; First time Arduino

Long time RaspberryPi fan here. My first Arduino though has arrived. First step is to print out a case for it (1/2 done already) …. Next steps? Who knows. I need a project for it.

centralized authentication

Isn’t it funny how what should be a simple task turns into a complicated project in no time flat?

After checking out letsencrypt.org for some web site SSL stuff, I decided it was time to start moving the self-signed certificates over to ‘real’ certs too.

I’ve for a long time run some internal LDAP services (hosted on RaspberryPi’s too) but something I’ve wanted to do for over a now is move to a FreeIPA system.

Now, FreeIPA doesn’t (from what I can tell) run on a RaspberryPi, so I decided to kick up a VM to run it. For testing purposes, it’ll be an ubuntu guest running VirtualBox on an older MacMini that I’ve got laying around. That is until I get my new virtualization rig in.

A word to the wise. Make sure you give your virtual machine 2GB of memory to play with. I also learned that the latest Fedora 27 kernel doesn’t boot under my VirtualBox environment.

A lot of trial and error went on in this endeavor.

First off, I’m running FreeIPA on a Fedora 27 host (not fully updated due to reason above). With a base system installed, do a ‘dnf install freeipa-server’. While I don’t have the notes available, I’m pretty sure I searched for freeipa (dnf search freeipa) and installed all of the available packages.

I wasn’t able to initially install and configure the DNS portions. But next was a simple ‘ipa-server-install’, selecting a lot of the defaults. I did get what looks like a working system up eventually (i really reinstalled the darn thing 50 times).

After everything was up and I could log into the WebUI, I did a ‘ipa-dns-install’. It should be noted that for this process I’m actually migrating all of the hosts over to a new network name. Also new IP address space. That’s going to be fun. Can’t wait to see how FreeIPA handles that.

In the mean time some reading materials (in no particular order):

Some things to keep in mind:

now with moar https

This morning was spent learning how this hosted wordpress site deals with https. Not something that is intuitive for me since I have always done it ‘my way.’

Folks should now only be directed to the https version of the site. The initial homepage still doesn’t show as “secure”. My guess is there is a link or two on the page that it pulls in via http. I’ll look into that later. It’s not like I’m taking folks credit cards or anything personal.

Just, why not encrypt.

A co-worker put me onto letsencrypt.org. Pretty nifty if I do say so. The caveat to “free” certificates is that they are only valid for 3 months, but I saw a live demo where every three months you run a single command to ‘refresh’ them.

If it really is that easy, I’ll be converting most of my internal SSL certs over to them rather than self-signed ones.

The ‘trust’ aspect of the certs though is questionable for legitimate business use cases though.

Also, while you’re at it. If you want it — using the menu on the left (click the three little lines) you can find my current gpg key.

new year; new blog

On the todo list for some time has to been to revitalize the website. Welcome to WordPress.

Over the holiday season when I wasn’t busy with family [whom I am grateful to have near me] I was able to spend the a good portion of time working on the SANS holiday hack challenge. It was my first year and I learned a ton of practical implementations of theories I have spoken about over the past several years.

If you’re interested – go ahead and read my write up on this blog about how I worked my way through each of the challenges.

There are several other write-ups coming to surface now that the contest is over. A few of them (in no particular order):