Tag: devops
-
Dynamically-Deployed Static Site Subdomains on Caddy
Continuing my effort to find a way to serve my various static sites that meets my need for ease-of-use and low-maintenance, I configured a wildcard Caddy server with webhook-based monitoring of a GitHub repository. Here's how it works...
-
What breaks when one of your developers leaves?
On Friday, I said goodbye to a colleague as she left us after most of a decade with the company. Then this morning, all hell broke loose on some production servers.
Alt
DataDog graph marked to show a co-worker's departure on Friday evening being immediately followed by a flurry of errors which then stopped for the weekend before coming back in force on Monday morning.
It turns out that the API key that connected our application to our feature flag management platform was associated with her account, and hadn't shown up in the exit audit.
Let this be your reminder to go check where, if anywhere, your applications are using person-specific keys where they should be using generic ones!
-
Moving Three Rings' Servers
Yesterday I achieved one of my primary sabbatical goals and moved Three Rings' primary servers to a new datacentre and a new architecture. But while the visible part happened yesterday, the whole thing's been months in the making and builds upon the effort of an entire team of dedicated volunteers.
-
As well as the programming tasks I'm working on for Three Rings this International Volunteer Day, I'm also doing a little devops. We've got a new server architecture rolling out next week, and I'm tasked with ensuring that the logging on them meets our security standards.
Alt
Terminal screenshot showing a directory listing of a logs directory with several gzipped logfiles with different date-stamped suffixes, and the contents of the logrotate configuration file that produced them.
Each server's on-device logs are retained in date-stamped files for 14 days, but they're also backed-up offsite daily.
Those bits all seem to be working, so next I need to work out a way to add a notification to our monitoring platform if any server doesn't successfully push a log to the offsite backup in a timely manner.