391 – What the heck is an immutable website? Tim Nash knows and explains all

[00:00:00] Nathan Wrigley: Hello there, and welcome once again to the WP Builds podcast. You've reached episode number 391 entitled, what the heck is an immutable website? Tim Nash knows and explains all. It was published on Thursday, the 26th of September, 2024.

My name's Nathan Wrigley, and before I am joined by the ever so knowledgeable Tim Nash, a few bits of housekeeping.

The first thing to mention is that black Friday is just around the corner. It may seem like a long way off, but of course it's not. And over at WP Builds we're preparing you all of the WordPress things. We've got a page up and running. If you head to WP Builds.com forward slash black. It's a bit spartan at the moment, meaning there's not a lot of stuff there, but if you have a product in the WordPress space, that could be hosting, it could be a plugin, a theme, a block, anything rarely, then hit the add a deal button and completely for free we will add your black Friday deal, and we'll publish it onto this much visited page during the run up to black Friday.

If you would like to sponsor that page and get your product or service in a pride of place top at the spot, head to one of the black sponsor cards at the top, and hit the get started button. We've been joined by WS Form already, and we would really love to have some other contributors there as well.

So once again, it's completely free to put your products on there. Completely free to browse, search, filter. Go book mark it now. WP Builds.com forward slash black. Once more WP Builds dot com forward slash black, for all of your WordPress goodness in the run-up to black Friday.

The WP Builds podcast is brought to you today by GoDaddy Pro. GoDaddy Pro the home of managed WordPress hosting that includes free domain, SSL, and 24 7 support. Bundle that with The Hub by GoDaddy Pro to unlock more free benefits to manage multiple sites in one place, invoice clients, and get 30% off new purchases. You can find out more at go.me/wpbuilds.

We're also joined by Bluehost. Bluehost, redefine your web hosting experience with Bluehost Cloud. Managed WordPress hosting that comes with lightning fast websites, 100% network uptime, and 24 7 priority support. With Bluehost Cloud, the possibilities are out of this world. Experience it today at Bluehost.com/cloud.

And we're also joined by Omnisend. Omnisend, do you sell your stuff online? Then meet Omnisend. Yes, that Omnisend. The email and SMS tool that helps you make 73 bucks for every dollar spent. The one that's so good, it's almost boring. Hate the excitement of rollercoaster sales? Prefer a steady line going up? Try Omnisend today at omnisend.Com.

And sincere thanks go to GoDaddy Pro, Bluehost, and Omnisend for their support of the WP Builds podcast.

Okay. What have we got for you today. Well we've got Tim Nash. I like to call him, sir Tim Nash. Very knowledgeable he is all about WordPress and security. And he's discussing with us today something which I'd never heard of, an immutable website. And this is a real tin foil hat episode. So get your mug of coffee, put your tinfoil hat on and be prepared to geek out.

Tim talks about all of the variety of ways that you can lock down the operating system, as well as other things to make your website truly immutable and therefore extremely secure.

As I said strap in. I hope that you enjoy it.

I am joined on the podcast by Tim Nash. How you doing Tim?

[00:04:20] Tim Nash: Hi. Thanks for having me. I.

[00:04:22] Nathan Wrigley: If you could see Tim Nash, as I can see Tim Nash, then you'd, you'd realize Tim is surrounded by all sorts of lovely wildlife. He's, he lives in West Yorkshire, which is a delightful part of the world, and out of the window over his shoulder is just a lovely pasture with horses in it. It's very calming, Tim.

It's very nice.

[00:04:40] Tim Nash: I need to be calming influence. people come to me and they think I'm full of rage and worry and anger and all I don't. I just

[00:04:52] Nathan Wrigley: Yeah.

[00:04:52] Tim Nash: Discuss their problems

[00:04:54] Nathan Wrigley: the reason that people come to Tim all full of angst and anger is that Tim is a, a WordPress, general internet, online security expert. And, and I'm guessing that most of your clients approach you in a time of need and, fairly anxious things have gone wrong. Is that the case?

[00:05:13] Tim Nash: good percentage do I, forget, I'm in a bit of a weird position where. Half my clients come to me because they need to come to me right now, this very second, they have something that's going wrong and the other half come to me because they want to be really proactive and fix things before they go wrong.

Obviously we all prefer to be in the second group,

[00:05:32] Nathan Wrigley: Yeah.

[00:05:34] Tim Nash: the first group when they arrive, they're normally, I'm the la I'm the last recourse. So it's oh no. we're gonna throw the dice and we're gonna go to Tim and see if he can

[00:05:44] Nathan Wrigley: Yeah, when the internet started, I had such high hopes that humanity's best would be online. And everybody had pulled together for this fabulous resource, this amazing thing. It's such a shame that people decided to spoil it and create viruses and write malware and all of this kinda stuff.

But I guess, on some level, it keeps you, busy and keeps you in work.

[00:06:09] Tim Nash: yeah. And it's one of those things that's whenever someone's there's a massive vulnerability that's been released. There's part of me that's going, this is terrible, and obviously I want, don't want people to.

[00:06:21] Nathan Wrigley: Yeah. Okay. So there's every cloud and all that has a silver lining. That's great. I guess we should probably give you a minute to give us your bio paint a picture of who you are. You've been on this podcast, on the, this week in WordPress show loads and loads of times. So if you've been a, recipient of that, then you'll know Tim's voice and you'll know his credentials.

But for those of you that just listen to the podcast, will you just give us your potted bio for a minute or so?

[00:06:46] Tim Nash: Yeah, my name is Tim Nash. I'm a WordPress security consultant here in the uk and I have been involved in WordPress for so long that plugins weren't available when I first started. and I started in WordPress because I wanted to build a membership site. So I. Started to put it together, then plugins came out and so I started building a plugin and I ended up building one of the very first commercial WordPress plugins and thought, oh, I can make a business outta this tiny bit too early.

Commercialization of WordPress wasn't really a thing. and it but we did manage to get a little bit of a company going. we had an office in LEED at one point. We had six people. We had a telephone. You could ring us up for support. Imagine that. Now you could free up, Dave would answer the support phone.

but we were a little bit too ahead of our time. we got word for being a commercial.

[00:07:46] Nathan Wrigley: oh, so that was in the period where it was still verboten to pay for things?

[00:07:51] Tim Nash: Yeah. Oh.

[00:07:53] Nathan Wrigley: Oh, okay. That must have been a hard pill to swallow. Yeah.

[00:07:57] Tim Nash: It wasn't the nicest of feelings, I'll be honest. But, after that I, I worked for, some large organizations helping to drive open source and open source security into, big NGOs and government agencies. then I went and, worked for a hosting company and set up one of the first managed WordPress hosting and one of the largest in the uk.

At which point I went from, securing a dozen sites that were my Properties and little people, friends, family, a couple of clients, but to a hundred thousand sites overnight.

[00:08:36] Nathan Wrigley: Yeah,

that's

[00:08:37] Tim Nash: which suddenly this, like the scale changed dramatically. and when Covid happened, like, many people, I think I had a bit, some life changes.

And I said goodbye to the hosting company. Working at the hosting company, and took this role on as a security consultant. most people, I felt they met me. They've met me at a word camper or

[00:08:59] Nathan Wrigley: Yeah. Yeah.

[00:09:01] Tim Nash: and I'm probably best known for giving scary security talks, that I'm really lucky and fortunate that, my previous employer would sponsor me to go to these events.

So I basically had the job of going around the country scaring

[00:09:15] Nathan Wrigley: Yeah, it's

[00:09:16] Tim Nash: which is possibly the greatest job. but then when you've got a, I've got my daughter, and all of sudden it's maybe I should be at home a bit more. And at. Find, I want to go do that again for a little while.

[00:09:32] Nathan Wrigley: Can I ask you a question? This is apropos of nothing, but equally, it's an interesting insight. The security landscape seems to change. Really quickly, new OSS come out, new devices come out. As far as I can tell, it moves at a rate almost like nothing else. new threats are undermining things all the time.

You hear about things in the national press, but I guess in the tech press a lot more, what I'm trying to get at is, it the kind of industry which looking back is really difficult to keep up with. 'cause with the best will in the world, nobody's getting any younger. and it seems to be moving at breakneck speed all the time, and in fact accelerating all the time.

Is it really hard work to keep up with what's going on given that it just seems to accelerate?

[00:10:22] Tim Nash: it certainly can be. I found myself the other day I was listening to a podcast and there was somebody talking about something on it. I can't even remember what it was, and I was there going, I have no idea what all of the words they just

[00:10:37] Nathan Wrigley: Okay.

[00:10:37] Tim Nash: mouth. It's made. The security industry is very good at the marketing random terms and acronyms, and so everything has 28 alternative acronyms that you need to learn and mark and lots of marketing phrases that you ju to mean the same thing.

So there is a lot of, times where you going, oh, that sounds really new. And then you start digging away and go, oh. it's just this, they've just given it a fancy term and a fancy name and a lot of the new threats and things that are coming that you see coming out and they sound like they're new, really are based on, very basic principles.

And so there you can still learn to cybersecurity and yes, you have to keep on top of the trends, but there isn't too much stuff that's new. It's just how we're presenting it that keeps

[00:11:30] Nathan Wrigley: Yeah. Yeah. Oh, that's nice to know. And you seem to be on top of it anyway, and it seems to be something that keeps you going. So that was a nice intro. Thank you so much. However, having said all of that, that's not what we're gonna talk about. So maybe we should just erase the first five minutes of this podcast.

No, we shan't because this is about something else, and I've gotta say from the outset, dear listener, I have no idea. What is about to happen? usually I've got a fair idea, but the, this episode occurred, because of some conversation which happened, which I was not really party to, but I was fairly close by and I overheard it and, and said, Tim, will you come on my podcast and talk about that?

Do you wanna tell the story of, WordCamp Whitley Bay this year? Just quickly, and that'll give us the backdrop to what we're gonna talk about.

[00:12:16] Tim Nash: So I was at work at Bay. We were, it was the after party we were sitting around talking. just generally chatting about WordPress and there was a couple of us who were talking about, how we manage deployments and how we talk about sites and keeping things up to date. And I'm very much known as the, if you know me more recently, you probably know a massive advocate of automatic updates.

And that I tell everybody they should have automatic updates, that there's very few times you shouldn't have automatic updates. And I'm sitting there and I say, D la, automatic updates. No, you're doing to the person who was saying, oh, we can't quite do automatic updates, but we do that. I'm like, I encouraging them and saying, yep, and you can take the next step to automatic updates.

And then I drop the bombshell. Of course I don't use.

[00:13:13] Nathan Wrigley: Yeah.

[00:13:14] Tim Nash: I explained that, I use a basis of an immutable file system and an immutable WordPress. And we started talking about that and it turned out that, one of the large BIP agencies to have someone there as well. And they went, that's pretty much what we do. and there was this like shock from the other people who were like, what is this?

And I was. Oh, I thought this was something that everybody did. Obviously I didn't quite a technical and geeky thing, but it isn't common

[00:13:47] Nathan Wrigley: Okay. immutable in my head means non-changeable. It means that you set, it and forget it. That's where I'm at with that word, and I don't know if quite that maps to what it is that you are talking about. Okay. We are gonna be talking about immutable websites. I'm guessing this has got to do with the OS on some level or the, the operating system and how it interacts with the files and who's got permission to change things.

Go right back to the beginning, explain immutable in this context, and then start to lay out the jigsaw pieces so that we can understand.

[00:14:19] Tim Nash: let's start with the word mutable, because that's probably an easier one for people to get their heads around, because if we say, if something mutates, it changes. Now you mutate your WordPress site every time that you. You publish a post, upload a file, make changes to a plugin or whatever the, underlying bits of it change con, not quite constantly, but they, change.

And when you make something immutable. It stops changing. Now we use the word immutable WordPress in a way that is a bit disingenuous. So we, if we start by stepping right the way back with what the goal is, and if the goal is we don't want bad actors to be able to change the file system, we also don't want our developers to be able to change the file system or our random system admin or our authors, other people.

We, we want to lock down the file system, so it is. Impossible to make

[00:15:19] Nathan Wrigley: Okay, Tim, I'm just gonna stop you there, because knowing the audience that we've got, it's, I imagine quite a lot of the people have never interacted with the file system in WordPress. They, they, go to their URL, they type in WP admin or something, log in, and it's just there. It's all magic.

They, might have paid a, company to set up the hosting and, all of that. But, dear listener, your WordPress website, you could, if you wanted, view all of the files on your computer. there's just a bunch of files and a database. There might be some other complex things thrown in there, but it, is literally a bunch of files, PHP files and other things.

And, so that's what Tim's on about. You can, when you download it from wordpress.org. You are downloading a bunch of files, which then create a database when you set it up and, okay. Sorry, I just thought I should say that.

[00:16:06] Tim Nash: Yeah, that's all we get into

[00:16:08] Nathan Wrigley: Yeah, we really could. Yeah.

[00:16:18] Tim Nash: All these change. Let's now go. Okay, so our goal is we want to lock things down as much as possible. Why would we want to do this? we don't want necessarily, if a bad actor did compromise the site, the one thing we really would like it, not them not to be able to do, is to compromise the server as well.

If you compromise the site, your, and you are limited to what's in the database, maybe making entries and changes to the database, maybe some limited actions on the site. It's not great, but it's better than them having full sy system access to your server and everything. So we want to, from a bad access specific, we also want it so that, if we have a development team, so you have someone who you pay to look after the site and you make changes and then they make changes, but they've made changes in their own little special place, be that GI or some, sort of development system somewhere.

And then they push their changes, your changes get overwritten and it all gets very complicated. We wanna try and make it as simple as possible. So when we then think of this and we can go, okay, what do we actually need to write? So by writing we mean moving, putting a file there or putting a record in the database and that's it.

You, they're going, oh, actually. To publish a post, I need to have access to the database 'cause I need to be able to log in and it needs to look up to the database. I need to be able to push data into the database as the offer and I might have some pictures or something. So I need to upload those pictures to the file uploads. But that's all you need. So if we can make it so that just the uploads folder and the database are writeable. Is not, then we gonna have a much more secure

[00:18:13] Nathan Wrigley: Got it. Okay. So again, just going back when you're talking about, writeable, this is, an operating system level permission. So this is, let's take the example of just let's go with Linux. 'cause that's what everybody's on. I'm pretty sure, in the, olden days at least. Anyway, and.

There's all sorts of per, there's several permissions that come with any folder or file, one of them being write. So if you have a folder that has right permissions, depending on whether your user has that permission, you can write to it. But if you can basically say, okay, this folder is non writeable, nobody can write anything to it.

That means it's walled off. You've built a big brick wall around it, and if everything remains the same, nobody can change that. It's. Completely cast iron protected. Have I summed that up about right. Okay.

[00:19:08] Tim Nash: yeah. if we take our most simplest version, and we're gonna call this immutable Mark

[00:19:15] Nathan Wrigley: Nice. Okay. Yeah. Good.

[00:19:17] Tim Nash: So we, can just go into our, so normally when you set up a, a WordPress site, there's some instructions on mething called od,

which is, if you've never come across it, the chances are you've just, it's automatically been set up for you, but you can go in and you can change the file permissions, and we have.

Read, write, and execute. And for user group and everyone, and historically, everybody would set this to what's called 7, 7 7, which is to do with how bytes are formed, but it's basically, that would mean ReadWrite and execute for everyone group and you, which is

[00:19:58] Nathan Wrigley: Yeah. Bad, Yeah.

[00:20:03] Tim Nash: files. So we, if we set ev every file to be read only, except the uploads folder now in our little WordPress site itself is completely locked down. It still works. Nathan's looking

[00:20:21] Nathan Wrigley: Yeah. no. Sorry, I thought you

[00:20:23] Tim Nash: How does this still

[00:20:24] Nathan Wrigley: no. Yeah. I was just gonna say how exactly that my head was thinking, but if, everything is locked down except the uploads folder. You are gonna have to explain that because surely there must be situations where, hang on a minute, it can't, I don't know.

You wanna upload a oh or modify, so No. No. Alright. Okay. My head's got too much in it now. Keep talking because I am confused.

[00:20:47] Tim Nash: We, we, lock everything down. It's completely locked down. We set it all to read owning, except for the uploads folder. Our database was. Will we'll come

[00:20:55] Nathan Wrigley: Oh, okay. Alright. Okay.

[00:20:57] Tim Nash: but we'll assume that we can read the date, read and write into the

[00:20:59] Nathan Wrigley: Okay.

[00:21:01] Tim Nash: we can do, we can now publish posts. We can still log in.

our, as far as our, WordPress site's concerned, it's fine. we do have a few problems though.

[00:21:12] Nathan Wrigley: This is where I was going. Yeah, go on. Yeah.

[00:21:17] Tim Nash: Nathan wants to install a

[00:21:19] Nathan Wrigley: The first example that they don't belong in the uploads folder, in that sort of sense. Yep.

[00:21:23] Tim Nash: So you go to add plugins and you click the install now and you get a prompt notice up that says, I can't write to the plugin, and you go, no. Of the bad actors and

[00:21:41] Nathan Wrigley: Okay.

[00:21:42] Tim Nash: have several ways we can deal with this. One is we can say, Every time I want to update a plugin or add a plugin, I will run a script or go through and I will go to the, to the plugins folder and go.

You can now be write, you can have read and write permissions for this for period, do your upload, and then put it back to read only. This is cumbersome, it's doable, and in our very first thing, we could do this with some scripting so we can write a little program that goes through and every time we want to do an update, it sets the file permissions to let us update.

We do our updates, and then it sets the file permissions at the end and says, stop. And I, as a little experiment, when automatic updates first came out, there's some hooks and filters. So in WordPress, hooks allow you to add your own codes to the end of things. So I wrote a little thing that when it started the automatic updates, it's basically opened the, got the file permissions wrote, told the, the operating system, Hey, I'd like access.

The operating system fired a little scripts that gave the access. The updates ran and then at the end we had a hook that fired back and said, I'm finished, and it set it back to read only. That's fine. Gets cumbersome.

[00:23:06] Nathan Wrigley: Yeah.

[00:23:08] Tim Nash: But it works well. It has some big advantages when you have lots of admin users because admin users are horrible creatures who want to install things and cause chaos at wherever they are.

and you don't really want, if you've ever developed a scientific, you are an agency. the worst thing for you is a non-agency admin owner coming along and go installing the 18th plugin of the day because they've been testing it out. They can't do any of that anymore. You've completely locked it

down.

[00:23:41] Nathan Wrigley: I can well imagine that in, okay. So I imagine you, and certainly I was dealing with sites which were a little bit more dynamic than, that, people were logging in as bonafide editors or, curators, whatever it may be of content and then logging out again and what have you.

So those sites themselves needed to have that. that ability to upload content and what have you, but it occurs to me that a really significantly large amount of WordPress websites don't need any of that. they are brochure sites. They have, they, they're born, they're put into existence.

They've got five pages and. That's it. And they go out there and live their life online, and they don't need any updating of any kind. There must be literally millions, tens of, possibly hundreds of millions of those. So they'd be, perfect for that. And you wouldn't need to even need to.

Yeah.

[00:24:37] Tim Nash: you could argue that they probably should be static HTML sites and that you could generate the H TM L version, but if you didn't want to do that because maybe you have a contact form on

[00:24:47] Nathan Wrigley: Oh, yeah. Yeah.

[00:24:49] Tim Nash: or something that's slightly dynamic, then this is the ideal scenario because now we can lock the whole thing down.

We could still go in and they're posts and pages, but we don't need to worry about it. And when we do need to do updates. We can go and change. Now, not doing updates doesn't, mean that we, just because we've made everything read only doesn't mean that we've got rid of all potential attacks that could be done against this site.

Overwhelmingly, a large amount of, vulnerabilities are based around the database and gaining access to the database. And the database might still have things that we, they may, we don't really want them publishing posts. We don't want them ha gaining access to information. If you've got a contact form or an email list that's signing up, you don't want 'em to get access to that, that data.

But we can this is take the those and go, okay, there are other problems for another day to deal with. But by preventing them from being able to access 'em right to the to our file system, we are saving ourselves a lot of time. The problem is they can still write to the file system 'cause we've given them those uploads

[00:26:00] Nathan Wrigley: Okay. Yeah.

[00:26:04] Tim Nash: This is where we either we can do certain things like, Hey, don't let PHP files run in the uploads file. That's just a standard thing. You, should probably be doing that if you're not already. Now you, if you don't know if you are, then chances are you are, because your host will have set this up to stop this.

But, we can do certain things to prevent files being, executed. In the uploads funnel, literally we can tell it not to execute files in there. but we probably best off if we go to version two.

[00:26:39] Nathan Wrigley: Ooh, okay.

[00:26:41] Tim Nash: So version two is where we go. We've got our operating system where our server, and we've got our uploads folder, but we're gonna move our uploads folder somewhere else.

[00:26:52] Nathan Wrigley: Oh, okay.

[00:26:54] Tim Nash: move it to something like S3 Buckets, which is something that's set up by AWS. You might have come across them. They have just file storage areas. They call them block storage systems. And it's basically a remote location you can dump files to, and then you can pretend that it's connected through your site. people like Leno, you find lots of these solutions that refer to themselves as S3 compatible, and there are couple of really good plugins for interacting. We to make your, media library being uploaded on S3 or somewhere else, but then appear as if they're just on your local server. So now we can, have our uploads somewhere else and if they get compromised,

[00:27:44] Nathan Wrigley: Got it.

[00:27:46] Tim Nash: it's in a bucket somewhere else on Amazon system.

And to be honest, Amazon's big enough to be able to deal

[00:27:54] Nathan Wrigley: Yeah,

[00:27:58] Tim Nash: This sitting in file.

[00:28:00] Nathan Wrigley: So it's literally just a dump for files, And, if somebody compromises that, all they're gonna get is a bunch of files, which were probably publicly available anyway. They'll just have access to them. So you're removing that from your own file structure. Putting it somewhere else and then pointing WordPress.

Every time it wants something in the uploads folder, it just gets hijacked and it says, ah, the uploads folder is over here. and it, and WordPress imagines that, that's on your server, if you like. And then, and then grabs it. And I'm also imagining if it's something like AWS for AWS read all the other possible solutions out there, it's pretty cheap as well.

it's not something that's probably gonna cost you a lot of money.

[00:28:39] Tim Nash: Apps really cheap and we, if we are, if we're at the point where we're scaling to a point where it's not cheap, there are ways to make it cheaper. This is where you'd use A CDN and things like that as well to bring that cost down. So we are not even having to call those files every single time. We'd most more likely than have it on S3, but then call it via CD and call elsewhere.

[00:29:00] Nathan Wrigley: Okay, So we've locked the file system down. That was phase one or version one. Version two was then offload the media library or the, uploads folder, to somewhere else. Okay, great. I'm with you.

[00:29:12] Tim Nash: So now we're going, now we've got this scenario. we can introduce something like version control, so we can introduce our GI deployments. And this is something that most people who are in like agencies and developers will talk to you about. And if you are not a developer, you gloss over, but it basically means we can store the code in one place.

And go, when I'm happy with the code as it is, I'm going to push it, onto the server. Great. So now we can, so now, instead of having our little script that does our, that's doing anything complicated, we tell our, we have put a code in the Git repository, and then we say, have a mechanism that can deploy the code.

And then during that deploy, we make the system writeable, the code gets deployed, and then we make the system read only again.

[00:30:04] Nathan Wrigley: So that then therefore, let's say you're using, GitHub. 'cause that's one that everybody's largely familiar with. I imagine you can do all of the updates, you could write your new content, you could have the fabulous new plugin that you wanted, push it from Git and then it would become writeable for the period of time Git was interacting.

And then Git finishes and the handshake on the way out is right. Become immutable again. it.

[00:30:30] Tim Nash: The content, so the content is in the database is we're still, users can still log in. In this scenario, we're still publishing content through the websites. We would normally, we're not making anybody learn any weird esoteric systems or anything. So that is asked version two. We're feeling quite,

[00:30:49] Nathan Wrigley: that's pretty good. Yeah.

[00:30:51] Tim Nash: feeling good, this is feeling very achievable.

So we're gonna version three, which is most people's websites sit on something called a lamp stack. Linux MySQL often swapped out for N Engine X. My SQL often stopped out for something like MariaDB my C, but these all sit on the same server. So your MySQL is on the same server as your PHP and your patch.

They don't have to be, and MySQLs not designed that way. when you connect to the MySQL server, it's actually making what is effectively a network request. It's just looped

[00:31:34] Nathan Wrigley: happens to be local anyway. yeah,

[00:31:37] Tim Nash: So our next step move the MySQL. We say, Hey, in MySQL, can you be not here? Can you be on a different server?

[00:31:45] Nathan Wrigley: So this is exactly the same as step two, which was offload the uploads

[00:31:48] Tim Nash: Yeah, so up until now we've got step two. The only difference now is that we have my SQL on a remote server. We've got our file uploads on a, in a, S3 type bucket, and now we have a box and that has an files our Apache or engine X and our PHP on there. We can now lock this down even further because now we can say we, we can, if we're managing the server, we can start saying, now we don't need to have read access to anything but like a few things for sessions. okay, so now all the underlying stuff can start to be locked down.

[00:32:32] Nathan Wrigley: Okay. I'm, following you. I'm with you so far. So there's three, three different things happening at the same time now. Okay. Keep going. Sorry. This is great.

[00:32:40] Tim Nash: So now this, at this point we are like, wait, there's surely something systems and things that do this already. Because now our, to be honest, our little deployment scripts getting a bit longer or long

[00:32:54] Nathan Wrigley: Yeah.

[00:32:54] Tim Nash: we're having to do more and we quickly discover that there's something called immutable operating systems.

[00:33:00] Nathan Wrigley: Ew. Okay.

[00:33:01] Tim Nash: So first of all, we just, on our little adventure, we go, oh. You can make whole file systems read only

[00:33:12] Nathan Wrigley: Huh?

[00:33:13] Tim Nash: you can, rather than you changing individual permissions of directories, you can just say, no, I just want this entire volume to be read only please.

[00:33:22] Nathan Wrigley: So everything above the level of your WordPress website or from the WordPress website down, if you like, just make that okay. Okay.

[00:33:31] Tim Nash: But we can then, but then the idea of any mutual os is that, everything is frozen, all the applications are frozen. 'cause it, if you think of an a, an operating system, he's just a large collection of little applications. you, even when we talk about things like a kernel isn't one thing.

It's. A collection of little applications that interact, basically the center of your operating system, and then there are layers of upper applications out and all of a sudden begin, make all of these read only.

[00:34:02] Nathan Wrigley: Okay. Alright, and that's on the operating system level. What can you give, can you just shout a few names of immutable operating system vendors if you

[00:34:12] Tim Nash: things, so, probably the one that most people are familiar with is, something called N os, which is the declarative, operating system. but you've got most of the major Linux systems now have an immutable version. it's, it is the

[00:34:29] Nathan Wrigley: It's the hotness, the all the hotness. Okay. All right. Yeah.

[00:34:34] Tim Nash: But this hotness has been going for 10

[00:34:37] Nathan Wrigley: Okay. Yeah,

[00:34:38] Tim Nash: hotness is one of these

[00:34:39] Nathan Wrigley: quite warm now.

[00:34:42] Tim Nash: things in decades as rather than necessarily weeks or years. now we've got, we can lock the whole thing down, but deploying this is gonna get more clunky. We go to, forgot we're four, so we're gonna go up

to

[00:34:58] Nathan Wrigley: we were on four, I think. Yeah.

[00:35:00] Tim Nash: Okay, we're going up to five then. And you can get this concept of an image,

[00:35:05] Nathan Wrigley: Oh,

[00:35:06] Tim Nash: which is basically your whole operating system. Like you imagine you've got a hard drive and you take hard drive.

[00:35:14] Nathan Wrigley: freeze it.

[00:35:16] Tim Nash: Freeze it in time and take and put it so that you've got it. And now we, every time we deploy, what we actually do is we take our copy of our hard drive, we implant our work, new WordPress changes onto our copy, freeze that, and then push it back in.

[00:35:33] Nathan Wrigley: Oh, that's, neat. Yeah, I can totally see. It when it's frozen, if you like. It's, it just, it's not even there. It's just like it's stored somewhere, but it's in some sort of blob. And the blob needs to be un blobbed, for want of a better word, defrosted, if you like, and then you make your changes and then you refreeze it again, stick it in the freezer.

It's no longer available and so it literally cannot be messed around with

[00:36:00] Tim Nash: The whole.

[00:36:03] Nathan Wrigley: That's.

[00:36:03] Tim Nash: each coming in and the writeable parts are still separated away. We've got our database over here and we've got our uploads in our S3 bucket. So it doesn't matter what we change it for. Now, the thing about doing that is writing out an operating system, putting overlaying changes, making it immutable, and then uploading it takes a bit of

[00:36:25] Nathan Wrigley: Right.

[00:36:26] Tim Nash: I imagine most people do not, even though it's, we're measuring this in minutes, not in decades or anything, but it's still a lotted

[00:36:34] Nathan Wrigley: a bit of waiting around that is a bit of a pain. Yeah,

[00:36:38] Tim Nash: So this is where we do something called, red green deployment, which is basically we have a live server and then we just spin up another server with our new image.

[00:36:50] Nathan Wrigley: right.

[00:36:51] Tim Nash: When it's ready, we say, stop sending traffic to the old server, send it to the new one.

[00:36:57] Nathan Wrigley: Huh. Okay, let me just encapsulate that in my head then. at the point where you want to, at the point where you want to defrost it, if you like, you, you build that, you defrost it onto this red server, let's call it, and the green server is currently taking all the traffic when it's built, ready to go, you've done all your checks and you're sure that it's.

Exactly what you want. You just say, okay, now point all the traffic to the red server. It now becomes the green server. The green server becomes the red server, and you kill what's now the obsolete red server. And I'm moving my hands all over the place. Oh, you keep it as a snapshot kind of thing?

[00:37:42] Tim Nash: Now you've got a, if something has gone wrong, you can hop back.

[00:37:46] Nathan Wrigley: Got it. Got it. Okay.

[00:37:48] Tim Nash: This also means you can always do your testing

[00:37:51] Nathan Wrigley: Yes. Yeah. Yeah.

[00:37:53] Tim Nash: because you can test over here and make sure your tests are passing before you flip and move.

[00:37:59] Nathan Wrigley: Yeah, no, that makes sense. and then in that situation, I dunno, you might keep like 10 or 15 or a hundred or whatever, and I'm gonna call them snapshots of previous versions of your website until at some point you just decide, okay, those ones beyond the hundred, they just fall off a cliff now because we're happy, it's been going for a year or more without any problems.

We're probably okay. Gosh. Okay, so that's step five is, dare I ask, is there a step six.

[00:38:26] Tim Nash: at.

[00:38:28] Nathan Wrigley: He's roll. Tim at this moment is rolling. He is literally rolling up his sleeves and cracking his knuckles.

[00:38:35] Tim Nash: This, we got a, but it.

[00:38:42] Nathan Wrigley: Yeah, it really, at this point, it feels like you're in the weeds of tinfoil hats and propeller caps and all of that kind of, you're a real nerd at this point.

[00:38:54] Tim Nash: Yeah. And so we, then go, okay, we're never gonna get outta that stage. So we're stuck in tinfoil hat's world. But, we can, at least at, the moment we're putting up a. Server, we are gonna have to go, okay, we're gonna need to change the IP address on the DNS, so we're gonna have to keep our, and do all these hot swapping on DNS stuff and it's all gonna get very messy.

So we introduced our final thing, which is a load balancer. So we have a thing in the front that says, hi, I am wp build.com. It's not really, it's just a plate. It's just literally sitting there going. You want WP Build. It's that way and it points it in the direction of whichever server it wants to. Now we can scale this because up until now we've been talking about one site being on one server, one stack.

But let's now say JP builds. It's obviously as the premier WordPress, podcast.

[00:39:52] Nathan Wrigley: Oh, you're very kind.

[00:39:54] Tim Nash: a lot of traffic.

[00:39:55] Nathan Wrigley: Oh, it's almost immeasurable, Tim. Frankly,

[00:39:59] Tim Nash: way, way more than that. Bob Feller, who,

[00:40:04] Nathan Wrigley: I didn't say that, Bob. It wasn't me.

[00:40:07] Tim Nash: way, way more. So he need, so Nathan needs to scale out and he suddenly needs to have, it gets a rush of traffic. now we can use the load balancer and spin up, say, five or 10 of these servers.

[00:40:20] Nathan Wrigley: I see. Of course. Yeah, that makes perfect sense. got it. Got it.

[00:40:26] Tim Nash: We can just infinitely scale these out. not quite infinitely, but, we can have 5, 10, 15, 20 of these and 'cause our data is nicely set on our database and elsewhere, and we can likewise do the same with the database server. We can start putting that into a cluster. We can have multiple database server.

So this allows us to go from a site that is, quite on a little rickety foundations of its little VPS to being able to scale. Horizontally massively.

[00:40:58] Nathan Wrigley: To whatever

[00:40:59] Tim Nash: can then start

[00:41:00] Nathan Wrigley: throw at it. Yeah. Yeah. Yeah.

[00:41:02] Tim Nash: hey, only when we start, and this allows us to do things like, when we go to deploy our new version, that we are fully tested and you can still be doing daily updates with this, doesn't stop daily updates or automatic updates.

We're just having to, we just automate the process behind the scenes, but this allows us to go, okay. I want to test it by only sending 10% of my traffic to the new server

[00:41:30] Nathan Wrigley: Oh, okay. The load balancer takes care of that. You can just,

[00:41:34] Tim Nash: Yeah. So you can set up some things and now we have a really reliable, robust system that can be used in pretty much every scenario that you can think of with WordPress in a safe way.

and this is, at this point we go, okay. we have this, there is no way we are doing this manually. Now we are using some sort of what we call an orchestration tool. Now that can be something as simple as a bunch of bash scripts, be something like Ansible, but it can also be something or more complicated like Kubernetes and all sorts of things that are like that.

Or in cases of some operating systems like Nicks. Ns Os is also, is based off the Nicks language, which is a decorative file Try that again, is, which is a decorative language underneath. So when you open up a Ns file, it's just a big file which defines where everything goes, what you're doing, and how you're doing it.

You can use Nicks to deploy itself and your WordPress site. And it much like you could, use a Docker and Docker compose file if you're familiar with those. and suddenly we are in a position where making changes to your WordPress site is as simple as having a, bot that runs in your Git repository that say, that goes, okay, run the updates, grabs all the updates from WordPress org.

Runs a build process automatically. Deploys your site does the test. If the test passed, then the site gets updated. If the test fail, it sends you a message in Slack or wherever you'd like to get your messages saying, Hey, something's probably wrong. We probably shouldn't, we've not deployed this version.

Come check it out so you can still have your automatic updates, which is what I

[00:43:37] Nathan Wrigley: Yeah. Oh, no. No. Okay. Yeah.

[00:43:42] Tim Nash: I just don't use the automatic update.

[00:43:45] Nathan Wrigley: feature within what? Yeah.

[00:43:48] Tim Nash: automatic updates. I don't go and do updates to my site every day. Plugins every day. But I just, they just run automatically via, in my case, it's, via a Gitte instance.

But, you could do it on GitHub, you can do it on GitLab, you can however you get your, code deployment

[00:44:09] Nathan Wrigley: what whatever gets you have of choice. so it, it strikes me that, for the average user who is maybe listening to this podcast, a lot of that is very interesting, very technical and, probably just a bit of an eyeopener like, gosh, I didn't even know that kind of thing was possible. I'm just used to going to my affordable hosting and paying the most a, monthly amount and.

It probably isn't for them. if you've got a small site with not much traffic and you can actually keep on top of that and everything's okay, a lot of those cogs are probably unnecessary. However, on the enterprise level, not only does that allow you to just more or less scale it, and you use the word infinite, which I think is.

Basically appropriate, isn't it? it can be scaled to what, whatever resources are available in the, the company that's got that cluster or what have you, but you can just keep going. So it enables a big website, which might get crazy amounts of traffic on one day of the year because they've got this giant sale or something, but also at the same time, locks it down so they, they increase their posture.

Their security posture or decrease their, footprint for security. And at the same time have this system where it can be rolled scaled, measure the traffic, if they need it, they can just add loads more servers on the fly and then roll it all back. And it strikes me that at the enterprise level, this kind of stuff is, I'm, gonna say reger, is this the sort of stuff at enterprise level, which is just par for the course?

[00:45:35] Tim Nash: It certainly is part of the course. The wider ecosystem. and you'll find this in places, especially places that need compliance. because you can't, for many, organizations, they must know, particularly, let's say a financial organization, like a bank. They need to know what state their server is in.

And, another one of these sort of hot topics is this idea of code as comp, compliance as code, which is where you basically make sure that your codes, you write, tests inside your code stack to guarantee that your stack matches the compliance requirements that you

and this is one way to do this and to guarantee it.

but even if you are, right back at the start, we go right our way by back to level one. Just knowing and understanding, Hey, I have file

[00:46:31] Nathan Wrigley: Yeah.

[00:46:33] Tim Nash: I can reset these to read only in for certain things. I can make my WP config file, which you are never ever gonna.

[00:46:46] Nathan Wrigley: No.

[00:46:47] Tim Nash: That

[00:46:48] Nathan Wrigley: One moment in time. Yeah.

[00:46:50] Tim Nash: yeah, there was that one moment where you went, oh yeah, I needed it just to the start. But it's not something that you're gonna write into on a regular basis. You can make that read only.

[00:46:59] Nathan Wrigley: Yeah, so even if you've just taken that out, if you've just got this intuition from this podcast episode that the file structure itself is something that you can tinker with. However, caveat mTOR, don't just go, don't just go poking around if you don't know what you're doing. Coming up with little three digit codes, oh, I'll put a six and a seven and another seven in there.

Let's see what that does.

[00:47:24] Tim Nash: I, if you are gonna do that.

[00:47:27] Nathan Wrigley: Yeah. Okay. Yeah, exactly. yeah. oh, that's really fascinating. Do you know on the WordPress side of things, is this, something that, enter, enterprise is maybe the wrong word, some of the hosting companies are moving towards? do you know if, and we don't need to name names, but do, some of the vendors of hosting in inverted commas, do they offer this as sort of one click solution?

pay us a few hundred dollars a month and we'll just do all of this for you.

[00:47:58] Tim Nash: Yes. there are certain companies do offer something similar to this, and once you are talking to the, Hey, come and ask us for a quote

[00:48:08] Nathan Wrigley: Okay. Yeah, the pricing tier, that's Let's talk.

[00:48:12] Tim Nash: Yeah. That if they're not talking about this, then that there's nothing wrong. I, a few years ago I worked with a, a company who were having a massive DDoS attack, and I was really surprised, but Amazon themselves came along and said, let's help you

[00:48:30] Nathan Wrigley: Oh wow. Okay.

[00:48:32] Tim Nash: Up something not dissimilar to this. They had a bit more few changes along the way.

and I was really surprised. Hang on Amazon, it will talk to, oh wait, yeah. Yes.

[00:48:43] Nathan Wrigley: Yeah, normally it's pretty, normally it's, hard to get them to talk to you. But yeah, I guess if you've got, if you're spending several, if you've got a lot of zeros on your annual or monthly spend, they'll probably, come and find you. Honestly, Tim, one of the nerdier episodes. We've ever done.

And, and I thank you for that. Can I just say though, you dear listener, you won't see this, but at the beginning of the episode, there were four horses and a few other animals in the field behind Tim. As Tim kept talking, they've slowly gone and now that Tim's be quiet, they're coming back and there's a human there now.

So all I can say Tim, him is, your, speeches, they frighten off the animals.

Yeah. Oh. So there you go. That was a real nerdy episode. Really enjoyed that immutable hosting. Don't say that you, you didn't find that out first from WP Builds. 'cause you probably did. Thanks Tim. Really appreciate that.

[00:49:45] Tim Nash: You welcome.

[00:49:46] Nathan Wrigley: Well, I hope that you enjoyed that. Always a pleasure chatting to Tim. Extremely knowledgeable. Isn't he? A pleasure, listening to all of those things. Hopefully you enjoyed it.

If you did, and you want to make a comment head over to WP Builds.com. Search for episode number 391, and leave us a comment there.

Just before we go, don't forget our black Friday page WP Builds.com forward slash black. If you're in the business of WordPress, and you have a product or a service or whatever, it may be hit the add a deal button on that page to get your product or service listed for free. And don't forget the sponsor slots at the top. So bookmark it now. WP Builds dot com forward slash black.

The WP Builds podcast is brought to you today by GoDaddy Pro. GoDaddy Pro the home of managed WordPress hosting that includes free domain, SSL, and 24 7 support. Bundle that with The Hub by GoDaddy Pro to unlock more free benefits to manage multiple sites in one place, invoice clients, and get 30% off new purchases. You can find out more at go.me/wpbuilds.

We're also joined by Bluehost. Bluehost, redefine your web hosting experience with Bluehost Cloud. Managed WordPress hosting that comes with lightning fast websites, 100% network uptime, and 24 7 priority support. With Bluehost Cloud, the possibilities are out of this world. Experience it today at Bluehost.com/cloud.

And we're also joined by Omnisend. Omnisend, do you sell your stuff online? Then meet Omnisend. Yes, that Omnisend. The email and SMS tool that helps you make 73 bucks for every dollar spent. The one that's so good, it's almost boring. Hate the excitement of rollercoaster sales? Prefer a steady line going up? Try Omnisend today at omnisend.Com.

And sincere thanks go to GoDaddy Pro, Bluehost, and Omnisend for their support of the WP Builds podcast.

Okay. That truly is all I've got time for. I hope that you enjoyed it. We will be back next week.

We'll also be back for our this week in WordPress, after a little hiatus because of a trip to WordCamp US, but we'll be back and golly, there might well be some WordPress news this week. If you have been keeping your eye on Twitter or indeed anywhere else, there'll be some news there. This Week in WordPress you'll find at wpbuilds.com forward slash live Monday 2:00 PM. UK time.

Okay. That really, really is all that I've got time for this week. Stay safe. Have a good week. Here comes some cheesy music. Bye-bye for now.