Press This:The Importance of Automated Testing in WordPress

Press This:The Importance of Automated Testing in WordPress
Comments Off on Press This:The Importance of Automated Testing in WordPress, 11/05/2022, by , in Wordpress

Welcome to Press This, the WordPress community podcast from WMR. Here host David Vogelpohl sits down with guests from around the community to talk about the biggest issues facing WordPress developers. The following is a transcription of the original recording.

Powered by RedCircle

David Vogelpohl: Hello everyone and welcome to Press This the WordPress community podcasts on WMR. This is your host, David Vogelpohl, I support the WordPress community through my role at WP Engine, and I love to bring the best of the community to you hear every week on press this as a reminder, you can find me on Twitter @wpdavidv, or you can subscribe to press this on iTunes, iHeartRadio, Spotify, or download the latest episodes at In this episode, we’re going to be talking about the importance of modern tooling and automated testing and WordPress development in particular, of course and joining us for that conversation. I’d like to welcome to press this Mr. Josh Pollock. Josh, welcome.

Josh Pollock: Thank you. Thank you for having me. How are you?

DV: Good, good. I’m really excited. to have you on the shows. We were talking kind of before the recording you’re the most, I believe the most famous WordPress Pollock of all the Pollocks, right?

JP: comparative to Jackson Pollock less famous, but yeah, so it wasn’t

DV: Not nearly as WordPress famous as you are. So I think like you’ve got Jackson there.

JP: right? Yeah. I am a fan of his work though.

DV: Ah, that’s good to learn. For those listening though, and Josh is going to talk about today. Josh comes to us from a few different areas but particularly focused on plug in machine. We’ll talk a little bit about that, but also around his thoughts about why modern development tooling and automated testing are super important for WordPress development. So if you’re a cowboy or cowgirl coder, Josh is going to talk a little bit about why another path might be better. Some of his favorite tools for that and how to approach automated testing with your own WordPress development projects. Before we jump into that, I’d like to remind folks I know you’ve probably heard this on prior episodes. On April 25 2022. WP Engine will be holding our virtual decode conference. Like to check it out. Learn more about WordPress development on a variety of topics. You can visit events at WP forward slash decode dash 2022 Alright, Josh, I’m gonna ask you the first question I asked all my guests. Could you briefly tell me your WordPress origin story? When was the first time you used WordPress?

JP: Yeah, I use WordPress probably for the first time. Let’s say 2011 2012 to write a blog, like on in then I you like Googled how to do something and they were like paste something into functions dot php. So I had to like move to self hosted WordPress and I don’t think I really ever did much work on that blog. I got really distracted by like the code part. And that led me into like the WordPress community and volunteering with like the theme review team and then I am going to job at pods, which is a plugin that does like Custom Fields and Custom Post Types and has a UI for it and I got a job as a support person there. Scott Kingsley card, the lead developer there and everybody else really supportive and helped me learn like development and I got really into plug in development from there. That’s cool.

DV: When did you first get distracted by the code you said 2011 or 12 was when you tried to start a blog? Was it like shortly thereafter or?

JP: Yeah, yeah, so I’ve been that was like WordPress 2.7, I think was the first version I worked on. I mean, I think the first version like I used and I think the first version I contributed to was like three dots.

DV: So yeah, 2011 this would have been right after custom post type. So that would have been an exciting time to be in WordPress. I do think out of 237 some odd episodes I’ve done. You’re the first origin story. So I think you might have the distinction of that and all the folks I’ve interviewed these years but that’s pretty cool that you got kind of started there. In the blogosphere. And then quickly kind of moved on to the development side as really interesting. I mentioned earlier that year with plugin machine could you tell us what plugin machine does and what you do there?

JP: Yeah so this is a new product that I’m developing, you know, I mean, it’s my you know, it’s my solo. thing I’m building like I’m a freelance developer now, like, the in between parts of the story is I met some folks that were working on a plug in called cutter forms through pods at a company surrounding that for a while. And then I did a couple of jobs in the WordPress space and now I’m a freelance developer. And a lot of times I’m building WordPress plugins for WordPress sites. Like I’m not somebody that you would go to and be like, I need to build a site for my business. Like I’ll build the cool functionality for it. Somebody else will build us a theme or whatever and for that in for my own projects, I kept getting hung up on what were how do I start this plugin? Like I want to have blocks and I want to make sure that I’m using WordPress scripts, like the new tool for compiling JavaScript code, including React code and the way that works is compatible with WordPress. Right away. It’s the WordPress standard. It’s where he was like forking the last project that I worked on over and over again.

DV: That’s the start of all great software, isn’t it?

JP: Yes. So that kind of led to what I’m now calling plugging machine which is a tool that does a few things. First thing is it helps start plugins like it creates all the code you need with all of the correct naming conventions in dependencies to do things like use composer for PHP autoloader in dependencies or using the WordPress scripts for your blocks. All these different types of things. And this has gotten me I’ve always kind of been obsessed with the moving parts of bug in development, the automated testing, creating the right zip file that has all of the correct files that you want in there, but not the ones you don’t want. Like your tests you want to want to install. So I’m logging machines, this sort of complete tool for starting plugins, adding features to plugins, I need to add a block and the Data menu page and then creating like the final package version that can go into WordPress site.

DV: And commit development framework for plugins. I guess if I had to, like use just a few words to describe it. Is that fair?

JP: that’s great. I’m gonna write that down. No, no, this is what part of why I love going on podcasts like like you’re you’re you’ve got a great way to simplify it like it’s a development framework for WordPress plugins, like it’s a hosted service and a CLI that you use to have a UI where you can click like I want to use custom post types and I want to use blocks and then when you’re in your plugin, you can you know, type quick commands like plug in machine, plugins, zip grants, a zip file of your pocket.

DV: Those sorts of things. So I love it when software of course kind of originates in need and kind of accustomed way and it’s kind of interesting to hear the origin story of plugin machine appreciate you’re still kind of holding on to it but kind of coming out and you’re kind of homegrown approach and so like your journey started with, right, like literally no code type website will quickly get into kind of more advanced development. So help me understand our audience even understand like when you talk about, quote modern tooling with WordPress development what does that mean to you and why is that important?

JP: Yeah. So just for the context of how I’m approved of who I’m approaching says, I’m, you know, a WordPress plugin developer, you’re gonna web application developer, I’m talking about building WordPress plugins, custom plugins for your site. That might be unique plug in that does something that doesn’t exist, you know, it extends WooCommerce in some unique and interesting way for your business, or plugins that people are developing to distribute open source or to sell. And so when you’re doing that, you don’t you know, like when I started building plugins, like Hello, doubt, it’s just one file. You don’t really need anything special for that. But as they grow in complexity, you start having automated dependencies, right you want to use WordPress scripts packager, you want to use WordPress components package that you’re going to use the same inputs and select the sort of form controls as Gutenberg knows. As you do that, you start to need something that can automatically install it something like NPM or yarn and then you need a tool that can automatically run the build process to create browser safe JavaScript, web scripts. And then you start getting into what we call automated testing, where we’re writing additional code that runs our code to make sure it runs the right way. And there’s a lot of complexity there and I think that makes it hard because you’re learning these tools, and you’re learning how to use them at the same time and learning two things at once

DV: It was interesting because to hear you describe it, use the word automate it for every single bullet as you talked about modern to like from, you know, installing packages and dealing with dependencies and then running your kind of testing suite. And it seems like you know, if you haven’t done these pieces along the way, you have to learn both the automation and what the thing is doing to the software that you’re creating. And I could see that being, you know, a huge challenge for a lot of folks. I’m curious though, like about that journey, and maybe how folks can kind of get over that. We’re gonna take our first break. We’ll be right back. Time to plug into a commercial break. Stay tuned. For more, press this in just a moment. Over everyone, welcome back to press this WordPress community podcast on W EMR. We’re in the middle of talking to Josh Pollack, about the importance of modern tooling and automated testing and WordPress development. Josh, right before the break, you were kind of explaining modern tooling. You kind of had kind of gone through a list of a list of kind of key components of it. You kept emphasizing the automation prior to us going and making the point that people have to learn both kind of automated approach but also kind of what the tools are doing. Was that a challenge for you? As you started to adopt this type of development?

JP: Well, yeah, totally. In I’m a nerd for like, people. I hate that part of it. I enjoy it. But when I’m trying to write a feature, right when my brain is in the space of, I’ve talked to a client and they need this thing to look like this and go here. I want to like write PHP and JavaScript code. I don’t want to write a, you know, automated test setup or configure NPM even though I’m a nerd for that stuff. I don’t want to deal with it when I’m in the brain space of doing like doing the actual work. I’m in so that was always the challenge for me because I was interested in both, but it’s like everybody, it’s hard to do at once. So what plugin machine does for me and I, and for other people like me, is I click a few buttons and I get all of the parts setup in there all set up to be automated. And when I say automated, I mean things like I can have one command and the readme that sets up the local development environment and allows me to see it in the browser and run tests that I have GitHub actions that every time a change is made run the tests in in different environments with different versions of WordPress, different PHP versions. to automatically check if my code is cross compatible with all these different things. Because that’s a challenge in WordPress. And I, as a developer, don’t have to think that hard about it because it’s a few button clicks to get set up and then you look in the readme cut and paste this one command that does a bunch of stuff. This man does much stuff that allows you to see the why

DV: Like it’s a lot of extra steps, a lot extra things to get your head around. It’s great to use kind of put off the shelf frameworks to get you closer. But like what why like why go through all this trouble to integrate quote modern tooling into your development process.

JP: So for some things, it’s basically a requirement. Like if you want to use React inside WordPress for something like block, building like a cool admin page for your for your plugin, having a front end interactive element you’re going to need to use the correct WordPress tools to compiled in a way that won’t cause compatibility issues with other react based components in the WordPress site. So you’re, the more and more It’s becoming like effectively a requirement right like you might want to, you might have a plugin that’s been out there for a while, and you need to make some changes, but you don’t want to break the things that features that already exist. The best way to deal with that is to write automated tests that describe the way it works now and if you make a change that causes one of those tests to fail, stop back up, you know, fix that error instead of shipping it to your users. That’s another case where it’s like your, your need to make your customers happy and have a stable product becomes the need.

DV: Okay, so this isn’t really interesting because there’s been a lot of discussion about this recently in WordPress, which is, as Rob Stinson, one of my co workers here at WP Engine points out the easy things in WordPress are getting easier, like the block editor and the hard things are getting harder, like making a plugin and you’re kind of observations there on you know, kind of more advanced development approaches than the past relative to even incorporating react little in like using it as a framework. So that really rings true. And it sounds like though the benefit is your time from through particularly with things like automated testing from having to like recode stuff you shipped that broke and I’m guessing also like keeping your job if you’re or your clients if you’re shipping, lots of breaking changes and guessing that’s also a benefit like there’s a monetary benefit.

JP: Yeah, like I’m a person on that machine. Or this is sort of the joke behind plug in machine. Like I’m the name instantly. I’m not good at doing the same thing over and over. Same exact same way. Right. That’s why we use computers. Like we’re just like, Hey, I’ll tell you how to do with him. We call that code. And then we just run it over and over again trusting computers to do the same thing. Over and over again, the same way in so This to me is I don’t want the anxiety of what if the change I made broke in so I could manually test it exactly the same way every time and that as I said that time right and that’s human error or I just have a program that runs 48 And the more that I think we make it easier for WordPress plugin and theme developers to have automated testing without like, Oh, I’d love to but I don’t have time to figure out how to set it up. I’m the more that those easy things that are easy for the end user will be stable, right? Because it’s not just that we want the UI to be easier to use or easier to learn. We don’t want people like this is the complaint about WordPress. You get your site going and then you update your plugins. Right. Like this is the thing that everybody has problem with as a user level. It’s not something that we can solve for users directly. It’s something that we have to solve in the way that we build WordPress plug in from the way that we test that

DV: so like deadlines are always like your expectations. Like when do you want this tomorrow? Right? I don’t think anyone has ever not said that to me like, oh, we need it in six months. No problem, right. It’s everybody wants everything the next day. And so, teams are under this pressure. I’m just curious how you think about writing tests or test suites, you know, in giving people kind of a crawl, walk run, are there some key areas or do you do you like to start and like, try to write what you feel as a complete test suite or you try to pick it off in certain parts as as people are learning like, how do you how do you recommend? tackle it like go for the full test suite? Take off a chunk and then learn that way? Or how do you think about that part?

JP: This is a great question. I do this kind of consulting with folks sometimes where I like look at their code and not just set up automated tests, but like work with them on coaching them on what they should be testing. And a lot of times this is one of the things holding people back is they feel like sort of guilty that they don’t have any tests and then they can’t have complete test coverage. Right. And I think that’s a weird way to approach it because it’s like, you haven’t done a thing yet. Of course you don’t have the thing the result of the thing. And you haven’t written the test, you know, test but the tests are useful, even if they don’t cover everything. I think that’s the really the, the anxiety people have is I’m not going to get full test coverage. If I just write a few tests. It’s like, yes, but you’ve got yourself a step closer to that. You got started on it. You got an opportunity to learn how Tesco so for example, I have a plugin that I wrote for a client that adds a shortcut. Like that’s all it does. And so I wrote to to and it has the you know, if you’re not logged in, it’s shows a message to you about logging in. So I wrote two tests, both of them just call the function that’s renders the shortcode in make sure that it doesn’t throw an error. Those are the world’s most detailed tests. But when I first committed them after that my first pass of writing the plugin of testing and I had like a whole bunch of errors, just from running those tests, like just in the process of generating the shortcode I had generated a ton of PHP errors and I was able to work through and get those to go away. And then that gave me the confidence in the future that if something so one of the three or four different parts of what goes into that shortcode break. You know, that’ll fail the test.

DV: So that sounds like you’re thinking about it in terms of like the key functionality of the software you’ve created, identifying those key functions and then kind of writing tests. around those to start in order to isolate where in your software problems might be arising. Is that Is that a fair way to bring them?

JP: I would say because, yes, because that’s starting with two tests that say something broke, like really good test coverage. You would have like one test for each individual part. of the program. And so it’s like one test fails and you’re like, okay, that tells me exactly where I need to go in the in my codebase to solve. Maybe you’ll get there maybe that’s a way to develop a new product plugin. But if you have one test that does, you know, your shortcode it does your make sure that your blog can be able to add it in the post editor. Make sure that your form can be submitted and doesn’t make any errors. Those cover so much in then in the future when those break, you know, they fail for a specific reason. Then the second type of thing that I like so that’s like the first phase. The second phase is next time there’s a bug right test that fails because of that bug in then can pass once you fix the bug because now you have a little bit more detailed in your testing, and you have proof that you fix the bug and you have protection against happening again in the future.

DV: I like that see you’re kind of using future bags as they pop up as a way to add more test coverage and, of course the areas that needed most right the things that are breaking. It’s a clever way to engender that as a great suggestion. I want to dig a little deeper here and talk about this culture of Wild West coding and WordPress. We’re going to take our last break and we’ll be right back. Time to plug into a commercial break. Stay tuned for more pressing this in just a moment. Everyone welcome back to press this WordPress community podcast on WMR. We’re talking about modern tooling for WordPress developers with Josh Pollock. Right before the break we were talking a little bit about how Josh thinks anyways of approaching you’re kind of writing your test suites kind of focusing on the most critical functions first. I really liked your suggestion Josh about using kind of bugs in the moment to beef up your test suite. I thought that was really clever. You know, WordPress has a culture of cowboy cowgirl coding, if you will Wild West getting your dick also called Do you see a culture of automated testing starting to take root is this notion of like the quote hard things getting harder, like making modern tooling. You said also a requirement but also this notion of automated testing. Do you feel like that’s taking root in WordPress or do you still feel it’s like you know, the five minute install and 10 minute website?

JP: Well, it depends on the project, right? Like there’s something wrong like it’s great when you can do like plug in Like the one website with WordPress is really building myself right now. I don’t have any of that under version control. There’s no customers like there’s a little bit of custom code like that. I like modified into Hello Dolly. Right. It’s just like off the shelf stuff. But a think like I used to work at an agency. They’re all of the deployments were automated, right? Everything was being checked into version control, using a pull request workflow. And then when you merge to one branch automatically deploys the site. That’s gone an easier, there’s a lot more tools for that. There’s a lot more hosts that, you know, support that and have, you know, documented ways to do that and getting automated deployments for like, if you’re building a whole site. I think that’s a great first step because that’s an opportunity to standardize that part. And then in you know, in that pipeline, start adding tests to that. I think more and more people are doing that. And, you know, I use an FTP client to drag, you know, files up to a server earlier this morning, because sometimes it’s the only way to do it. Um, and I think, yeah, I think it’s getting better but, you know, it’s not easy enough for people. Like it’s not, I think, because like composer isn’t a native concept or press corps that makes it more difficult or NPM. I think there’s a lot more work to be done in that space. And like I’m really I’m super interested in that way. If I had more time I would consider the problem of composure isn’t. Works with WordPress. Lead isn’t a great tool. It works for the whole project, but like it can’t recursively install dependencies those kinds of problems

DV: But what about like your test? Your testing do you use like IT folks here because I feel like that’s a big blocker for a lot of people trying to adopt modern development was is like, you know, what, test suites can I use? What testing tools can I use? I’m just curious, like, if you have any recommendations for folks listening around automated testing and tools or testing suites or frameworks for WordPress that they could consider

JP: well, so for WordPress developers you’re doing like plugins. You don’t I just use PHP unit which is the standard for WordPress plug in it for PHP testing. And then there are some tools that have been developed by Yoast that are open source and as part of for that make it easier to test with PHP unit and those all gets set up automatically on your machine. And then for JavaScript jest, which is like React. I mean, that’s part of the WP scripts. Those are the things I use but I also for people who might be doing site development you can do automated testing, with tools like go students specter or Checkley, where you’re able to program the test by like, you have a browser extension. In then you go into the browser, you click record and you do things like log into your website and add a post or add a block or that kind of thing in then they can run in on your website, every you know, once a day or once a week. And that’s a way to do no code automated testing, that you know, people who own websites and they’re you know, it’s really big when your client sends you a message and says, Hey, this page is broken. That’s like not a great way to start an interaction with your client. It’s so much better to say hey, our automated check notice that the last update broke the section and we’ve already fixed it.

DV: I think out of all the modern workflow episodes I’ve done over these years. I don’t think one person is ever introduced that concept of that. That’s really clever. This has been awesome. Josh, thank you so much for joining us today.

JP: You’re welcome. Thanks for having me.

DV: If you’d like to learn more about what Josh is up to maybe expand your own modern WordPress developer journey check out Thanks everyone for listening depress this WordPress community podcast and WMR. This has been your host David Vogelpohl. I support the WordPress community through my role at WP Engine. And I love to bring the best of the community to you here every week on Press This.

About Vikram Rout

Vikram Rout has been a blogger, digital marketer and an SEO expert at, one of the fastest growing custom design crowdsourcing platforms. Over the years, he has been helping small businesses and startups improve website design and SEO strategy, content marketing and user experience. You can engage with him on here.