Don’t Push Rocks Uphill: Deploying WordPress with Capistrano and Composer – WPCampus 2017

Looks like everyone's here Charles Fulton presenting Couple things before we get started. If you have phones, please silence them we have surveys , hard copy surveys, some online fill them out afterwards. there will be time for questions after Charles is done. If you can, speak up so it so will be recorded If not, I may word out the question again so it will be on the recording. Charles will be introducting himself more thoroughly as he begins That's all we have to cover. So let's begin All right, good morning. Thank you all for coming I Talk today is entitled. Don't push Rocks uphill Deploying wordpress with Capistrano and composure these slides are available online if you'd like to follow along jump ahead of me, whatever My name is Charles Forton. I'm a web developer at Lafayette College mostly, originally a moodle Developer by training since about 2008 and Moved over wordpress administration in addition in 2013, which is kind of a surprise most unpleasant one Lafayette is small Liberal Arts College in Easton, Pennsylvania That's in the lehigh Valley right on the border with New Jersey on the Delaware River. I can see New Jersey from my office we have an enrollment of 2,400 Strong Commitment to open source and an engineering program which is odd for a college of our size. We do it to compete with Lehigh We're small enough that there's just one central or IT organization. There's no satellite IT or anything like that I'm part of the web development team. We come under IT there are three of us So we own platforms or say the branded website we own the platform But we work very closely with our colleagues in communications, who actually own the content and that's partnership, and it works very well Web Development owns many things this is a non entirely exhaustive list of the different platforms that we sell host and manage and that's three of us working together well two developers and a Manager and you can see it's quite varied. Yes. We really do own the college's entire MySQL Database infrastructure Which is a Galera cluster and some replica nodes, so we're generalists by inclination because our responsibilities are so varied All right, with preliminaries, wordpress started with single wordpress and new installations back before Proper multi-site in 2009. Faculty ,staff, student blogging this before I was at Lafayette and apparently my predecessors liked what they saw because when we moved from Static flat-file sites our first CMS in 2010 we did it with wordpress. wordpress 3.O, first multi-site release One thing led to another and the wordpress footprint at Lafayette is now rather substantial um Now obviously we don't have a hundred themes and plugins on any one site a very different sites at different loadouts but there's a lot of there's a lot of bureaucracy and a lot of overhead and What I'm going to be talking about. It's just how it is We're managing the deployed state at each of these 12 separate installations without driving ourselves completely insane So to give you sense of where we're going to go over the next 25 30 minutes or so we have a solution which we're using composer to manage the actual version history version state of each of our themes and plugins number using Capistrano to actually Deploy the tool what we like about this solution is that wordpress composer and Capistrano are not really designed to work for each other But each is very good at the one thing that does and we brought the three tools together and have a fairly solid solution so our considerations for deployment I'm going to talk a little bit about lafayette's environment,some of these are very specific to Lafayette and May not apply to your institution We harden the document roots on our web servers with SELinux, apache can't write to it's own document root That's great for security But it does mean that you know no upgrade button, no installing things from the interface. It's just right out We don't do that. So we need a solution that Gets us there differently, but as you saw from the previous slide. web development owns everything including our own servers. We have root We have shell we do not have to go through Intermediaries to actually Deploy things so that gives us some freedom and how we architect a solution We have discrete development staging and production environments they are on separate networks from each other They can't talk to each other well everything can talk to production because us in the public But we need to be able to have something where we can go to development we can go to stage we go to production and have the same outcome each time And I said we're generalists We have four moodle environments, whatever we built for wordpress. We want to be able to take the work We did and with some minor retooling Adapted to moodle Adapt it to the various custom Php projects we support As we ruled out a solution that was too specific to wordpress Git is really important in all of this this is not a Git talk. I'm not really talking about Git today That's totally different subject There's and I have a slide at the end of the presentation if a list of a whole bunch of really awesome get talks I'd like to thank WP campus slack helping me compile that list But I'm not going to get into the weeds on Git though in the beginning or 2013 WordPress was in a pretty bad place at lafayette to the point where we were starting to think about moving to drupal just because of the management problems that we were having With wordpress site. that obviously didn't happen, but we were just moving zip files around Manually on our remote servers when we need to do an upgrade Upgrading at the time we had ten multi sites upgrading them for next major wordpress, release was a six to seven hour process based over two maintenance windows One per week, and that's not accounting for overhead That's just the actual time spent moving things around and getting things where we wanted and doing the browser-based upgrade We'd had no version control in place. We didn't know what particular versions of theme for plugins we had like I said, confidence level shaky at best We had and continue to have an entirely normal directory structure this will become important later now core wordpress themes and plugins in the usual place, uploads directory about half our multi sites are of the right vintage to have at least blogs.dir in addition to uploads or sometimes only blog.dir, is you know keep changing? Which from a version control perspective is slightly obnoxious because you have things like themes and Plugins or thrown version history? but they've got to be installed inside the path of wordpress which has its own history and that made our heads explode a little bit and We needed to find a way to work around that that we didn't hate So we started with the idea? That you should treat each wordpress installation as its own software project So why not? Projects have dependencies, wordpress has dependencies. this is what's going to go into any particular wordpress installation and Those dependencies are going to come from a fairly wide variety of places most themes and plugins from but then there may be things that are wholly out on somebody's github or maybe in your own private github or self hosted gitlab or wherever or Maybe it or something like gravity forms which you're just buying off a website and downloading manually and then probably shoving it your own private github things can come from a lot of different places and the default state you have the wordpress installation, it looks something like this um Which is not great? It's not the best. I'm not quite sure what core wordpress is in this metaphor I think it's potentially uh I think this But we don't want that that and that's where we were in 2013 What we are looking to get is something more like this. where everything is nicely lined up, we know what's in each box we know where it's going and We can move things around with confidence. That's where this is where we're going to try to get to Managing the project we're not managing the code and this is the key difference We have a git repository. That's the project That's the particular each individual multi-site installation, but we're just going to track component versions. We're not going to actually commit code because we don't want to commingle all the history of say a theme, a plug-in or theme with Core wordpress and we as an organization. We want change and change management to be purposeful And we don't want it to be too easy to modify a plug-in or core wordpress In order to suit some particular use case that came up as if we're going to core hack as it were We want there to be a little bit of overhead like okay? We did this, we made a new version, we tagged a new version and we're keeping track and our internal documentation We did this thing we want to get to a point where we can't state exactly what version we have deployed in any one place So I'm going to start with a cautionary and make fun of myself a little bit talk about our first stab at solving this which is not the ultimate Solution we went with a git submodule based approach and This was it Yeah, I'm giving a minute I'm not here to not here to bury sub-modules But I am here to state that you probably shouldn't do what we did but with so git sub module is In a git repository you can say on Part if you're going to have this other project that this particular path, so if you can see here for example As you follow on line, I'm pointing at the screen You have a sub-module. It's a multi-site plugin manager It's a plug-in it should be installed with this path, and it should be installed from this repository And that's a way to incorporate a project with its own history into your project in this case a core wordpress installation and You simp and instead of actually working the code then you're just keeping track of the repository That was added where it should be installed and at what hash at. What place it is checked out Until you wound up, then with a fairly reasonable working repository structure you git modules value of Core wordpress, and you have You know it still remembers the directory structure of themes and plugins the problem is you are of course polluting the history of Core wordpress in order to do this the other problem of course is Presupposes that every single theme and plug-in and other dependency you are adding to wordpress is available in git For your private projects, for the premium things you download off the internet. This is true for themes and plugins off of course these are not managed in git You can download them as a zip file, or of course. There's the subversion repository That's used for publishing and making things available on now. I've made my peace with that as a deployment model as a Plug-in developer, but it does mean that you're going to have to do a little bit of extra work And so what we did was we set up a process to convert each subversion repository that we needed into a git repository This was a I'm here to say, I now regardless as a mistake Though it's a mistake because of the overhead the wordpress theme and plug-in subversion repositories I mean ultimately there's just two of them, but you only works for particular Path at a time, they're massive, there are tens, Maybe hundreds of thousands of revisions Importing a theme or a plugin from one of them in the initial import took about three or four hours each And that's assuming you didn't fail, because of a non-standard structure particularly older wordpress themes you know just any one random person setting up how to structure a repository had all kinds of problems getting certain order themes to import having imported it You create a private git repository you push everything in and you hope the tags that you got were somehow reasonable We did this for two years and it was just a horrible horrible slog Actually deploying when you have this project repository of all the sub modules so it's cloned out on the server you Initialize the sub models do a whole bunch of fetches and other magic And you have some shell scripts to help you do this and make sure you don't get things wrong And I never liked that because of course you're not going to ship for some shell scripts in The repository because that might be in a shell script exposed to the public which I don't know seems bad But it meant that there were things about deploying the project that were external to the project repository And that had its own set of challenges because it meant you might forget to do things So lots of overhead it was fast. It was absolutely fast and We had fairly high confidence level that these particular versions were checked out So that was good this say this saved wordpress at lafayette because we got our house in order But it was obviously not quite what we wanted so When we started to retool in 2015 we were looking for a way to bring the dependency management to the projects and Come up with things that were maybe a little lighter weight So to keep the metaphor going about I'm pushing rocks uphill if we could just add the rocks could just stay there and not have To be pushed uphill during a deployment That's obviously that I mean unless you enjoy pushing rocks uphill, show of hands, anybody do that for exercise recreation no one Good So we need a solution where you're overlaying the dependency management we are working with the projects with the dependencies not against them and Maybe not making too much work for ourselves because heaven knows everybody else is doing that for us So what we want to be able to get to is a situation where this is just a JSon Mock-up JSon file. We can just define our Repository worpdress installation, and then that's it without the additional overhead this very simple example we are saying we want akismet at what amounts to a reasonable version and core wordpress How do we get here? How does this actually become actionable? So composer is Act manager and in our defense. It wasn't really mature in 2013 when we started building the solution In this composer example. This is a composure.json file it contains Metadata for a project So you can see here. There's a name of a project description of it, and then all these dependencies. Where you have these key pairs defining a particular projects name and the version constraint and This is semantic versioning on the full discussion of what semantic versioning is is Beyond the scope of this talk But just saying is expressing confidence and saying well, maybe I need things that this version Maybe it doesn't matter. Maybe whatever version is available and each of these names is just a reference to a package in a composer repository in this case the central composer repository which is called packages and In this case a project that's registered on packages someone has simply said is the name of my project And this is a source control repository where this project can be found so them composer can ask Packagist. Hey? Do you have do you know about this project in this version? It says sure thing boss. You can go download it from here It's that way you're piggybacking on somebody else's version control, but you don't have to re-implement it for yourself Now of course this helps you not at all with wordpress because wordpress projects are not registered in Packagist So I'm sure a couple are but in general. They are not and for your own private projects a private based themes Private plugins you're not going to want to put them in a public repository it might not even be possible let alone why so you need way to make them available to composer manifest Well simplest way to do it is Composer will like you call just about anything a composer repository. So long as there's a composer.json file in it defining What the project is so you could just tell it here look at look at my private github go look here, and here's the code and that works until you have say 20 private projects Which winds up being a massive amount of overhead Especially if any of them change let's say for example actually, this is not a hypothetical you have six different multi-site installations at your college all using the base theme and the base themes and internal name needs to change You might have to go through and change that in each project. Make sure you do it, right? It's introducing a source for error and it's unwieldy There's a project called Satis Which lets you, so you take all those definitions Put them into a composer.josn file And that gives you a nice little interface web service that you have a composer repository that serves up all of your private projects So you put this you put all that nasty stuff in Satis And this is actually sort of extracted that this is a real example. These are all pointers to lafayette's internal git lab instance and then all that has to go into the project is simply a reference to wherever your Satis project is hosted. This is very flexible in addition to your your own private projects If someone is managing something like github that has a composer.json file You could throw that in there that way you're not having to remember Oh, where's this person's github repository? Just put it here and then it changes you change in one place and Obviously again elephant in the room this does not help at all with the theme and plugins on they don't have composer.json files and Yet, you don't want to be in a situation where you're maybe trying to you know copy them over again add them Fortunately this is also a solved problem and not by me Outlandish built A composer mirror for the themes and plugins database called wordpress packages or WPackagist It queries these with a web service and builds actually a list of again all the for all the name and version pairs so that this requirement w package plug-in akismet and this version has meaning it gives you that zip file and so add a stroke all themes and plugins so long as the Plugin author has adopted a rational tagging scheme and thus far my experience all but one has you're able you can use that to get the Versions of these particular packages so it's solve problem without jumping through a whole bunch of hoops so there again The other project repository you have a pointer to wpackagist you have a pointer to your own private installation if you need it, and now it's actually possible to Define all the wordpress theme and plug-in dependencies that your project has In composer with these version constraints without actually having to manage any code in your project repository you're just managing the state I left wordpress core to the end It's possible to manage wordpress core with composer. We chose not to Mostly because there are some interesting situations that happen of trying to do a full site install with composer. It's a bit Arcane I have a link to a blog post at the end of this slide in which I discuss exactly what the challenges were, but We opted to keep wordpress in a sub-module using the official github Clone of WordPress if you go down this path you'll soon become very aware that when wordpress releases this lags behind by about 30-45 minutes But so a project so far with core wordpress and all our themes and plugins is simply a composer.json file and the sub-module Core wordpress, and that's it. Which is very easier manage So how do we actually like this sounds great fine? Thanks? How do you actually get this on a web server and do something with it? Tool for this is Capistrano It is a ruby application because of course we're going to we are going to deploy a Php app with ruby so this might be considered an off-label use I like to think of Capistrano as telling a Deployment or developer story because in it you put all the information that's necessary for deploying an application As you can see our project has suddenly gained a whole bunch of extra files Things like the capfile the Gemfile are just kind of helpers that I'm not going to go into today but in deployed on our beats just a final in which you tell Capistrano various things like what's the name of your Project what git repositories is it located in What path should this project deploy to on a remote server? What tasks should I run should install composer module should it? Initialize a git sub-module should you do various NPM tasks are there shell commands, all these things get encapsulated in this file and then within deploying defined environment so you can say and my staging environment is on these servers and You should do these additional things from deploying stage And you should use this git branch by default when you're deploying to stage Just to prevent yourself from accidentally doing something that you shouldn't have done another thing we like about Capistrano as implementation is when you do a deployment there's no, so I deploy off my laptop I Didn't need to do any additional configuration on my laptop to do this beyond cloning the repository and setting it up But there's no laptop specific configuration. There's a server-side Configuration, I'm able to deploy this project because logged in to my laptop as made with my ssh keys I have the proper access the web servers, and I also have the proper access to our remote git repositories which Capistrano will be downloading from as it installs the various composer themes and Plugins So if we onboard with the developer. We just mean tell him or her get the repository, and that's everything you need to know We wouldn't do that because that's cruel, but we could So on actually on the remote in a Capistrano deployment That's getting back to saying I was talking about with eventually with the challenges with having themes and plugins inside Core wordpress An issue that we had with some model-based approach is when you're doing that you things like your uploads directory sitting inside Kind of untracked things in the git repository It's kind of messy and you also run the risk you're doing things with that repository of accidentally blowing away your uploads which would be bad I think so nice thing about Capistrano is it does a lot with symlinks So you've defined things like your htaccess file that you can fake upload you put them in this directory That's outside of your deployment route and every time you do a deployment Capistrano Symlinks them into the correct place, so Your your WP config is where it's supposed to be so speak the file actually lives outside the document root Yeah, like I said it loves symlinks, so you do a deployment and all of a sudden there are symlinks all over the place The real Magic real Deployment Magic is happening in Releases because every time you do a Capistrano deployment you get a time-stamped Sprinkle a timestamp check out where it puts all of you know put over wordpress, your themes, your plugins everything goes into this one Timestamp directory, and then when it's done it Creates a current symlink that points of that where it previously pointed at the last known good release means It's very easy to rollback if you realize you did something stupid and so they roll back and it Just takes that symlink can point it back at the previous release and then flags that release you just did as bad So won't try to go back to it again That was a capability. We just really didn't have with sub-modules. I mean we could Yeah, if you were to roll back sub-module release you'd start checking out different hashes and some of the sub modules and it would Get very bad inconsistent Not that we've actually had to roll back So if tasks it's a pluggable infrastructure, obviously Capistrano does not support composer out of the box But there's some helpful person on the internet who maintains a module that does just that Again sub-module support to nPM. There are any number these are just the ones that we generally have to use in wordpress architecturally It's really not that complicated to Deploy this is what typical environment file might look like and This is a case where Capistrano let us fall into the capability We didn't even know we wanted in 2015 the ability to do a high availability Deployment which is almost live at this point because if You define multiple servers Capistrano will execute the deployment tasks in parallel with the final symlining occurring simultaneously on remotes so we're never in a Situation where there's one version of wordpress deployed on server a and one version of wordpress Deployed on server b. that was a problem that we realized that Capistrano was solved the minute We started talking about how I availability we never had to think about it Similarly it lets you define roles we can do this these words to be anything just but App and web for one just web for another which is a helpful way for us to say to Capistrano You've got a task of running the WP cli database upgrade command please only do it on one server, not both That said, I don't know what would happen if you tried to do that I don't know if there's proper locking or not, it might work. I said This at some level this is deployment of documentation Everything you would need to know about deploying this site in theory is located in these various configuration files So now there's no we don't have any external tools that live somewhere else that are necessary to manage this process I Mean we do have Additional documentation, I wouldn't actually just throw this repository a new developer and say hey, please go to redeploy the production website tomorrow But ultimately everything that's necessary to do that is located in those in that repository So now I'm just going to quickly step through certain. What certain tasks look like in this environment So power press, power press has new releases all the time any other power press users Good Lord, what do you use for Podcast But in this case we can just do a composer update on that name and it goes out goes to Finds that yep. We've gone from 7.0.3 to 7.0.4 Downloads we update the launch file push deploy, and it's done That's all we had to do this process takes approximately 30 seconds And it's also depending on the version constraint we could set a version constraint though. It's just like 7.0.*. So we're just getting the incremental releases so again this depends on the person having a responsible Tagging scheme, and we've certainly been bitten by that before That's the command to redeploy production. I thought about doing it before I found out it wasn't presenting off my own laptop But live Demo seem bad live demos with production. You're just asking for it But taht's the command to redeploy production now. We it takes about three minutes We're trying to get find a way to get the sub-module piece to run faster. It's the it's the bottleneck, but that's it Database upgrades same idea we just define Define a shell tasks to be run on the remote we use WP Cli to your database upgrade and Then I can just run it from my laptop whenever I so choose and I never had I'm never actually having to go on a server get on worry about paths Remember the exact incantation for the command because it's not necessarily something I do very often And realistically. I'm doing this what three times a year for each multi-site installation probably at about the same time This is particularly handy We had a problem we have base theme with some fairly complicated cSS and Javascript And when it was handed off to us from the shop that developed it the minified Javascript and csS Versioned in git, which is not completely unreasonable, but it's obnoxious Because you have code you have merged collisions all the time, and you can't tell what changed Because you can't diff a minified file And we identify that something we wanted to fix And we realized eventually that we could do is just define a grunt task that does all the minification told Capistrano yeah every time you redeploy this site, please run, please step through that and just Please minify and release each time because you'll forget one time and break production because production will have no theme We did this uh about six months ago. It's been completely rock solid sped everything up So having gone through that um the deployment is about as fast that was the sub model-based approach which makes sense a lot of the same kind of activity but the overhead went from very high Essentially none just so long as there's nothing weird about the them or plugin and our confidence went way up Because it's so easy to deploy Same thing is basically the same command deploy to stage, Deploy production. You're going to get the same result each time and Where we used to only do kind of big catastrophic deployments. Maybe every month every month and a half? We'll deploy during the weekday. Weldd deploy during business hours we'll deploy whenever is we actually need to because something is ready to go out whether it be a Theme update a plug-in update just that the only exception is we still say we still save the big wordpress Core updates for Maintenance window out of an overabundance of caution And and also just really stress on the database So again our approach here is that we are managing Versions and state we're not actually managing code in a project repository Project repository which is keeping track of what we have and where it is With I like to think a minimum of bureaucracies, you can't eliminate bureaucracy, but you can eliminate unnecessary bureaucracy and He was we got deployments fast. We got the overhead low and we raised our confidence level Which was ultimately the goal so, few resources I wrote these three blog posts are publicly available just mostly just Recapitulating in text form a lot of the things I said they're also going certain examples and details and also an extended discussion of why I should not deploy core wordpress with composure these are a bunch of great talks on git and Development mostly coming out of the wordpress community but some just more general as I said this is not a git talk But if you're unfamiliar with git you will need to do that before dealing with some of the other things and I've outlined today That's the link for I believe this correct link for leaving feedback this presentation as I said is publicly available Thank you so much for coming I really appreciate have such a large group of people coming to hear me speak today And this time I'd be happy to take any questions that you might have Yes, oh I'm sorry. Yes The question was how big our team is the web development team and at Lafayette under it is Three people two developers and a manager who also does some development tasks our Communications department has a digital has a digital communications team of three who do design work High-end content, and then of course we work very closely our systems administration team, but they're not directly invlovled in the Web stuff, css a good point We needed something that can be done by Both myself and the other developer and it couldn't be too involved because they're just not very many of us and we have a lot of responsibilities Yeah good question so question was given the growing maturity of WP Cli would we consider using that to manage updates I Mean the first answer gee I haven't looked at that part of WP Cli Functionality enough to know whether I would I think the answer is still being no Because I like being able to look at a project repository and see what my version Constraints are and just see what the deploted state is my project particularly over time so I mean Wp Cli is coming a long way and doing fantastic things and we use it for a lot of administration tests I don't think we would use it to manage updates But you absolutely could that's absolutely a valid solution Anyone else? Non Capistrano, but we put a few real code for wordpress in regars to moodle Oh, yes, thank you, so yes, so As I say we needed a generalized solution solution you may have noticed that really none of the none of this was really specific to wordpress except kind of wpackagist even then that's A general solution pointed at a particular thing So we took this strategy and used it for our self hosted moodle environments any and same thing using composer to manage the Dependencies using Capistrano to Deploy and to get around the question of a moodle Plugins generally have their own git Repositories, just because how that community does its development, but again though compose.json files I worked the collaborative liberal arts moodle project to build a composer mirror for the moodle theme and Plugins repository Very much like wpackagist and that's something that lafayette maintains Yes, we're able to apply the same solution to our project with very little Retooling and for us that was a huge win Thank you. I did not plant that question So the question was how do we manage updates to composer and Satis themselves and are we worried about our external dependencies, so Yeah, Satis is an open source project It's managed on github so from time to time we download updates to that and redeploy the project We manage the product that project with Capistrano and composer. Which is a little little recursive composer itself course the Php binary and From this needs to be done a little better We should be managing this with puppet or not, but just from time to time. We update the binary on our remotes The Internet's built on baling wire as we all learned from the various Amazon outages. There's not a lot of stuff We've got that just does not work if somebody has problem um It's a risk that we've accepted and realistically You know you don't know what's going to break. I mean if goes offline my wish that we did have all those old clones of the Themes and Plugins so that we still do a deployment, but we could also just wait for it to come back up There's only only so much risk mitigation. You can do before you drive yourself insane Certainly it if composer or Capistrano were to cease to exist. We eventually be forced just a passage of time to to architect a new solution I Don't want to say one thing at a time because that's kind of flippant, but if it happens it happens but they both seem like fairly stable Mature projects and With a lot of people who are more important than us using them, so it's a risk We're willing to run Anyone else, yes. Yeah so The way to do that, and there's a lot of people doing this I mean I'm not not talking about anything new or revolutionary. Oh, I'm sorry yes the question question was I could go into detail Why we're not installing wordpress core with composer And a lot of people do, if you look at the WP deploy or the Bedrock projects is the things they do But they also do that by changing the way the wordpress diagram root is organized, moving things We didn't want to do that because we were overlaying this on our existing deployment And we didn't want to undertake the engineering to do that and the reason they tend to move those things is if you Install composer, to install a full site like wordpress Because of the order of operations it winds up destroying all the symlinks that Capistrano put in so then you have no WP config file it just We kept running into problems, and we said nope. This is stupid So we know what sub modules are this works and also note that it was easier for us to Occasionally hack wordpress core which we have done from time to time So that's what I mean is a combination of factors, and we do the same thing with moodle for essentially the same reasons Yeah, so the question was if we were starting today would we use a build system instead of Capistrano, maybe I should mention we actually did come up with a way to automate this But my boss who's not here won't let me do it because he's afraid of it, but it's not quite true But we could we have gitlab Ci for continuous integration, and we could and we had a container set up with Capistrano so we could just push to it, and then it would redeploy on the remotes, so we have automation, but the lack of automated acceptance testing for wordpress Led us to not pursue that Because we want without that since we've got the ability to automatically detect a um Fail deployment. You know beyond things on point like fail deployment. As it. Yeah the codes. They have a wordpress doesn't actually load We decided to keep that part of it manual. I don't I don't know to be honest, maybe So the question was why not automat and verify on DEV Part of the answer is specific to Lafayette Network Topology dev is in a really weird place and the Ci server can't talk to it and also I Would say that there's a certain Institutional reluctance to have a process that works differently depending on Network environment, that at some level, that is you can say well, it works this way with Devon stage but then you do this thing with production we've tried to avoid having situation where that was the answer because It just goes to a long-standing concern about having inconsistencies between the environments So I'd say that's why we could do that absolutely not thought about it, but at the moment. That's not something. We're pursuing Please feel free to catch me in the hallway

Leave a Reply

Your email address will not be published. Required fields are marked *