Preparing custom and contributed code for Drupal 9.

>> MICHAEL LUTZ:  All right.  Hello, everybody, and thank you for being here and welcome to preparing custom and contributed code for Drupal 9.  My name's Michael Lutz.  I'm a Senior engineer at Four Kitchens and I want to give today a brief runthrough of the state of Drupal 9, where development is at and try to give you all some idea of what's going to be involved in preparing code you maintain to run on Drupal 9 along with most importantly showing you what steps you can take right now.

So let's start just touching on where Drupal 9 development is at right now.  It's very exciting we have a firm release date.  Drupal 9 will be released on June 3, 2020.  We had a few potential release Windows and the core Developers managed to hit all the requirements for earliest potential window on March 13th on committing the Last Mile stone that was required to release Drupal 9 and beta and as of now that is scheduled for release today or tomorrow.  I think it would have been out earlier this week    pushed it back a little bit.

So now that we reached those milestones on time we're going to look at getting RC out preferably early May and Drupal 9 will be released on time on June 3, 2020 and at the same time we're going to release the Drupal 8.9 long term support the LTS edition.  On top of all that the Drupal 8.9 beta will also come out most likely next week and at the time that beta comes out we'll branch off Drupal 9.1 so even as we finish ramping all the release development of the next iteration with all these settings and features that everything that comes along will be starting next week.  So now we know when Drupal is hyper converge 9 is coming.  Let's get to this presentation.  Then comes the question of upgrade.

Today this presentation will be focused on the first group, people that are maintaining sites.  However a lot of what I talk about especially when it comes to preparing code and updating code so going to apply equally to both groups so this presentation should be useful to everybody especially when we start talking about deprecated code [ Inaudible ].  In order to successfully upgrade a site, and I really want to stress that this prose is an upgrade.  It's not [ Inaudible ] data migration.  The steps needed to perform a successful upgrade there's a few of them.  And the really good news is you have everything you need to do most of these steps right now in the Drupal 8 site in order to be ready to hit the upgrade button on June 3rd.

And that's really one of the take aways I want everybody to get out of this presentation is that it's not about waiting till Drupal 9 comes out and then doing something.  It's a whole bunch of things you can do right now so that the day Drupal 9 is released you're ready to go.  First thing you want to do is make sure your service supports Drupal 9.  I'll give you details but Drupal 9 will have different server requirements than Drupal 8.  You'll want to make sure any custom code you have, custom modules are compatible with Drupal 9 and that will be the largest portion of this presentation is dedicated to how to get that code updated and ready to go.  In addition to your custom code you have to audit your contributed modules you're using.  Make sure they're ready for Drupal 9.  If they're not, [ Inaudible ] we'll talk a little about that when we get there.  Finally if you're on an older version of Drupal 8, you need to upgrade to Drupal 8 first because you'll only be able to update from Drupal 9 to one of those two versions and once all of that is done you're able to upgrade to Drupal 9.

>> DWAYNE:  Michael, a quick interruption.  Is it possible for you to move closer to your mic?  We're getting a couple notes that you're a little bit fading.

>> MICHAEL LUTZ:  I'll try to speak a little louder, if that will help.  Sorry about that.

Speak a little louder here.

Anyway, the last step obviously has to wait until Drupal 9 is released.  Wait two seconds here to make sure I'm using the right microphone.

>> How is that?  Is that better?

Is that better?

>> DWAYNE:  Yes, much better.

>> MICHAEL LUTZ:  I was not using my headset mic.  I apologize for that.

>> DWAYNE:  I should have told you about it earlier.

>> MICHAEL LUTZ:  I guess so.  I thought it was on.  My bad.

>> DWAYNE:  Moving to Drupal 9.

>> MICHAEL LUTZ:  All right, anyway, keep moving on.  One of the last steps will obviously have to wait till Drupal 9 is released.  The first four steps are things you can do right now to be ready to upgrade and I want to talk a little bit on each point right now.  So first off, preparing your server environment.  Drupal 9 shouldn't require any more hardware.  You don't need a beefier server.  In fact because we removed a lot of deprecated code it should at some level be more efficient than Drupal 8 but in order to support future development on the platform the minimum version had to have been increased and the biggest example is with PHP itself so Drupal 8.8 required PHP 7.0.  Drupal 9 will require PHP 7.3.  It's based on symphony 4.  Symphony 4 requires PHP 7.2 so we knew we would have to have at least a minimum of 7.2 and on top of that 7.2 will reach end of life shortly after Drupal 9 is released so this December, 7.2 is going to reach end of life, so using PHP 7.3 as a minimum allows us to use some more modern versions of some of our dependencies like PHP unit which is going to minimize the possibility of some future security issue is going to force us to do a dependency update that will force us to do a PHP minimum version update somewhere in the middle of the Drupal 9 cycle so we should be able to be using PHP 7.3 as a minimum all through the Drupal 9 cycle.

But in addition to PHP minimum supported versions of the various databases have been changed so my SQL or Percona have to be in versions 5.7.8.  If you use Maria DB, 10.2.7.  SQL light you have to use 3.26.  Post gre, you have to be in version 110 and you have to be on drush 10 to work with Drupal 9  Drush 9 will not work with Drupal 9  If you're on managed hosting you quite likely meet these requirements already or you will by the time Drupal 9 comes out but if you self host or if you haven't updated some of your software versions in a while, make sure you meet these new requirements.  PHP I want to talk real quick, typically upgrading your PHP version is fairly painless but there seems to always be just like a couple of incompatible backwards, backwards compatibility breaking changes to be aware of.

A couple of the common ones right now there's a warning that started getting thrown in PHP 7.2 if you try to count an uncountable object, it used to just return 0, and now it throws a warning.  There's a new warning in 7.3 if you use continuing side of a switch statement so each version of PHP just has a few things like that to be aware of.

I've put links up here, each version of PHP also comes out with a migration guide which will    there's a link inside that page that lists to all the backwards incompatible changes of that version so if you have to upgrade your PHP version it's good to be aware of them, review them and be aware of any changes you might have to make before you're ready to upgrade your server.

All right.  So next we're going to talk about upgrading your code to be compatible with Drupal 9  So what does that mean?  Well, deprecations.  Everybody's absolute favorite thing.  Beginning in Drupal 8 we changed the way we developed the next major version of Drupal.  While Drupal 8 itself was developed completely separately from Drupal 7, Drupal 9 has been in development inside of Drupal 8 for the past few years and this has been done partially through the regular introduction of new features and more importantly for this discussion through the introduction of deprecations so through each of Drupal 8's minor releases, new and improved APIs have been created, and at the same time the old APIs have remained in the code base.  These old APIs are for the most part still functional in Drupal 8.  Drupal 8, and Drupal 8.9 which leads me to one of the other big takeaways for today is that Drupal 9 is really just Drupal 8.9 with all this old deprecated API code removed, and then a few dependency updates such as moving from Symfony 3 to Symfony 4 so what this ultimately means is if you can adjust your code now to use the newer APIs that are already there instead of the old deprecated APIs it's going to work in Drupal 8.  It will still work in Drupal 9 and I want to go through a common example of what this means here going as far back I believe as Drupal 5, Drupal provided an easy way to display a status message or warning message to users.  This method here, Drupal set message takes two arguments, a message and a message type for severity which is a string, it's either the word status or the word warning or the word error.  The method stores the message and when it comes time to display the next page you get your pink or yellow or green box at the top of the page that has the message in it and this method still exists in Drupal 8.9.  If you call it, it will still work.  It's still going to display your box but it is not the right way to display a message anymore.  Starting back in Drupal 8.5, the functionality to do this was replaced and it uses the modern service class called the messenger service.  This class has dedicated methods for adding messages of various severities.  It has ways to retrieve and delete the messages, it's a service class.  It can be overridden.  It can be mocked in testing and it's really just overall a much more modern and useful way to package this functionality.  As of Drupal 8.5 if you wanted to display a message to the user this is the way you should be doing it.  

However, over in the old bootstrap.inc. file we still have the Drupal set message function.  It's been rewritten.  All it does now is take those arguments and pass them back through to this new service but if we deleted that function, any code that called it would be broken so what do we do?  We leave it in and deprecate it and so how do we deprecate it?  We put a comment on it that it's deprecated.  We issue a change record to tell people they should stop using it and we trigger a silent error that's if you're using automated testing tools you'll be able to find out you're calling something you shouldn't have.  So in Drupal.org, this method is not going to exist anymore.  If you have code that calls this method, for example we have Drupal set message oops you did something wrong comma error so we'll call this an error, if you have code that calls this it won't be compatible with Drupal 9  It will throw an error and so to adapt your codes follow the instructions in the change record.  In this particular case it's pretty simple because you need to replace this with this and Drupal add error, oops you did something wrong.  Because this new API is a service class and supports things like dependency injection and has additional methods around it you may also want to take the opportunity to further refactor your code to take advantage of that.  If you're calling this from a place where you can manage dependency injection you might want to inject a messenger service directly rather than calling it from the global Drupal class but becoming compatible Drupal 9 means replacing calls from the Drupal 9 with the call to the service.  The link is a link to the change record that describes how you would go about fixing any calls you have to Drupal set message.

So the big thing to do to get your code ready is go through, find these deprecations and we'll talk more about that later and then adjust them so that you're using code that's calling it the new way and then finally once all your deprecated code is taken care of and your module is compatible with Drupal 9 the last important step is declare it to be compatible with Drupal 9  Previously in your modules info.yml file or before Drupal 8.  You used to declare the version your module was compatible with.  So this key doesn't support the ability to declare a module to be compatible with more than one version of Drupal at the same time so it was replaced with a new core version requirement key which follows Composer style syntax to declare which versions of Drupal you're compatible with.  So one important thing to note is if you want to be compatible with versions of Drupal prior to 8.7.7, leave the core key in simply because before that Drupal didn't know what this new key was so without the core key it won't know your module was compatible.

So it's okay in this case so leave it in where it says core 8.X.  As long as you add the core version requirements key your module will still be compatible with Drupal 9 knows to use the core requirement key instead.  If you're not worried about being compatible with older unsupported versions and only need to be compatible with Drupal 8.7.7 and higher which includes all supported versions of Drupal right you can remove the core key completely in your info file.  I wanted to point this out because it's one of the tricky things.

But back to deprecated code.  Let's talk about how you can tell where you might be using deprecated code.  So there are a number of ways to find out about code you might be, find out the code you were using is deprecated.  In an on going process and once we get into Drupal 9 and start deprecating things for Drupal 10, I strongly encourage you to keep up with change records and release notes on any minor version that tells you new things we've deprecated.  At this point in the cycle we want to figure out where we're at because we've deprecated all we're going to deprecate for Drupal 9 so it's about getting a list.  There's a few ways to find out and get a list of what deprecated code your project might be using so we'll talk about automated testing.  We'll talk briefly about using IDs to detect deprecated code and finally we'll go into static analysis tools.  One of the best methods for detecting deprecated code use in my opinion is automated testing.  Deprecated code paths trigger this usually silent user deprecated error that can with the right configuration cause automated tests to fail so if you have good automated test coverage, then you can test your code against a new Drupal minor version as they come out, and you'll basically be flagged if any of the APIs you call are deprecated and I find this useful both in contributed modules I maintain and also in Drupal sites that I maintain with custom code as well.  I try to keep the automated test coverage to cover as much as possible and then I get a flag basically when I'm testing against a new version.  It can get tricky sometimes especially for a contrib module maintainer to keep all the tests green you want to keep green particularly if you're trying to support multiple minor versions of Drupal with the same code base so if you're trying to support Drupal 8.8 and 8.8 and 8.9 all at the same time, if something was deprecated in 8.8, but the replacement isn't available yet in 8.7, then you might not    you might get test failures against 8.8 that you're not ready to fix yet.  If this is on a custom site or custom code and you're running automated tests against your custom code, you should be, then you probably won't have this problem because you can write your code to match the actual Drupal version your code is running against.

The other tricky part here if you have dependencies on other modules executing deprecated code, you might also get failures you can't easily fix but I find this information useful because then you know this dependency you're using is not ready for Drupal 9 and you can keep an eye on it or find a replacement and figure out what you want to do and there is ways around both those issues.  You can detect which version of Drupal you're running against and alter the code path accordingly.  You can fix issues with dependencies using custom error handlers but they can get tricky and very advanced and they kind of add in a bunch of work arounds you end up having to remove later so you have to evaluate whether it's worth it to do that or not.  Also with automated testing there's certain classes of, there's certain classes of deprecated code, the really big one being just deprecated constant we can't trigger an error on.  However, the next two methods we have to detect deprecated code do quite Pell with deprecated constant.  Many IDEs are able to detect indicate use of deprecated calls.

When you hover over it, it will tell you the method is deprecated. I find this useful when I'm writing code, make sure I'm not using anything that's deprecated and it's also handy just to glance at a file and see if you have any deprecated code it wants to stand out for you but it doesn't give you the full deprecation message.  It doesn't tell me what I should do.  It doesn't give me a huge report of all the code in my project that might be executing deprecated APIs so I find this useful but it's not the primary source when I'm preparing something for Drupal 9, it won't be my primary method of finding out what deprecated code might be using.  That will most likely be a static analysis tool.  We've had a number of static analysis tools that have been introduced to help detect the deprecated code usage in products.  The current best practice is to use the upgrade status module.  We started with a command line tool called Drupal check, and this is probably the best option right now to find your deprecated code using static analysis so this module has the ability to scan, detect deprecated code usage and list it out by extension through both custom and contributed code so this shows the contributed modules and what possible deprecation errors might be there.  There's another section above that lists for custom code so it's separated out and organized.  You can fix one thing at a time.  This module can give detailed information as to what calls are made, what the replacement should be and in this example here you see some things listed as fix now and a couple things listed as fix later.

That just goes back to what I was talking about before, there's a few things that may have been deprecated in Drupal 8.8, the replacements aren't available in Drupal 8.7 which is still supported so those are the ones this will say fix later.

You have a couple of choices that you can do there.  You can either wait, have a patch ready, release a new version as soon as Drupal 8.7 is unsupported so you release a new version along side Drupal 9.  You could release a new version now that only supports Drupal 8.8 and above so you have this older version which supports 8.7 and the newer version that supports 8.8 and 9.0.  Then you have to maintain two version.  You could also commit work arounds like I talked about earlier, various ways to figure out whether your replacement exists and use it if you can.  Otherwise fall back to a deprecated code.

Again this adds a little technical debt because you want to pull those out later and if you go that last route you're still going to show up in the static analysis list because they don't have a way to know that you figured out in runtime whether you can use the deprecated code or not.  It sees you still have that function call so you'd still show up on one of these lists even though you know you've adjusted for it properly.

So you found all your deprecations.  What are you going to do to fix them?  In upgrade status you'll usually see what your replacement code path is.  If you need more information you find the change record for that particular one.  If there's something more advanced you have to do, the change record will detail what you have to do.  But a lot of the deprecations have a one to one direct replacement like that Drupal set message we mentioned before.  So there was a lot of moving these single functions into container services, and so in those cases it's usually just a direct replacement, switch this line over to the new one.  There are tools that can help automate that and I won't talk a lot about this but the rector project is a project that aims to help with some of the most commonly used deprecations.  It currently is limited.  It can only do a few of them but the top few deprecations including things like Drupal set message account for a vast percentage of the detected deprecated API usage in all contrib projects.  The last I saw it took care of the top 4 or 5 deprecations which covered about 30 to 40% of the total number of deprecations there are in contrib and if you're interested in learning more about how you can automatically fix your deprecations or if you want to help improve this today so it can fix even more deprecations than it can now, strongly invite you to join Dan Montgomery.  There will be a presentation automate Drupal 9 upgrades addressing code as a community.  It will be in room 324.  The session immediately after this one so at I believe 3:45 central time here in room 324 he'll give you more information on how it works and what you can do to help make it more powerful.  I'm running low on time here but the next step after you fixed up your custom code is determine the D9 readiness status of the contributed modules.

And fortunately as we saw earlier the upgrade status module is another great way to figure out what the status is or at least start to figure out what the status is of your contributed modules.  And if you run this right now against the standard site and the standard set of modules, guess what?  You're probably going to find out a lot of them aren't green on this.  That's okay.  This Saturday I'll be helping to host a table for D9 readiness contributions and we're going to help submit patches to some of the popular contrib modules and help get them ready for Drupal.  Also to note this is able to pull the maintainer supplied information from Drupal.org so in this case you can see the version of bLazy I've got installed has 119 errors.  It's not Drupal ready, but there's a note that says I'm on version 1.  There's a version 2 out that's D9 compatible, so I know I need to upgrade bLazy.  Whether you're able to help us contribute back and help get more of the modules ready on Saturday or not this module on the screen will give you valuable insight telling you when you're ready to upgrade your site.  As we get closer to Drupal release you can keep an eye on this screen.  Any modules that don't seem like they're moving to green go into the issue queue, and see if there's an open issue.  If the module is unmaintained, at least you know and you can start looking for a replacement module or better maintained alternative.

Finally the last step upgrading Drupal core to at least 8.8.  Drupal 8 maintained code to support the database changes you need to upgrade from one minor version to the next so to slim down Drupal and give a fresh start all that update code has been removed.  So anything before 8.8, the code to upgrade to the next version has been removed so in order to upgrade to Drupal you have to be on at least 8.8 first so hopefully you're already either there or you are ready and just finalizing the last steps of your upgrade from the still some supported 8.7.  I'm sure nobody in this room is on an older unsupported version, right?  Nobody's on an unsupported version.  If you are you need to upgrade to at least 8.8 in order to be compatible.

That's it.  If you have followed all the steps in this guide, and if you prepared your modules and themes, audited your contributed modules, and made sure you're on a modern version of Drupal then your upgrade to Drupal should be similar to any other upgrade that you do.  Ideally this is through Composer.  You might need to adjust your Composer JSON file to allow Drupal to be installed but at that point a simple Composer update, update your database with drush and cache clear, or whatever process you use to test an update over a Drupal minor version and that's about it for me.  I want to encourage everybody to come to Contribution Day.  I'll running a table.  There will be a table for the rector project we discussed and just you don't have to know code to give back.  There will be a first time contributor workshop and we want everybody there.

We'd love it if people would provide me feedback.  This is my first virtual presentation session I've ever given so I'd love to hear what I could do better through the link there or just on the session note page.  There should be a form.

And I don't know I really have time for questions right now.

>> DWAYNE:  We don't actually.  That's what I was popping up to say.  We're right at time as of right now.  But there is a Slack room we're asking every, all the speakers to go to the Slack room.  This should be room 325 and hopefully you'll be there to answer any questions we have there.  There were a few things in chat, if you haven't kept up with chat, there's a conversation but thanks for the recommendation for the next talk to see.  With that we're going to go ahead and move rooms, and I'll stop recording.

Everybody, give a big round of applause here, a thumbs up or something, to Michael.

>> MICHAEL LUTZ:  Thanks everybody so much for being here.