Routing: Implementing routing stages #5264
Conversation
|
14 errors in unit tests and some minor codestyle issues. |
|
Sorry, the majority of those errors was a typo, a missing +. Now everything should be fine. |
|
BTW: The PHP 5.6 tests throw a notice before the test is started. Something is wrong there... (Unrelated to this PR) |
|
Hello Hannes, thanks for delivering this! I've yet to test it and even to look at the code, but just reading your description, and especially your frustration for having to drag behind the weight of the B/C anchor, made me think something: Wouldn't it be possible to introduce in com_config "experimental" switches for testing new (possibly B/C breaking) upcoming technologies? Those switches should be accompanied by humongous warnings stating that they are meant only for testing purposes, that most 3rd party extensions will not work when they are turned on and that there is no firm commitment that the functionality will be implemented in upcoming official releases. This will allow 3rd party extensions developer to prepare for "the future" and core developer to receive an early feedback... Just my 2 (probably O.T.) cents... |
|
Hi @smanzi, that would indeed be a possibility, but I would strongly advise against it. There are 4 reasons against this:
Based on these points, I would not introduce such options as a general measure. Especially the first point is important to me. Compare Joomla and how much options Joomla provides simply when writing an article compared to Wordpress. Yes, the two systems are very different, but having several magnitudes more options than a system that aims at the same market, does not throw the best light on J. |
|
@Hackwar To be honest I agree only with your points 2 (strongly agree) and 4 (possibly agree). Solutions for the other points:
On the other hand I concur that smoothly integrating two "lines of code execution" in the core can be quite a mess and will most likely imply altering the "current line of code". I see this as something that should be reserved for exceptional cases, like probably a "new router" is, not something to be used for trivial alternatives. |
|
The router is by 80% aimed at third party developers, since it provides them a way to write a router in 5 lines of code and they don't get a serious headache just because of that aspect of their component. There would be little gained if third party devs did not adopt this code as quickly as possible. |
|
So... how do you see the future? An "early" 4.0 (will allow breaking B/C) or the frustration of having at hand fantastic code ... that can't be used? |
|
First of all, there are ways to get what we want right now without breaking B/C. And then I don't know what you mean with an "early" 4.0. Nothing is promised for 4.0. If the PLT decides to start work on 4.0 after 3.4 is released, so be it. 4.0 could mean that we have this router and some other minor details and that is it. Or it means something completely different. |
|
by "early 4.0" I was meaning exactly what you said: "this router and some other minor details". Well, maybe not just minor ones: I don't have examples at hand right now, but I would say everything that we are eager to do but can't because of the B/C commitment |
|
but... can you elaborate on the concept of "there are ways to get what we want right now without breaking B/C"? |
|
These stages things for example are a way to rewrite the current router to a rule based concept which is backwards compatible in a lot of ways. There are also questions how far our backwards compatibility goes. If we for example said that non-SEF URLs are not covered by that backwards compatibility claim, then we could do different things while still be part of 3.x. |
|
You can test this without #5105. 5105 however needs testing, too. 😉 |
|
got it! 😜 |
|
@test success
Question: should this be tested with SEF engines (like e.g. sh404) for compatibility or is this guaranteed to be B/C in this context? Personal note: ehmm... are you aware many of those who crowdfunded your initiative are expecting you to get rid of the ArticleID from SEF URLs, aren't you? I Understand this is not acceptable in the core because of B/C, but do you plan to release a system plugin (or anything else) to achieve that? |
|
Regarding sh404sef: Old code will be backwards compatible, but if you are using code written for whatever Joomla version this one will be released in, you might get conflicts. If sh404sef does not use the core router and instead deploys its own stuff, they would be disabling these stages effectively. Regarding your personal note: This is a place where I plan to add an option to disable or enable certain features of the routing system, which would include the IDs. So it will be part of Joomla and it will be in in a backwards compatible way. |
|
Perfect! 😄 |
|
I found an older version of sh404SEF in my filesystem and quickly glanced through its code to find its router integration. From what I could tell, they're extending In general with regards to backward compatibility, we can't control what happens if an extension is extending a library class and we change the internals of a method to extend what it is able to do. If the base API continues to behave the same pre- and post- change, then to me it is a B/C change. Unfortunately, once code is rewritten to make use of those newer features, that would cause an arguable B/C break if the extended class isn't updated as well. It's a risk we take any time we change the internals of a class or method to add new features. |
|
Hi all, 1 - strictly about sh404SEF, I do extend JRouterSite, but to be honest that's not really needed and I can probably put proxies in place if some methods I need/use were changed, which I doubt. sh404SEf is really API-compliant in that matter and only uses the additional rules it attaches to the site router. 2 - Foreseeable issues may more likely be if the routing system changes its behavior. LanguageFilter plugin is a good example. If Hannes suggestion to change it's behavior to NOT set/unset the language code were applied, that is broken between a pre-processing and a post-processing stage, then the data received by whatever routing rules are added will be different from what it is today. LanguageFilter routing has been causing much issues, and all SEF extensions (except sh404SEF) simply disable it for multilingual sites (and replicate its features elsewhere). As Michael outlined, I think a B/C break would happen if changes were made to the language filter plugin routing rules to take advantage of this. Otherwise, we should be good and I'm sure I can accommodate that PR, maybe without having to do anything actually. If the LF plugin is modified in such a way however, then I think that should not happen before 4.0 I'll get back to this post in the coming days with some test results. @Hackwar I think something that could be quite useful would be to keep and make accessible by all parsing or building rules the original input data (ie the original $uri) before it's been modified by any of the attached rules or the component itself. One reason for the difficulties with the language filter plugin is what you outlined, ie the language information is lost on the other parser/builder after the LF plugin has handled it. |
|
Hi @weeblr-dev, I've been working on the language filter plugin in PR #5140. Please look there, too. Regarding storing the original data somewhere: I'm a bit hesitant regarding this. I personally decided against it, since I think it is not only a clean way to clean up after you, but also a way to override something. It allows me to prevent the default code from working on this stuff by removing it from the query. But if there is a majority to implement something like this, I would not fight for dear life. 😉 |
|
Hannes, I've been testing with your PRs #5140, #5264 and #5276 in a multilingual environment and I have some results I would like to share with you hoping that you find them useful. The test have been performed using: 2 languages:
3 menu structures
3 categories
2 articles (both flagged as featured)
menu items, categories, and articles have been associated between themselves "Remove URL Language Code" (for default language) is set to: No Multilingual Status is:
From the above structure I had expected the following URLs to be valid and generated: What I get is instead: i.e.: the aliases of the home pages are not stripped away Language switcher does his duty, no problem. The "expected URLs" do works if entered manually, but the bad news are that also "chimeric" URLs do work, like e.g.: While others do give a 404, like: The discriminating factor seems to be that the language code must match the language of the item requested (disregarding categories and menu items in between) P.S: Of course the above has been performed with the latest 'staging' at the time of this writing (head at b858931), integrated with the 3 above PRs. |
|
I have the awkward and embarrassing feeling I was badly wrong about the URLs I was expecting... 😳 The 'chimeric' URLs issue remains, as before (without this PRs), so I would say... 'business as usual', with 100% B/C Sorry! Sergio |
|
fixed |
|
Ok, I implemented the changes that Johan proposed. Could we then get a review by a maintainer now and merge this stuff? 😉 |
|
@Hackwar Thanks looking good. One final remark. I'm not sure about the additional class properties that have been added without the underscore. I understand that in a future version the properties should not be underscored but adding them without using them makes little sense ? ProposalWhat about using the correct properties and removing the underscored ones and then dynamically assigning them again by reference when the object is instantiated ? Example : $this->_vars =& $this->vars; This should work, I think. The benefit of this is that you don't need to add unused properties to the class definition. Keeping things nice and tidy. Removing the legacy support also becomes easier later. |
|
Hi @johanjanssens, those properties are not added by me. That is stuff from 1.6 times, I think. In any case, the by-reference-trick will only work as long as no one assigns a new array to that property. I agree that we should get rid of that old crap, but I fear that it wont be possible before 4.0. Feel free to duke this out with the PLT. 😉 |
|
@Hackwar You are right sorry, forgot to check the blame. This really smells bad. But if it was there, let's leave it there. |
|
I updated the unittests to test properly for the new routing stages. |
|
I had to change the order in which the methods are called in the router to solve a logical error that I made. So far the order was:
This would not allow the language system and the Itemid-lookup to work correctly and since the Itemid lookup should be pulled into the router, all plugins depending on the Itemid to be present in the buildRules step would fail subsequently, so I had to reorder this to the following:
It should be noted that the first order (preprocess() after buildRules) has been shipped with 3.3.6. So this would technically be a break in backwards compatibility if we change the order now. However, this requires that code in the preprocess() component router step has to rely on data from the buildRules step to even create an issue. This again means, that someone had to implement this so far undocumented and in the core unused feature. I would take the chance here and change this, especially since otherwise none of my routing changes can be merged before Joomla 4.0. |
|
Imho we can take that B/C risk since I doubt anyone already used that new method. It would mean that they built an extension with a minimum requirement of J3.3.6 but they didn't really gain anything out of it. |
|
Could we then merge this stuff? |
|
@test |
| // Process the parsed variables based on custom defined rules | ||
| $vars = $this->_processParseRules($uri); | ||
| // This is the main parse stage | ||
| $vars += $this->_processParseRules($uri); |
There was a problem hiding this comment.
@Hackwar For code readability sake it would change this to $vars += $this->_processParseRules($uri, self::PROCESS_DURING);
|
@Hackwar The changes order for the build rules makes total sense and it more logical. Should the same change be made for the parse rules too to keep the implementation consistent ? |
|
There is no preprocess step for the parsing for component routers, so there is nothing that we could change there. 😉 |
|
True. All good then. |
The issue
The current system only allows to modify the URL before it is send to the component router. This means that for example the language filter plugin has to add the SEF prefix before the component router is even started and also has to delete the language value from the URL before that time. That means, that that data is not available for the component router. This is a more general problem that our routing system is very inflexible.
The proposal
This PR implements a 3-stage processing for the URLs: preprocessing, main stage and postprocessing. This allows the language filter plugin to add the SEF prefix in the preprocessing stage and to delete the language query parameter in the postprocessing stage, making that data available to the component router in between. At the same time, this introduces deprecation notes for all the methods that should be handled as rules in this 3-stage process.
Backwards Compatibility
This PR should be backwards compatible. There are 4 methods that have been changed, the methods to attach rules and the methods to process these rules. Since the new, second parameter is optional, existing code should not create any issues. Existing rules are still stored in their respective arrays and only the pre- and postprocessing steps add new keys to that array, which would otherwise be ignored. That said, no old software should be affected by these changes. The position of the main stage for parse and build, which represents the rules that we have so far, does not change either, which should prevent any changes in our intermediate results.
Personal thoughts
All the code that is in the methods around these process*Rules methods should go into a rule and be pushed into the specific stage. The problem is that that change WILL change the way our routers behave towards third party software. That is most likely a break in backwards compatibility, which would force us to postpone that step until Joomla 4.0, which frustrates me immensely...
Another thing that bugs me personally is, that we call the main stage '' in the method parameters, which looks bad. However that makes the nicest implementation from my perspective.
How to test
Testing here means that nothing changes between applying these changes. So please first check that your site works fine, then apply the change and see that it still works fine.
This PR needs a review by a maintainer.
This was made possible through the generous donation of the people mentioned in the following link via an Indiegogo campaign: http://joomlager.de/crowdfunding/5-contributors