How to dynamically manipulate the typoscript result in V12

majpay · February 12, 2024, 4:13pm

Hi!

My use-case is very simple: I need to dynamically manipulate the following config:

plugin.tx_seo.config.xmlSitemap.sitemaps.pages.config.excludePagesRecursive

In V11 this was straight-forward. I added a USER object to seo_sitemap:

seo_sitemap.5 = USER
seo_sitemap.5.userFunc = My\Implementation->doSomething

The My\Implementation::doSomething would simply manipulate the typoscript array:

$request->getAttribute('frontend.controller')->tmpl->setup['plugin.']['tx_seo.']['config.']['xmlSitemap.']['sitemaps.']['pages.']['config.']['excludePagesRecursive'] .= ',' . $dynamicValue;

Why do i want to do this?

I added a backend field for users to prevent whole page trees from indexing. Usually my customers want test pages to try out things and this was my solution… they could add as many pages as they wanted and all of them were excluded from sitemap generation and had robots=noindex (different “hook”). But there are other use cases as well and i want to be able to configure this in the pages settings.

This will not work in V12 anymore because ext:seo now uses the FrontendTypoScript request attribute.

The FrontendTypoScript has no api methods to modify the typoscript and i do not find a Event i could add a listener to. I only found one event to manipulate the selected template records but i need a dynamic value…

I think i might be able to clone the FrontendTypoScript object, manipulate the clone, clone the request, add/override the request attribute, make my USER object ContentObjectRenderer aware, set the new request in the ContentObjectRenderer and achieve my goal this way … but that seems pretty complicated for such a trivial thing…

Or can i safely use the AfterTemplatesHaveBeenDeterminedEvent to inject some “inline typoscript” like it was added to the sys_template.config field?

What is the correct API to do such a thing in V12?

layne.obserdia · February 13, 2024, 7:09am

I would give the AfterTemplatesHaveBeenDeterminedEvent a try. We use that to dynamically load TypoScript based on current root line from the file system. So this should work.

I am not too deep into this, but could imagine this will also be respected by caching, which you might not want in your context. But you’ve used USER instead of USER_INT already, so this might be okay I guess?

majpay · February 13, 2024, 7:53am

I would give the AfterTemplatesHaveBeenDeterminedEvent a try. We use that to dynamically load TypoScript based on current root line from the file system. So this should work.

Yes i already tried that and it solves my problem. But thats no good api in my opinion. You would have to duplicate parts of the logic from the TyposcriptFrontendController (which template is root, loading order…) every time you use this event to add dynamically generated typoscript. In my first change i simply add it to the first templateRow in the hope this is the root template, that is used in my projects. Should normally be the case but who knows what my coworkers are doing

Some Event like BeforeFrontendTypoScriptIsStored with the capabilities to alter the tree would be much more intuitive i guess.

lolli · February 14, 2024, 3:20pm

Well. Yes. I’ll try to add some background:

TSFE->tmpl has been substituted with the new TS parser. That property still exists in v12, but is unused and has been removed with v13.

The fact that the determined TS ‘constants’ and ‘setup’ are added to the Request directly with v12 is in general a very good thing, since it is one part in the puzzle of modeling FE state better, which ultimately leads to getting rid of Request attribute ‘frontend.controller’, $GLOBALS[‘TSFE’] and TypoScriptFrontentController class. This is an ongoing v13 process and we already made quite some progress.

The easiest way to deal with this in v12: Have a middleware after PrepareTypoScriptFrontendRendering (maybe even after the shortcut/redirect middleware), to manipulate (and re-with()) the ‘frontend.typoscript’ attribute to your liking, and if you can, without involving ContentObjectRenderer at best, to reduce complexity and overhead. This approach is similar to what you found already. This is why there is no event in v12: It can be done with a middleware. Also note that we modeled TS as object tree in v12, the array representation is only an extract from that: The object tree is the ‘leading’ structure. Core v13 starts actively using this in favor of the array representation, and maybe at some point in time, the array may be gone. This is why you may want to manipulate the object tree instead of the array with your custom middleware, if you want to be future proof (unfortunately, the object tree is still marked @internal, so it kinda depends on which specific risk you want to take with this detail: using the array and risking it breaks later, or using the tree and risking it breaks).

Note the v12 code-state is a bit ‘under construction’ in this area. The fact that TSFE->getFromCache() is such a longish thing is simply due to the fact that we did not manage to take next steps into v12 before time run out. I’ve added a couple of weeks worth of work to v13 in this area already, and we’re close to refactor exactly this part in v13, which makes the entire thing much easier to grasp and follow, and - of course and as usual - more flexible along the way. Looking at v13, you’ll find that a lot of state that hangs around in the TSFE ‘god’ class has already been turned into properly modeled Request state - basically all state that is important on the path to getFromCache() is done at the time of this writing already, except getFromCache() itself: That upcoming patch will crack some hard nuts related to long standing ‘locking’ related bugs, and it will make the ‘page cache’ logic much more powerful.

There is one additional detail you may need to want to keep in mind when transitioning TS related custom code from before v12 to v12: The new TS parser comes with improvements related to caching: TS is now cached across pages, plus it can be re-used when calling cached pages that have USER_INT or COA_INT objects. The first one means that a request to one page warms up TS of other pages that have ‘similar’ TS. The latter one means your FE with USER_INTs no longer scales with the amount of TS, but makes it a nearly static offset only. This has a huge positive impact on pages with INT’s, since an entire per-request parsetime multiplier has been substituted with a rather small, rather static offset. This is achieved by multiple pretty effective cache-layers within the FE TS parsing logic, including a ‘final’ uppermost layer to cache the full FE TS, which can be re-used across pages, and which did not exist before.
This is the main reason there is no general “manipulate your TS here” event within getFromCache() or the calling PrepareTypoScriptFrontendRendering middleware: When you add dynamic code at this point which adds TS state based on the current rootline, this is cached, and then spills over into other TS of pages that re-use the cache entries, because final cache TS does not include root line information, but only TS record information. This is also the reason why you won’t run into “cache overspill” trouble when you manipulate the TS records itself in an earlier middleware (as you outlined, and as b13 ext:bolt does as well, but only on a ‘per-site’ basis). However, in your case, it may be better to do your dynamic thing in a middleware after the TS Request attribute has been added, when TS stuff has been done, as outlined above, since that will suppress a multiplier to the number of different TS cache entries that can exist.

I hope that insight is helpful …

majpay · February 14, 2024, 3:36pm

I’ll stick with the event listener. Its determined when this is called, middlewares are not, they can be disaptched at different positions depending on the composition of your installation. So no - i am sticking with my working hack. Knowing i might need to refactor that hack again in V13 …

Yea TYPO3 never gets boring (and that’s not meant positively). The times i can spend on making actual features for our customers gets shorter and shorter because simply “keeping up” with all the breaking changes has gotten too time consuming IMO.

lolli · February 14, 2024, 3:44pm

Huh? Of course you can precisely define at which point your middleware is positioned. That’s an important key feature of the middleware structure. Did you miss something in this area? How do you think this can not be defined in a deterministic way? How can we help you here?

majpay · February 14, 2024, 3:50pm

I can register the middleware “before middleware A” and “after middlerware B”, so it will be dispatched in between. But in some installations there might be another middleware just between. I am not doing all of that in the implementation where i fully know of all the possible middlewares, i would have to add the middleware in my extension, that is installed in many projects and not all of them are composed of exactly the same modules.

We had situations where problems occured because in one project the ordering of middlewares has been changed to avoid some bug with static route dispatching. So there is a slight chance something like that happens.

Would not be a big problem in this case but i think i wont win anything by removing the event in favor of a middleware.

lolli · February 14, 2024, 4:06pm

I see. So you know how to position middlewares correctly using ‘before’ and ‘after’. Note the DependencyOrderingService is designed to not throw exceptions when an ‘before’ or ‘after’ reference does not exist, so you could add relations that aren’t always there to take care of your ‘in some projects’ scenario.
But, when you do have middlewares that actively collide with each other, then you of course need to define these conflicts in composer.json, you’ll otherwise run into unexpected things. There is nothing the core can do about this, if a conflict definition is missing that you established with custom extensions. I however don’t understand in which way your projects can become a victim of this with your seo sitemap use case, except you have two extensions that try to solve the same thing?

majpay · February 15, 2024, 8:36am

Note the DependencyOrderingService is designed to not throw exceptions when an ‘before’ or ‘after’ reference does not exist, so you could add relations that aren’t always there to take care of your ‘in some projects’ scenario.

I did not know that, thank you.

But i am going to stick with the PSR-14 listener. Adding middlewares has become pretty common but i actually try to limit the use of a middleware to really manipulate the HTTP Message (for example content replacement, added of Headers, Cookie handling etc) itself and not as a “bus” for all kinds of hacks. Thats what i like to use events/signals/hooks for (events nowadays in typo3). I think thats what PSR-15 and PSR-14 are meant to be used for.