Future of pagination

Teams and Committees Core
1

Hello TYPO3 folks,

as Extbase should be compatible with PSR-7, too, the Fluid Widgets have gone with TYPO3 11 and with this change the loved f:widget.paginate has gone, too.

Don’t get me wrong: I understand this step. I understand the need of this removal. But I will show you the problems of the current alternatives:

  • TYPO3 comes with a very reduced SimplePagination. I only knows 4 links: first, back, next, last.
  • It’s now the decision of each dev to implement the SimplePagination or not.
  • Some devs will implement there own Pagination: glossary2/Classes/Pagination/GlossaryPagination.php at 5.0.0 · jweiland-net/glossary2 · GitHub
  • Some devs will create individual TER extensions for pagination. EXT:numbered_pagination
  • Some devs will only respect 1-2 pagination libraries.
  • Some devs will allow changing pagination by EventListeners: https://docs.typo3.org/p/jweiland/glossary2/master/en-us/AdministratorManual/Pagination/Index.html
  • Integrators are lost, if an extension does not support any pagination
  • Integrators have to check pagination for each individual extension, how to configure and implement another pagination. See ExtConf of news
  • Integrators are missing a global configuration to set a pagination for whole TYPO3 system
  • Integrators are missing a configuration for an individual pagetree

In my opinion TYPO3 needs a cool and simple solution. Following just two ideas:

  • Something like a PaginationRegistry
  • Implement ONE good full featured pagination into core.
  • Maybe you have a better idea
2
  • Something like a PaginationRegistry

This would go the opposite direction of the intention behind providing a pagination interface. The idea is to allow extension developers to make their paginators work in exactly the way that’s appropriate for their extension - it was never intended to be a facility to provide packages that contain implementations, from which extension developers can then pick and choose.

  • Implement ONE good full featured pagination into core.

Can you describe what precisely you think a “full featured” pagination should provide? That would help a lot if the expectation is to implement the second of your proposed ideas.

  • Integrators are lost, if an extension does not support any pagination

This is one thing that is very unlikely to ever be changed. The whole idea behind removing the pagination widget was to ensure that pagination happens in the controller, which among other things implies that the controller receives the “current page” argument and feeds it to the pagination. Changing this back can only be done exactly by providing the very widget that we wanted to get rid of.

Of the points you raised above, I definitely see the developer-related ones as valid concerns (and I agree that those could probably be solved by shipping TYPO3 with a more flexible built-in paginator) - but the integrator-related ones are problematic to say the least. The presumption appears to be that integrators should always be able to completely replace or override how pagination occurs, and this is not the case. It is and should be up to the extension developer to 1) implement pagination and 2) if desired, provide ways to change things like items-per-page, number of numeric page links, etc. - EXT:news is a quite good example of how such a thing can be achieved by an extension that wants to allow an integrator to change certain aspects of pagination.

In particular, I don’t think it’s a good idea to provide a global pagination behavior. It is very intentional that pagination is completely controller/view-specific, including the decision whether to have or not have pagination. That being said, I could see a valid use case by for example being able to configure the default paginator on a per-repository basis, e.g. through an API on Repository that returns the appropriate Paginator implementation for that repository. But this does not (and should not) remove the extension author’s responsibility of, for example, adding controller arguments to receive the currently selected page and pass it on to the Paginator.

3

For me this point is the most important one:

I want to define a pagination “component” once for a site. It should look and behave the same then for all paginated lists in all extensions.

4

Maybe take a look here to get some inspiration: https://github.com/BabDev/Pagerfanta

Biggest issue: Not being able to get a total count of results on the pagination object. When I first saw this, I questioned the whole concept that was added to core. No thank you.

5

That’s not what I had in mind when creating SimplePagination. The implementation of the core served the purpose to keep existing paginations intact (extension manager, backend user module). The implementation is marked final for a reason. The interface is the important piece of code here. It guarantees how your paginator behaves. Hence, it was intentional to keep SimplePagination simple and let people implement their own implementations if needed. All that means, that we should rather talk about the interface instead of the implementation.

That’s what I had in mind back then. But those don’t need to be extensions, it can be simple composer packages.

Furthermore I second what Claus already said. I think the important part is the integrators’ experiences. That can definitely be improved but instead of giving the integrator all freedom like with the widget in the past, we should rather think about standards. It would be good to hear what integrators actually need besides a guarantee that an extension provides certain pagination data to be present.

As Claus said: Don’t. It contradicts the idea of the interface. Rather focus on that.

Could be a compromise although I personally don’t think that a specific implementation is important, rather the interface (again).

6

Something in the core would be a good thing.
A Middleware that sets all needed values for pagination

But I think I will go back to an own Extbase pagination

And stay on it.

Starting with first Extbase version without f:Widget we need to make our own.

When the view helper was introduced. We switched. But then need to go back to our own extension as only objects or arrays work.

Back to default as this was fixed.

And now again in 11 change all templates i think…
This is another unnecessary change.

7

Thanks for bringing up this discussion. In my opinion there are 2 main usages of a pagination:

  • 5%: specific implementation for a specific controller/usecase
  • 95%: a kind of standard pagination which does what it should.

the latter one was done by using <f:widget.paginate> or <n:widget.paginate>.

I got 2 main issues with the current pagination implementation in the core

1) Configuration of implementations
As extension author I want that others can use any implemenation they want

        $paginationClass = $paginationConfiguration['class'] ?? SimplePagination::class;
        if ($paginationClass === NumberedPagination::class && $maximumNumberOfLinks && class_exists(NumberedPagination::class)) {
            $pagination = GeneralUtility::makeInstance(NumberedPagination::class, $paginator, $maximumNumberOfLinks);
        } elseif(class_exists($paginationClass)) {
            $pagination = GeneralUtility::makeInstance($paginationClass, $paginator);
        } else {
            $pagination = GeneralUtility::makeInstance(SimplePagination::class, $paginator);
        }

this is my current code in EXT:news, see https://github.com/georgringer/news/blob/master/Classes/Controller/NewsController.php#L245-L252

2) Missing important features
Given 1000 records and 10 per page, I would get 100 pages which doesn’t make sense, so I created ext:numbered_pagination and a feature request for the core https://review.typo3.org/c/Packages/TYPO3.CMS/+/70069

8

Why not use DI here. Say, your controller needs a pagination, you use the Interface in the constructor and let users of your extension choose the implementation via services.yaml? That’s the idea of the Interface at least. That should define how pagination behaves, which methods there are to call, which data to gather in the view.

Your numbered_paginator shows it quite well. It implements the interface AND adds more methods which are not covered by the interface. That leads to a situation where integrators cannot be sure what methods to call i.e. which data to access in the view. The core can only document data accessors of the interface.

Question is, if we

  1. Stay with the current idea of having one or many interfaces in the core that define the behaviour and have others ship the implementation.
  2. Abandon the interface and only deal with implementations.

I’d prefer 1 and think about what paginations are possible and if the core can come up with proper interfaces. It still doesn’t solve the problem that integrators have to look up what extension supports which pagination but we would still stick to some kind of standard.

I don’t know if someone already came up with possible ways to paginate lists but that could be a good starting point to see what people might need.

9

You are aware that the “users” here should be integrators… so no services.yaml stuff etc for them, way too complicated.

10

Why not using DI? Because people might want different implementations per page because one time it is a blog, on another page for press articles and finally for a gallery . Yes that happens. Maybe this can also be solved by DI - don’t know but this was the cool thing about <f:widget.paginate>, integrators could just switch within the templates.

And yes, ext:numbered_pagination is really not perfect, it was more a fast way to make the code I had within ext:news available for others.

11

I have programmed over 50 extensions and I’m pretty sure that I will not add and support 50 paginations. Having the same code in one project again and again can’t be the right decision.

At least it should support same features as f:widget.paginate. first, previous, 5, 6, 7, next, last. Nice to have: It should support POST data, too, to prevent the lost of search filters.

Not really. Changing a resultset within Fluid is a damn bad idea. That was the idea to prevent. It’s the job of the repository to reduce the resultset and not controller. The controller just catches the URL arguments and traverse them to the repository.

Right. That has already be happen. In2code has posted a twitter feed where they have introduced a new ViewHelper.

See answer of jonaseberle1. It is the case. TYPO3 is not a Framework. TYPO3 is a CMS and it should always respect all kind of members (Devs, Integrators, Editors).

TYPO3 comes with a pretty good paginator since over 10 years and yes, I have checked AbstractPlugin, too, which also comes with a pagination. I don’t want to throw away SimplePagination, but TYPO3 should deliver a respectful successor for f:widget.paginate.

This is pure Dev thinking. As I told to Claus, TYPO3 is not a Framework. It’s a CMS.

Middleware is too early. A middleware does not know which plugins are on a page. Pagination should be handled within a plugin itself. Else you will get problems to differ the plugin namespaces correctly.

Good point. Don’t forget plugins with a search form. Either the SimplePagination nor the Extended version will respect POST requests (removed with TYPO3 10). So, each plugin with a search form needs its own implementation. I would prefer 20% to 80%. But yes: A good successor of f:widget.paginate will be helpful.

Thank you for your code snippet. It is short and easy to implement, but I don’t like the Integrator to insert FQCN in TypoScript. But it’s definitely better than DI, Service.yaml or EventHandler solutions.

Is it a job of an integrator to change DI?

Let us try to get something like numbered_pagination into core. That will reduce the problem by 80-95%. I can live with that as we have a successor of f:widget.paginate in Core. Please backport to TYPO3 10.

12

Can you elaborate on this? Why do you think you need the same code over and over in the same project?

That’s up to you to implement as you are handling the current page in the controller. It’s additional data that you need to send to your repository to first reduce the data set which then gets paginated.

We never did such things in the view. We sent additional parameters to the controller which then led to magic going on in the depths of extbase, altering the incoming request. We did eliminate that and as a consequence, the current page pointer needs to be handled by developers in their controllers.

True, we might need to talk about different terms here. The logic that splits result sets into different parts are currently called paginators whereas the part that is responsible for creating the UI elements is the pagination. The QueryResultPaginator alters the underlying QueryResult which came back from the repository.

I tend do disagree. There is one part that is done by devs: Handling incoming data and fetching the data to be paginated. That’s what devs now need to do to enable integrators to build the UI. What you describe is the previous approach where all result sets were paginationable out of the box with a lot of magic and flaws involved. It’s a trade off. Like Claus said, if you want that version back, you need the exact implementation as before.

I remember there were issues with pagination via ajax for instance. Or multiple paginations on the same page. All that can be implemented properly now. Also, offset pagination can now easily be cached. There are a lot of benefits now when dealing with edge cases but the default case is not properly covered. I understand that.

I’d like to state again that this is not an issue of SimplePagination or any other pagination, it’s a shortcoming of your personal implementation when using the QueryResultPaginator. It’s your job as a developer to reduce the result set before passing it to the Paginator which only splits the result in desired chunks.

Given all the information about Paginators and Paginations, can you specify what exactly you want the integrator to do and what not? f:widget.paginate looks like a mere view related thing but it’s not. What would a new view helper do for you? What is it’s responsibility?

No, I give you that. DI is the wrong approach here because your problems aren’t really located in the pagination, rather in the paginator and the need for a developer in the first place.

13

Just a clarification: this was true for AJAX pagination, but was not really true for inline pagination. Inline pagination would modify the query associated with the query-result before it was (re-)triggered, as such it would indeed trigger in the view. See https://github.com/TYPO3/typo3/blob/9.5/typo3/sysext/fluid/Classes/ViewHelpers/Widget/Controller/PaginateController.php#L175 (in non-AJAX mode this code is triggered by the wiget ViewHelper through a sub-request that is indeed processed directly in the View). The idea was that because the QueryResult doesn’t actually get dispatched until you start touching it (iterating, getting first item, counting, etc.) delaying it until that point allowed it to be modified before being dispatched. Unless, of course, you had already touched it in the controller or before the pagination widget was encountered.

On a side note this is actually still true for the QueryResult - the SQL isn’t fired until you start iterating, counting, etc. and one could argue that this is approximately as bad practice as the pagination widget itself (but in part, this approach only exists because of the intent to modify the queries in the view w/o triggering the SQL interaction). See https://github.com/TYPO3/typo3/blob/9.5/typo3/sysext/extbase/Classes/Persistence/Generic/QueryResult.php#L93 which is fired by next(), offsetGet(), getFirst() and others, but not fired until you call either of those methods.

14

I like the idea that an integrator can define the looks of a paginator once for a complete site.
Maybe the paginator widget was not the right/best way to do it.

From the discussion I am not sure if those who argue against the old widget understand the benefits a core API has for extension authors and integrators. To me it seems that some fight even the idea and not specific ways of possible implementations.

15

It is really cool that everything around pagination is flexible with the pagination API and developers have the freedom to create their own custom solution as required.

But I think that a reasonable amount of developers may just need the NumberedPagination class which @just2b created to implement a pagination that replaces the functionality of the old pagination widget. It is somehow sad that the feature request (https://review.typo3.org/c/Packages/TYPO3.CMS/+/70069) did not make it into v11, because chances are high that the code now will be duplicated over and over again into several extensions (at least in TER). Why duplicated - we have composer and the package can be required, right? Yes, that is true, but for non-composer installations you have to bundle external dependencies into the extension, which will lead to this kind of duplication.