I want to open a discussion about documentation and what is important to the community and I see no other place to do that. You can comment but mostly, I would like people to think about these things. And think about what can be done and should be done. And who can do them.
I have worked in the docs team for 2 years. I think we have made some good improvements, but overall the content is not in good shape. I think some of the new pages or chapters are quite good but a general structure is missing and some manuals have not been touched in over a year (or in years) and are frankly in very bad shape. These not so good documentation pages are not just a problem by themselves but also ruin the general impression people get from the docs and causes frustration and demotivation for the people working on them. Also, there is no way for new people to tell if a documentation or Wiki page is outdated.
There is no process or workflow for reviewing the manuals.
There is little coordination. No sprints.
Also, there are very few people reviewing and merging. I am not willing to do this any longer. So currently, there is 1 person left for reviewing and merging.
And then, there is also the Wiki …
To be honest, the documentation content (still) feels like years of neglect even though a few people have been working really hard and doing a great job improving things. But it seems all the effort is like a drop in the bucket. So, why is that?
When I talk about these things I sometimes get answers like:
If it (documentation) were important to people, they would do something about it.
Especially the wiki is full of very outdated and misleading content for newcomers, so I suggest to remove it.
The Extbase and Fluid documentation as well is heavily outdated and full of misleading things. I’ll take a deeper look into that one, once TYPO3 10.4 is out.
Thanks for bringing it up! Yes documentation is absolutely a very important thing and there have been great changes in the last year’s.
What I miss:
removal of wiki
better structure in core api, e.g the xliff and crowdin changes are not really on correct place or maybe there are?
I especially thank you and Daniel for your work. If it is about too less appreciation by the association, eg funding, I am sure we can figure something out.
My question: who is currently the team lead and what is her/his opinion. How much money has been spend on what?
There is some kind of saying we all know: “Everyone hates documentation.” - i.e. to state it properly - nobody loves doing it. This is true as soon as there are “huge” barriers to do so. As the lead of the form initiative I have spent hundreds of hours working on the docs and I am not satisfied at all. Most of the time I am struggling with the setup arround it, wasting precious time, not being able to get the job done. I see the biggest problem in using the restructure text format. It feels a little bittle like TypoScript. I do not know a single project were RST is used. And it does not even fullfil the requirements. We have to do a lot of tasks manually when it comes to generate some technical documentation out of the YAML files. Till now, nonody knows an answer and it seems like writing an own script to do the trick.
In our company, we have tested a lot of different technical solutions. Working with different wikis (like Confluence and Redmine) as well as MkDocs (https://www.mkdocs.org/) amongst others. Guess what, a couple of weeks ago we decided to move to TYPO3 v10. Why? Because it does the job. It is a great fit for us. And not because we love TYPO3 (we do, really do). After some requirements engineering we saw that TYPO3 is the tool we need.
I am totally with you that we do not do enough. But I am pretty sure we would do more as soon as we are able to do so. Right now the whole setup is soooo complex. I do not have a solution when it comes to work together on documents including all the possibilities of versioning and so on. But RST seems ancient to me and stops people from contributing. Maybe moving to md helps. But in my case with the form docs a change of the “language” would not help.
Interesting perception, Björn. Even though I’m also not a big docs contributor, I stumble quite often over RST based docs, especially when we talk about bigger projects. E.g. https://docs.ansible.com/ansible/latest/index.html
Besides that, I am currently not aware of any other text-based format which provides nearly the amount of features that RST has. But please proof me wrong.
First of all, I would like to thank everyone in the documentation team for all the work that has done the last few years to improve the TYPO3 documentation and all the processes around the topic. The docker container is really helpful for e.g. extension developers to quickly check the rendering of the doc files, that was IMO a really huge improvement.
I absolutely agree, that the Wiki should to be shut down asap. The content is outdated and may be misleading for TYPO3 beginners searching for TYPO3 documentation / best practice.
When it comes to RST, I do not think it is a barrier for documentation. ReST is strongly used in the python community (e.g. Django or Flask) and the process of automatically generating documentation for various versions of a software is “easily” possible with RST.
Maybe there could be added a notice to the “not so good” documentations, so readers know, that content may be outdated or has to be improved. Is there a central place, where those “not so good” documentations are collected?
It would be interesting to know what are the biggest problems were, specifically. Then we might be able to see where exactly the problem were and if we can prevent them in the future. But that is a little off-topic for this thread which is about the question how important documentation is and if we are doing enough.
I agree with the points Markus wrote. In addition you might also want to look at:
“Common Pitfalls” which I wrote specifically because I stepped into every one of those traps listed or at least spent time cleaning up the work of others who did. But, it takes just a few minutes to read that chapter. And the things to avoid are pretty simple.
Use shortcuts, e.g. for Visual Studio Code (instead of having to lookup the syntax)
Some people are willing to use far more complicated things than reStructuredText and the current workflow, e.g. TYPO3 , but with the documentation, as soon as difficulties come up the whole thing in general is questioned. Incidentally, difficulties are not always off-putting, they can make it even more of a challenge and more attractive
Most of the documentation is for developers and integrators. You should also consider the target group who will edit documentation. I would expect these to be able to deal with Git, Docker and reStructuredText. And improve the current tools where they are lacking (e.g. reStructuredText and sphinx support in PhpStorm).
I would much rather use the current workflow than use something like TYPO3.
Until someone comes up with a proof-of-concept of something that has the current features and is better, I think we should stick with the current solution.
The “Extbase / Fluid book” has gotten some updates. I never read the entire book, but I believe that it is still outdated in parts. Also, it is very long, partly redundant and I think there are 3 example extensions mentioned (all of which were not updated). Also, some of the general Fluid stuff is out of place in there. At the time the book was written, Fluid was the thing to be used with Extbase. That is no longer the case, the combination with Extbase is not a given. Personally, I would throw the entire book away and start from scratch.
I think mostly “TYPO3 Explained” is - if I may so - in pretty good shape because this has gotten the most updates. But ask anyone - have they read the entire “TYPO3 Explained”? Can they tell you how much is up-to-date?
The “Getting Started Tutorial” has been fully reviewed and updated by me for TYPO3 9.
The problem is keeping track of what is up to date and not is a real challenge. Reviewing an entire manual is a big job. We do get a number of updates to the documentation by contributors, but these bigger tasks - they are more difficult to do and coodinate if no one is around to do this in one batch.
Lolli has been doing a lot of reviewing, improving and writing new stuff, cleaning up the structure and merging all the other separate manuals (like FAL, TypoScript Syntax, Inside TYPO3 etc.) into TYPO3 Explained. That was a huge accomplishment (which did not get enough recognition IMHO) but also a lot of work.
If there is no one around to tackle these bigger tasks, eventually, the documentation goes stale, especially if you take the speed of core development into account.
Bad documentation hurts new people the most, but they are the people who can do the least about it because for most tasks you need a good understanding of TYPO3.
I could have done more about the content, but some parts of TYPO3 still baffle me. It is still possible to review existing stuff or write new documentation but that takes extra time for diving into the subject and clarifying open question. (The advantage is: after you are finished writing the documentation, you will be an expert or die trying. )
I don’t think that ReST is a big hurdle for documentation contributors. It’s fairly straightforward - at least the part where you write a headline and a few paragraphs and you use the online editor.
Being forced to ReST for an extension is no fun at all. This is a barrier. Just see how many extension come with GitHub flavoured MarkDown. But this is not the point of the discussion.
IMHO the documentation is mostly good, but could be better. The reference sections are mostly on the point, if not a bit short. The occasional “have a look at the code” is a bit annoying. I read the docs because I do not want tor decipher the source code
Maybe we need a decision where examples and best practices are to be found. Currently I think they are scattered. Sometimes they are within references sections sometimes not. Personally I prefer a strict separation.
And kill the Wiki ASAP. Any random Google hit of a TYPO3 blogger will be more up-to date and accurate as its content.
It drives me crazy that more or less everyone thinks we are the only project using RST. Don’t know the cause of this idea.
I’ve worked with markdown, confluence, redmine, wikimedia markup, HTML and TYPO3. And nothing is satisfying to me as rst. It is possible to use a small subset to get the job done, but it covers you if you wanna have more complex stuff. You can use your favourite text editor and short fixes online through GitHub.
Proposing TYPO3 involves more hazzle then rst, as you would need to work with an RTE (which really sucks and never results in what you expect). Also you need another instance to maintain, another account, further setup of hosting and TYPO3 updates, extensions and user permissions. How to handle reviews? etc.
Is the current state of the documentation a problem?
(The other observations are also important, but maybe we can handle that elsewhere?)
Some example of what I would like to know:
For example from agencies: How long does it take agencies to get a new developer, integrator or whatever up to speed so that they are productive and produce good results? Assuming the new employee already has solid skills in PHP, Web development or something similiar. How important is the documentation or are other resources sufficient? What resources do you use?
For people from other countries besides DACH (Germany, Austria, Switzerland): How important is the documentation for you? Is it helpful? Is it enough?
From people getting started with TYPO3: What is your experience?
From people,also using other Webframeworks, CMS etc.: How does the TYPO3 documentation compare to others?
First of all: For me documentation is as important as code and features.
From my point of view documentation is that important because:
People have a look at docs before deciding for a specific technical solution, e.g. use TYPO3 or some other CMS? If docs don’t provide an easy introduction and a comprehensive reference, it might be out.
It allows people to try out the system without external help from people. No Chat, no youtube, etc. Just docs and that’s it. That’s how I’ve learned more or less everything during my career. Docs are the most important part, which are extended by all other kind of things.
Documentation is marketing. It communicates the featured and available API, which promotes possibilities for developers and therefore agencies and customers.
Regarding the structure. I’m not new anymore, but when I compare docs.typo3.org, or even typo3.org to other software where I start with, I’m missing the following regarding structure and content:
Short comprehensive tutorials. We already have larger tutorials for some topics.
But I would like to have a very short introduction how to get started which doesn’t cover every possible situation and setup. It should just define the best practice workflow as short as possible to get up and running. One could even say select your environment “ddev”, “lamp”, etc. and you will end up in your getting started.
Also it would be cool to have a large list of “how to” tutorials just like https://tutorials.ubuntu.com/. So “How to create a form?”, “How to create a custom content element?”, etc. We partly have those information already and could cross link for deeper insights. But short tutorials to most important things are really missing. I had the idea to solve this using https://www.skilldisplay.eu/ and build how to trees there with links to existing documentation, but that would not be the ideal situation.
I’m missing best practices. TYPO3 is really open minded and there are so many ways to reach a specific goal. But how do I know which one I should use? There should be pros and cons listed for specific topics.
All in all I would say TYPO3 still provides a very huge documentation regarding references, which is more or less up to date and complete.
But TYPO3 massively lacks good introductions and tutorials that are short, easy to understand and just working.
It’s not that important for me personal, as I know where to find info nowadays. But I would say it hurts TYPO3. Especially if we wanna expand outside of DACH region, e.g. Africa, Canada, India and all other places on earth.
But I also think we are doing already the possible. Support on Slack and StackOverflow as well as Facebook and twitter is available. Taking into count the number of people and their other responsibilities, docs are as good as possible already.
I think we just need more people, typical chicken and egg issue.
In my company, lesser experienced TYPO3 devs often come to me asking technical questions whenever they are stuck with a problem. What I often notice is:
Documentation is often perceived to be lacking some information or outdated, but everyone appears to be used to it and just accepts the fact.
Some outdated solution, that doesn’t work anymore for current TYPO3 versions is copied from some tutorial and they get frustrated because the advertised solution doesn’t work. Usually some years old article found via google, that never got updated, not the official documentation. But sometimes official docs are not immune to this problem, look at the current state of the extbase documentation.
I like the idea of Daniel about the small tutorials, which could help a bit with this problem about all these outdated, external tutorials, that the doc team has no control over.
First, I‘d like to thank all people, that are involved in the documentation writings or updates. From the outside point of view, the documentation has made huge steps forward in the last years.
The following things are just my uninformed thoughts after my last core contributions regarding the main barriers to keep the documentation up to date or improve the documentation:
a) The documentation is not part of the core contribution workflow
If I suggest a change, which isn‘t a bugfix or a clean up and push it to gerrit, I can‘t update the documentation at the same time. I‘m forced to write a change log file, but can not update the documentation itself. Then the review process may take some time and my suggestions may get merged days or even weeks later.
To keep the documentation up to date, I would have to track the change, even if I‘m not part of the merge process. That’s why I can not know when I need time to update the according part of the documentation. At this unknown point in time, I would have to set up a different tool chain to create a new pull request to update the according part of documentation.
To be honest, this kind of process might work well in my employed work, but is too hard to organize in my spare time.
Furthermore, it takes more time to write the same content several days in apart.
Especially for small reference changes, it would save a lot of time if the documentation is part of the core workflow.
I mean this kind of changes:
I suggest removing a TCA option and would simply delete some lines of TCA documentation.
I suggest introducing a new event and would like to add it to the list of events.
I‘m totally aware that this kind of changes are not part of the main process (and the main work) to write a consistent and comprehensive documentation. But it may help to reduce the workload for reference updates. The saved time may could be used to write tutorials, getting started docs etc.
I think the time it takes to go through all change log files and update reference parts is much higher than if you could do it at the same time — especially if you have to write a change log file with more or less the same content.
b) The tool chain
What I miss is the integration of rendering process to enable a quick preview of rst files. That might be something like an additional option in the ‘runTests.sh’ file.
If I suggest a change, which isn‘t a bugfix or a clean up and push it to gerrit, I can‘t update the documentation at the same time
I also find it unfortunate that there are 2 different workflows for docs and core. I once created this overview and the whole time I thought: I shouldn’t even have to do this and even with the table, it is not intuitive.
The suggestion to move docs (or at least core docs) into core repo has also been made by others.
I agree that often the docs are updated long after the patch is merged and a new release is created and this is bad.
However, just moving it to the core will not automagically get the work done,.
I personally also see some pros for doing it the way it currently is being done:
If you have to also update the docs in every core patch - this creates extra work for the developer and the reviewers.
Sometimes things are changed several times until the next release
Not every coder is good at writing documentation - so if you force people to change the documentation along with the code you may not get the results you want.
I actually think the current way would be ok if there was less delay in updating the docs.
And I think this will be improved in the future - it already has.
, but with the documentation, as soon as difficulties come up the whole thing in general is questioned. Incidentally, difficulties are not always off-putting, they can make it even more of a challenge and more attractive 