We recently replaced Gitbook with TerminusCMS for a much-needed upgrade of our technical documentation. In order to help our users understand TerminusCMS and learn from our mistakes, this three-part blog talks about how to use TerminusCMS as a backend for technical documentation.
Number one in the series, this one, talks about building the schema and the content curation process for non-technical users. Part two focuses on incorporating parsed code for our client reference guides. The final part looks at connecting with TerminusCMS and fetching and renderring data for the front-end design with some performance improvement stats thrown in for good measure.
Why Move from Gitbook?
We launched TerminusCMS in January 2023 as a headless content management system for complex environments and we needed to dogfood our product. There were also some issues with Gitbook. We were on the free license for open-source developers and these issues impacted user experience, the main two being –
- Slow page load
- Poor search engine coverage
The Building Blocks
Now that we’d decided to use TerminusCMS, the next port of call was figuring out what to do. The building blocks of the project were –
- Schema – The data model and requirements
- Document curation – Non-technical users to input content
- Auto-generated docs from code – Pages such as client reference guides are automatically generated from code and we needed a way to incorporate these needs into the schema and database
- Frontend build and deploy.
Schema
The schema was the obvious first port of call. We used a discovery session with the team to decide the content and data requirements and went straight into modeling.
The TerminusCMS schema is based upon a simple JSON syntax and you can model content/data either with a UI or, as we did, the JSON editor.
Page Structure
The initial schema was a little convoluted as we discussed reusing content blocks across the site. Our early schema had a page document that linked to several other documents and subdocuments, such as the title, body, sections, and subsections that might make up a page. This idea is a good one as it facilitates the ability to dynamically build front-end pages using elements and makes it easier to programmatically deliver content.
However, with our limited time, this was a little complex for our needs, so we ended up with a simpler page structure like this –
{
"@id": "Page",
"@key": {
"@fields": [
"slug"
],
"@type": "Lexical"
},
"@metadata": {
"order_by": [
"slug",
"title",
"subtitle",
"body",
"media",
"seo_metadata"
]
},
"@type": "Class",
"body": {
"@class": "Body",
"@type": "Optional"
},
"media": {
"@class": "Media",
"@type": "Optional"
},
"seo_metadata": {
"@class": "SEOMetadata",
"@type": "Optional"
},
"subtitle": {
"@class": "Subtitle",
"@type": "Optional"
},
"title": "Title"
},
The page class is made up of a string property called slug, which is the document key. Page includes linked properties to other documents and sub-documents. We also used @metadata and order_by to make the properties order in a logical way for those curating the content.
To make the content curation experience better, we use "@unfoldable": [] to tell TerminusCMS that these documents are to unfold in the document editor so it appears that the nested documents are part of the page. Subdocuments ("@subdocument": []) are automatically unfolded. See the body class for an example of an unfolded document within the page class –
{
"@id": "Body",
"@metadata": {
"render_as": {
"value": "markdown"
}
},
"@type": "Class",
"@unfoldable": [],
"value": "xsd:string"
},
You will notice that body is set to render as Markdown using the @metadata tag to provide rich text editing for the page body.
We created a single page class with a very generic structure to cover all bases. You could easily be more specific if you were approaching a project with more planning than our dive right-in approach. For example, you could create a range of reusable templates to use across different sections of your site such as calls to action and FAQs.
Site Navigation
The documentation navigation is not optimal and definitely hacky. A lesson in hindsight tells us that planning the site navigation prior to adding the content would be better.
It would make sense to add the navigation structure to each page as they were added. For example, including a `parent page` link property, navigation label, and order to the page document would mean nav menus can be generated with queries and avoid the need to maintain separate documents.
However, for the sake of speed and shipping the new docs site, our marketing person (me – and I am by no means technically proficient) created four document classes using the schema modeling UI (I mistakenly set the document keys as lexical using the menu name property and this will cause problems later down the line, but I’ll deal with that when it’s a problem – so ensure to put some thought into your key strategy).
Here’s the menu section of the schema –
{
"@id": "Menu",
"@key": {
"@fields": [
"MenuTitle"
],
"@type": "Lexical"
},
"@type": "Class",
"Level1": {
"@class": "Menu1",
"@type": "Set"
},
"MenuPage": {
"@class": "Page",
"@type": "Optional"
},
"MenuTitle": {
"@class": "xsd:string",
"@type": "Optional"
},
"Order": {
"@class": "xsd:integer",
"@type": "Optional"
},
},
{
"@id": "Menu1",
"@key": {
"@fields": [
"Menu1Label"
],
"@type": "Lexical"
},
"@type": "Class",
"Level2": {
"@class": "Menu2",
"@type": "Set"
},
"Menu1Label": "xsd:string",
"Menu1Page": "Page",
"Order": {
"@class": "xsd:integer",
"@type": "Optional"
}
},
{
"@id": "Menu2",
"@key": {
"@fields": [
"Menu2Label"
],
"@type": "Lexical"
},
"@type": "Class",
"Level3": {
"@class": "Menu3",
"@type": "Set"
},
"Menu2Label": {
"@class": "xsd:string",
"@type": "Optional"
},
"Menu2Page": "Page",
"Order": {
"@class": "xsd:integer",
"@type": "Optional"
}
},
{
"@id": "Menu3",
"@key": {
"@fields": [
"Menu3Label"
],
"@type": "Lexical"
},
"@type": "Class",
"Menu3Label": "xsd:string",
"Menu3Page": "Page",
"Order": {
"@class": "xsd:integer",
"@type": "Optional"
}
},
The `menu` class represents the sections of the docs site, for example, how-to guides and explanations. It features these properties –
- Menu Title – A string property should we want a title for the menu (we haven’t used this in the front end)
- Menu label – A string for the menu label in the site navigation
- Order – An integer property for data curators to add their own menu ordering
- Page – A link property to a page document
- Menu1 – A link property to the menu1 document. This is a set to link one or more Menu1 classes to the menu.
Then we have Menu1, Menu2, and Menu3. They follow similar rules, Menu1 links to Menu2, and Menu 2 links to Menu 3 to create the parent and child navigation structure. For example, Menu1 has the following properties –
- Menu Label – A string property for the navigation label
- Menu1Page – A link property, linking to the page relevant to the Menu1 class
- Level2 – A link property to the Menu2 class. A set to add none, one or several. Users can link to existing Menu2s or create their own, where they can link to Menu3 from Menu2
- Order – An integer property so users can order to their liking.
Full Schema*
We have included the full documentation schema for your reference and to show how we dealt with media and metadata for SEO.
* Please note that the schema does not include the auto-generated from code documentation, we will cover this in the next blog.
[
{
"@base": "terminusdb:///documentation/data/",
"@schema": "terminusdb:///documentation/schema#",
"@type": "@context"
},
{
"@id": "Page",
"@key": {
"@fields": [
"slug"
],
"@type": "Lexical"
},
"@metadata": {
"order_by": [
"slug",
"title",
"subtitle",
"body",
"media",
"seo_metadata"
]
},
"@type": "Class",
"body": {
"@class": "Body",
"@type": "Optional"
},
"media": {
"@class": "Media",
"@type": "Optional"
},
"seo_metadata": {
"@class": "SEOMetadata",
"@type": "Optional"
},
"subtitle": {
"@class": "Subtitle",
"@type": "Optional"
},
"title": "Title"
},
{
"@id": "Body",
"@metadata": {
"render_as": {
"value": "markdown"
}
},
"@type": "Class",
"@unfoldable": [],
"value": "xsd:string"
},
{
"@id": "Title",
"@key": {
"@type": "Random"
},
"@subdocument": [],
"@type": "Class",
"value": "xsd:string"
},
{
"@id": "Subtitle",
"@key": {
"@type": "Random"
},
"@subdocument": [],
"@type": "Class",
"value": "xsd:string"
},
{
"@id": "Menu",
"@key": {
"@fields": [
"MenuTitle"
],
"@type": "Lexical"
},
"@type": "Class",
"Level1": {
"@class": "Menu1",
"@type": "Set"
},
"MenuPage": {
"@class": "Page",
"@type": "Optional"
},
"MenuTitle": {
"@class": "xsd:string",
"@type": "Optional"
},
"Order": {
"@class": "xsd:integer",
"@type": "Optional"
},
"menu_order": {
"@class": "xsd:integer",
"@type": "Optional"
}
},
{
"@id": "Menu1",
"@key": {
"@fields": [
"Menu1Label"
],
"@type": "Lexical"
},
"@type": "Class",
"Level2": {
"@class": "Menu2",
"@type": "Set"
},
"Menu1Label": "xsd:string",
"Menu1Page": "Page",
"Order": {
"@class": "xsd:integer",
"@type": "Optional"
}
},
{
"@id": "Menu2",
"@key": {
"@fields": [
"Menu2Label"
],
"@type": "Lexical"
},
"@type": "Class",
"Level3": {
"@class": "Menu3",
"@type": "Set"
},
"Menu2Label": {
"@class": "xsd:string",
"@type": "Optional"
},
"Menu2Page": "Page",
"Order": {
"@class": "xsd:integer",
"@type": "Optional"
}
},
{
"@id": "Menu3",
"@key": {
"@fields": [
"Menu3Label"
],
"@type": "Lexical"
},
"@type": "Class",
"Menu3Label": "xsd:string",
"Menu3Page": "Page",
"Order": {
"@class": "xsd:integer",
"@type": "Optional"
}
},
{
"@id": "Media",
"@type": "Class",
"@unfoldable": [],
"alt": "xsd:string",
"caption": "xsd:string",
"media_type": "MediaType",
"title": "Title",
"value": "xsd:anyURI"
},
{
"@id": "MediaType",
"@type": "Enum",
"@value": [
"Image",
"Video"
]
},
{
"@id": "SEOMetadata",
"@key": {
"@type": "Random"
},
"@subdocument": [],
"@type": "Class",
"description": {
"@class": "xsd:string",
"@type": "Optional"
},
"og_image": {
"@class": "xsd:anyURI",
"@type": "Optional"
},
"title": {
"@class": "xsd:string",
"@type": "Optional"
}
}
]
Content Curation
The most time-consuming element of this project was the content curation. It requires humans to read, write and copy-check. As this was about dogfooding, we didn’t look to import the documentation from Github, we wanted to use it like a user would so the non-technical members of the team divided over 100 pages between them to add to TerminusCMS.
We used the TerminusCMS document explorer to add the content. This is automatically generated based on the schema. Although not a task to inspire the imagination, the process eliminated many bugs from the UI and was of value to improve user experience.
The standout features that helped the collaborative content curation include –
- Change requests – Making changes in a change request branch while others worked concurrently in their own branch helped to speed up the process. The diff viewer was a useful tool for copy-checking and quality control.
- Conflicts resolved – On the rare occasion the team member’s change requests collided, the conflict check and resolution tools help to eliminate content being stamped.
- Low-tech barrier – The document explorer takes minimal time to get accustomed to and requires little training for users to add and edit content.
The content curation process has played an important role in focusing the product roadmap to make it even easier for non-technical people to curate content. These include –
- Media library – TerminusCMS doesn’t currently have a media library, but it is something we plan imminently. For the technical documentation, we used a sub-domain pointing to a GitHub repo for assets
- Drag and drop ordering – A simple feature, but useful for ordering elements such as navigation
- Internal linking within Markdown – The ability to add internal links within Markdown to other pages in the database.
That concludes the end of part one. There are some aspects of our schema design that we could have improved on, but overall, it functions really well and required minimal effort to set up.
Next up will be Robin, who is writing about how we parsed the Python and JavaScript client reference code into TerminusCMS.
Following that blog, Robin will conclude with an article about how the front end was built. Due to the speed up on the site, we couldn’t resist putting out a blog showing the speed comparisons between TerminusCMS and GitBook.