Whitehall Publisher vs Wagtail CMS

In fact, GOV.UK content is managed and published across a number of web applications built by the Government Digital Service. The most significant of these is Whitehall Publisher which is currently responsible for managing ‘government’ content on GOV.UK.

As you might expect Whitehall has good, open documentation, and it has many notable design features to accommodate the complexities of publishing on GOV.UK:

• Whitehall mandates the use of a selection of preconfigured content types each designed for use in different scenarios.
• Many of the capabilities in Whitehall are built to well defined publishing models like workflow and states.
• Whitehall uses GovSpeak for formatting content, which is essentially Markdown syntax with some customisations for specific content types (such as steps and abbreviations) together with shortcodes which allow for reuse of centrally managed repeating content such as office addresses.
• Finally, Whitehall manages content using taxonomy and categorisation, rather than a page hierarchy.

The Torchbox team had the opportunity to get to know Whitehall Publisher last year when we were working at the Department for International Trade (DIT).

Whilst most government departments are obliged to publish and manage their content via Whitehall, some use cases are exempt, and that’s the case with the Department for International Trade’s great.gov.uk website (because it’s considered a marketing site - not something accommodated by Whitehall’s design / content types). During our stint at DIT we had the chance to interview a number of content designers working on great.gov which was extremely interesting, because they were able to directly compare their experience of working with Whitehall with their experience working with Wagtail.

Opinion was divided about Markdown at DIT. Some content designers with Whitehall experience were very comfortable with GovSpeak Markdown and could work with it naturally and at speed. Those who were new to it found using the syntax unidiomatic and wondered why it wasn’t possible to have a ‘Word’ style, rich-text editing paradigm. In part this is a matter of taste; it’s easy to understand why Markdown would be adopted within a tool like Whitehall as a simple, learnable, syntax for applying semantic markup - GDS equivalents in other countries have made similar choices (take a look at the US 18f’s Federalist system, for example). However, to us reliance on Markdown feels brittle in comparison with to a ‘content element’ model (like Streamfield) which conceptually removes the need for lots of custom page types but still reinforces editorial rigour.

Our answer to this in Wagtail was to go with a ‘best of both worlds’ solution. We extended the React-based Draftail editor to support editing in Markdown for users who prefer it, whilst leaving open the option of using the rich text editor and pasting in formatted content for editors who prefer to work that way. Wagtail doesn’t actually store Markdown, it converts content into a storage format that provides flexibility in terms of how content can then be surfaced and repurposed.

One of the objectives of the 2016 rebuild of the GOV.UK content publishing apps was to streamline the 141 content types in play to a more manageable number, and the May 2020 update of the documentation shows how successful that initiative has been. Most of the consolidated document types are accompanied by detailed guidance, laying out a range of structural, formatting, and stylistic criteria that should be considered to ensure clear and effective communication, and conform to best practice. However, this does also mean that there’s quite a bit for content designers to remember. Consider the News article: news story content type. The guidance comprises a range of general advice on best practice such as these pointers on length:

And there are some much more specific stipulations e.g.

• headlines: aim for 65 characters max
• use a colon not a dash to separate: Google does not recognise dashes
• paragraphs should have no more than 5 sentences each
• use video and images

Within the editing UI the more specific guidance is backed up by field character count messaging.

We’ve started thinking about ways we could help content designers further in Wagtail. One thing we can already do is offer automated analysis of reading age and readability. These can be used to give inline guidance at the point of content entry that make it easier to assure compliance with a content style guide and accessibility guidelines. Adding support for validation of rules like "paragraphs should have no more than 5 sentences each” could also be done relatively easily. And it’s interesting to think about AI-led approaches to auditing for tone and style so that (for example) we could assess content against the Writing for GOV.UK guidelines and provide a score (a bit like an automated accessibility test) that could, in tandem with a defined threshold, help with the 2i (more later) or sign-off processes. Automated testing of this sort might not be perfect (…yet…!) but could be helpful in providing a steer so that users responsible for approving content (like managing editors in Whitehall) are able to prioritise their time to be as effective as possible.

Another complaint we heard from DIT content designers was that it was awkward to integrate the tools they were comfortable working with to create and edit content collaboratively - such as Google Docs, Sharepoint, Word - this was perceived to be a problem with both Whitehall and the great.gov Wagtail. On closer inspection this issue broke down into two parts. Firstly - how can I easily understand where a piece of content is in the approval workflow? Secondly, is there any way I can avoid wasting time pasting content from a document into a content management system and meticulously re-applying (and checking) formatting?

Since then we’ve tackled these issues in Wagtail in a number of ways. First up there’s the issue of moving content from one format to another. Pasting formatted content from desktop or web-based authoring tools into a content management system has historically always been a bit hit and miss. Wagtail’s Draftail editor provides good support for this but the reality is that when you’re working with formal content types for web publication you’re very likely not going to be dumping content into a massive ‘rich text’ field - you’re going to be carefully authoring content across a number of different fields each with their own important guidelines and constraints. For this reason we’ve built Wagtail Content Import, which allows one-click import of content from the editor that you use every day - MS Word, Google Docs etc. - into the relevant fields and StreamFields within Wagtail - thus retaining content structure and semantics.

On the workflow and management side of things, what we’ve heard most from content designers is that they need to know where a piece of content is in the process - in particular so that blockages can be identified and solved (both individually but also systemically). "If a document has been gathering dust in 'legal review' for 28 days, then I want to know.” One of the most practical examples of this we saw at DIT was the 2i review process. For the uninitiated, a 2i or ’second pair of eyes’ review is what takes place when a piece of content is created and submitted for publication: another content designer will then check that content against the GOV.UK style guide and makes sure that tone, style and content are ‘on point’, as well as checking that the page appears correctly.

However a weakness with the 2i review process in Whitehall is that the workflow isn’t fully automated. Once an editor submits a document for review, no-one with appropriate permissions receives a notification: the document’s status is changed, but it’s up to the individual concerned to check / prompt someone to make sure that the review is done. It’s also possible to ‘Force Publication’ of a page which can result in content going live that doesn’t meet the style guide (check out this title!). At DIT initially we created a specific queue for content awaiting review which was visible on login in Wagtail admin, and as a quick fix we also experimented with automatically sending out Slack messages to the Content Design team whenever content was submitted for 2i. But we also saw the opportunity to go way beyond this and build in full support for configurable workflow, and this work has now resulted in Workflows in Wagtail.

Since our work at DIT we’ve begun factoring in some of the core capabilities of Whitehall (and newer initiatives like Content Publisher, which aims to address usability issues such as those our DIT colleagues reported), into our roadmap planning for Wagtail. We’ve also been paying attention to the work that’s going on at GDS’ counterparts globally - particularly in the US, Canada, and Australia - where the common challenges of government publishing are being tackled in both similar, and different ways.

If Wagtail can cope with the sorts of complex publishing scenarios that a site like GOV.UK handles (just have a think about the logistics of History Mode, for example…) then it should be able to cope with just about anything that gets thrown at it, but we’re also committed to making Wagtail a better fit for public sector contexts. Wagtail is already extensively used in local and national government in the UK, and within the NHS; and in the US you’ll find it powering sites as diverse as the Federal Election Commission and NASA’s Jet Propulsion Laboratory.

The wagtail-localize app is a direct outcome of this work - the internationalisation and translation functionality it provides was heavily influenced by what we learned at DIT (and at Mozilla), and it will soon be part of Wagtail core.

However, another GOV.UK feature that we thought was pretty compelling was Content Data. This provides a powerful dashboard that can be used not only to comprehensively audit content, but also to see how individual documents are performing based on a combination of page views / analytics data and end user feedback. So we’re now keen to develop something conceptually equivalent for Wagtail that will make content performance directly accessible to editors and site-wide audits and QA easy. In fact, we’ve already started.