Salvia software as a solution for web accessibility monitoring
When the Digital Services Act, which was enacted with the adoption of the EU Accessibility Directive, entered into force in Finland in spring 2019, the Accessibility Monitoring Unit had been working for more than six months. Communication of accessibility requirements and instructions on how accessibility requirements could be met were the highlighted as the most important needs. Over the past four years, we have answered almost 2,000 questions from our unit’s joint e-mail. This website has been viewed almost 900,000 times.
In addition to providing advice and instructions on accessibility requirements, the Regional State Administrative Agency also supervises compliance with these requirements. According to the European Commission’s Implementing Decision, Finland was required to use an automated method to inspect 185 websites during the first two monitoring periods and, starting from 2023, 240 websites. When testing such a number of sites, the challenge is not only to identify issues in accessibility. Without effective solutions, the time spent on recording observations made during inspections, reporting on them and possible translations of texts could become overwhelming.
We were able to start the preparation of the monitoring technology in spring 2020, when the first monitoring period defined by the European Commission was already underway. However, the problem encountered by Finland and many other countries was that there were very few if any ready software options available at this time. The European Union had funded various projects to meet the needs of supervisory authorities, but they were only about to be completed in 2021, making it difficult to rely on them. In some Member States, accessibility had already been enacted in legislation, and they had developed their own software for monitoring. However, these software had been developed for those countrys’ own conditions and needs and were not necessarily completely applicable to Finland’s fairly straightforward supervision. The further development of such software could be just as costly as creating completely new one.
Open source program libraries and the ACT Rules
When approaching the end of 2020, the WAI-tools project proved to be the most important aid. The international W3C, Norwegian and Portuguese supervisory authorities, and Dutch Accessibility Foundation, which carries out accessibility audits, took part in the EU-funded project. Other project participants included the University of Lisbon, Siteimprove, and Deque from the Netherlands, all of which developed accessibility testing software.
During the project, the Accessibility Conformance Testing Rules (ACT Rules) for systematic testing of WCAG accessibility requirements were completed, and software developers included these in their own software: University of Lisbon to QualWeb-core, Siteimprove to Alfa and Deque to Axe-core. The software included in the project were also published under an open source licence. This allows anyone with programming skills to make the most of these testing software libraries in any manner they wish.
Usability of testing software
Even so, these accessibility testing engines, cores or modules are not completely ready for use. They are software libraries in which it is possible to submit technical requests through interfaces to assess the accessibility of the given online content. The software responds to this request by returning a machine-readable technical report on the results of the testing. In practice, such programme libraries are components that operators are intended to integrate into their own systems in a manner that is suitable for their own purposes.
For commercial operators, these testing cores are only part of the service package they provide. The core of their business is to develop how customers can utilise the testing service. The supervisory authority’s needs for automated mass monitoring and long-term monitoring are very different from those of authors, web developers and service providers of individual accessibility auditing. On the other hand, for an individual service provider a continuous automated monitoring of the accessibility of the entire website can be most beneficial. For the purposes of the supervisory authority the reliability of the results and their adherence to the legal requirements are also emphasised. Not all other actors may have a similar need to distinguish statutory accessibility requirements and the observations related to user experience in as much detail.
Various tools for testing accessibility
QualWeb-core is used by the Portuguese authorities’ Access Monitor tool and anyone can also try the testing tool on the service’s website. Siteimprove uses Alfa in its own commercial service, and Deque’s Axe-Core is partly in use, for example, in Google Chrome’s Lighthouse product. QualWeb, Alpha and Axe each have browser plugin versions too. Based on the accessibility monitoring reports of EU Member States, each of these three software programs or the commercial services using them were also used in the supervision of several European Union Member States.
Other testing tools used by the supervisory authorities of several Member States include Wave developed by a non-profit named WebAIM, the W3C’s HTML validator and the Nu Html Checker, Accessibility Insights Headings map browser Add-on and the TPGi company’s Color Contrast Analyzer and ARC toolkit. Belgium (Bosa), Spain (OAW), and Luxembourg (simplA11yMonit) are among those who have developed their own testing software. According to reports, a PAC validator was mainly used for the validation of PDF documents in the Member States, but Luxembourg had also developed its own instrument simplA11yPDFCrawlerin for this purpose.
At the Regional State Administrative Agency of Finland, we chose to develop our own software that would use QualWeb as its testing engine.
Finland gets its own testing software
The activities of the supervisory authority must be productive, so the implementation of the monitoring of hundreds of websites required predefined user interfaces and automated work stages. It is not only a question of how much time the procedures for a single inspection would require, but also of the quality of monitoring and reporting, and thus of the rights of the parties involved. It would simply not be possible for people to review machine-readable inspection reports containing hundreds of thousands of lines with reliable accuracy. If accessibility monitoring authorities decide to issue orders to service providers to rectify shortcomings, the results of the inspections must also be presented in a sufficiently specific and comprehensible manner.
In other words, we had to develop user interfaces that worked in web browsers for conducting tests based on our own needs. In addition, the results had to be saved, and a functional user interface had to be available for browsing and downloading reports. Finally, the software had to automatically produce a report on the inspection results, which could be sent as such to the service providers. The user interfaces and reports should also be available in Finnish, Swedish and English. This required new development work, which focused on utilising the QualWeb testing core.
The service we developed between April and October 2021 with TietoEVRY met these requirements. It is called ‘Salvia’, which is derived from the word “saavutettavuusvalvonta” the Finnish term for accessibility supervision. As a rule, it took less than an hour for the authorities to review one website in Salvia and report on the results. The ease of use and speed of the service corresponded to the objectives set for it.
During the development work, the Regional State Administrative Agency had several discussions with QualWeb developers and received valuable support from them. During the cooperation, QualWeb also published its localisation add-on, which had already been under preparation, which meant that the Regional State Administrative Agency only needed to produce its own translation files for the titles, descriptions and results of the ACT tests. Now the Portuguese QualWeb software by default includes support not only for English but also for Finnish and Swedish for the ACT rules on WCAG 2.1 level A and AA criteria.
Further development of the service
Even so, the service had to be further developed in 2022, as its first version lacked the possibility of inspecting mobile versions of websites and e.g. services that required users to log in. The user management and performance of the service were also improved. QualWeb’s built-in browser add-on is now used to perform evaluation to sites that require login as well as single-page apps. Testing these services requires some manual work and, at the same time, more time than fully automated tests, but even so it is now possible to inspect them and report on the results.
However, the identified deficiency is that Salvia can evaluate HTML pages only. The accessibility of office and PDF files that are covered by accessibility requirements must be inspected by other means. In addition, there are still a couple accessibility deficiencies in the PDF files produced by Salvia. In particular, the authorities should take these deficiencies into account and correct them as necessary, for example in Adobe Acrobat or PDF XChange Editor programs, before forwarding reports. However, based on the Regional State Administrative Agency’s investigation, it seems that even though there are a large number of program libraries providing PDF document printing, a few are able to produce the files in an accessible format.
Efficiency, openness and transparency as objectives
From its very start, the premise for development work was the openness of the end result. A product developed with public money should be available to everyone – both commercially and non-commercially. For this reason, it was selected to use program libraries whose licences also enabled commercial use without the requirement for operators to publish the source codes of their own solutions. This was also believed to best benefit the promotion of the accessibility of digital services in society, which is also the primary objective of the Digital Services Act. By publishing the software under an open source license, we also offer others the opportunity to participate in its development.
Another key aspect was the transparency of supervision work. By selecting a testing core that uses international ACT Rules, accessibility inspections can attain the best, most comprehensive and most verified results. Publishing the software code also strengthens the legal protection of service providers, as the methods used in the inspections and their results can be verified by anyone. By offering Salvia’s software package also as Docker images that can be easily installed on their own server or desktop computer, service providers are also better placed to survey the accessibility of their services independently and in advance.
The most important reason accessibility monitoring exists is to promote the accessibility of digital services. It is in everyone’s interest that the largest possible number of actors are able to promote accessibility without the need for authorities contacting them first.
