Nightwatching Koha for 10 years

by Marcel de Rooy

Koha is the Maori word for gift. It is the name for one of the most used open source library systems worldwide. At the turn of the century (triggered by a Y2K bug) Koha was developed for a New Zealand library, the Horowhenua Library Trust. It went live on January 1, 2000. So we have reached Koha’s 20-year milestone now! In 20 years it grew to a system used all over the globe, having over 38,000 patches or contributions from more than 400 developers, as visualized once here. According to Marshall Breeding’s currently 5,151 libraries on all continents are using Koha, including public libraries, school libraries, national libraries and special libraries (such as the Rijksmuseum Research Library). And note that although this number is a good indication, it might even be higher.

At 10 years a marriage is traditionally called a pewter (tin) wedding anniversary. And that’s what we celebrate too. How so? Well, on January 1, 2010 after a period of extensive testing and converting data, the Rijksmuseum library started using Koha in production. And 10 years later, still happy together! Interestingly, pewter or tin bends. Obviously, we needed to bend a bit ourselves when we started using a new library system, completely different from our previous one. But luckily we could also bend Koha and customize it to our needs as a special library. And we bent Koha but we didn’t break it. We will be addressing this topic later on again.

Should we still explain the Nightwatch? No, you know that for sure. In this post we highlight some recent developments in Koha, some of our own experiences in using Koha including our customizations. And finally turn to some things we will be working on next year.

Jumping from Koha 3.0 to Koha 19.11
When we developed serious interest in Koha at the Rijksmuseum, Koha was at version 3.0. The system had been further extended, a larger user group had been created, and some serious support providers had appeared on stage, like Biblibre in France, Catalyst in New Zealand and Bywater Solutions in the US. Modules like Acquisition, Cataloging (incl. copy-cataloging) and Reporting, as well as the open character of Koha (open source, open standards, incl. Z3950/SRU protocol, MARC21) helped us over the line.

Around 2010 the Koha community decided to switch to the current release cycle of two new versions each year (in May and November). We started at version 3.0 and in the following years jumped from 3.6, 3.8, 3.14 to 3.22. In 2016 the release numbering was adjusted to time-based numbering. Koha 3.24 became 16.05 (i.e. May 2016). Afterwards we went from 16.11 to our current 18.05. And as expected, recently Koha 19.11 was already released.

A huge number of (smaller) enhancements and new features were added to Koha during these years, as can be illustrated by one of the recent release notes. Additionally, each release also came with lots of major and minor bugfixes. Some larger projects the community worked on, spanned multiple releases. Mentioning just a few important ones: a new REST API has been built (an interface for developers to let their code communicate in a standardized way with Koha). Koha objects were developed to form an object oriented abstraction layer between the code and the database (based on DBIx::Class as ORM tool). More focus was given to automatic testing using generated test data. The performance of Koha was improved with Plack/PSGI and Memcache. And Koha was adjusted to allow using Elasticsearch as search index engine instead of the older Zebra index.

What did our watchmen see?


The Night Watch


How did (and do) we use Koha specifically in our Rijksmuseum context? Some numbers to begin with, comparing numbers from our start in 2010 to the current ones. In these 10 years the number of books (magazines, catalogs, etc.) went from 184,000 titles up to almost 300,000. Meaning that we catalogued already more than 100,000 records (almost 40%) in Koha. The number of items went from less than 200,000 to over 350,000. All these items were tagged with an RFID label with a barcode a huge job in itself.

Between 2010 and 2015 we also made some moves on the technical front: we started on a Linux virtual server (VPS) running Koha using Fedora, a Linux distribution, which is based on Red Hat. At that time we still installed Koha mainly manually running a “make install” process. At the end of 2015 our library servers were moved to the Cloudstack platform, and at the same time we switched to Debian Linux, allowing us to simplify and better automate the Koha install, running community Debian packages.

Concurrently, we followed the Koha community with regular updates, as described earlier and we gradually extended our use of Koha features. In 2015 for example, we decided to make a more intensive use of the Circulation and Patrons modules. From then our users were allowed to reserve books via our public catalog (OPAC) and we registered the process of reading books in our reading/study rooms with check-outs and check-ins in the system. Logically, our users needed an account, which was made possible by enabling self-registration on the OPAC. From about 600 accounts in 2015 we grew to the current number of over 3,000 active ones.

Another module that we started using later on, was the Serials module. Instead of a manual administration on a so-called cardex system, we now keep our records in Koha. And as a last example in this series, we should mention the Inventory tool in Koha. After we had labeled more and more items with RFID tags, this tool has allowed us to compare the results of RFID scanning with our catalog data in Koha. It has enabled us to find wrongly placed books and report missing books more easily.

Another recent technical development to highlight, is our containerization of Koha during the last year. Software containerization allows you to put an application (like Koha) inside an isolated, controlled environment benefitting areas such as installation and maintenance, portability, security and stability. Starting with our Koha 16.11 version, we created separate Docker containers for Koha and its SQL database (actually MariaDB). This made it possible to easily test different Koha versions next to each other on the same test server before moving them later to the production server. Our test environment now runs on an Azure Linux server and our production runs on a Debian 10 “Buster” server acting as the host for Debian Stretch based containers. And yes, this made us leave the Cloudstack platform; we are running now more containers on fewer servers.

Remember that we can bend Koha, customize it to our needs? Although we always try to stay so close as possible to the community version and minimize local changes, we have needed some smaller changes for styling, reporting, cataloging plugins, local html pages and so on. Changes of such a nature are less likely to be of use for other parties. But we have also made other changes in the course of time, more feature based, that we were able to contribute back to the community and which were incorporated into standard Koha. We will address a few of them in the next section. Allowing SRU targets in copy-cataloging, sharing lists (virtual bookshelves), consolidating code for handling XSLT stylesheets are just a few earlier examples of that.

A more recent example of customizing Koha was our extension of the article request feature. This Koha feature allows you to offer the service of photocopying articles, book chapters, etc. to patrons. We extended that by allowing requests for scanned copies of library materials by internal patrons (museum employees). We made a series of reports for that on Bugzilla, the Koha community bugtracker, and some of that work already found its way back to the community. Other parts should still follow.

Appreciating the gift: contributing back to Koha
Although this post is not specifically about open source software, an open source software project like Koha only succeeds if its users are not only downloading and using the product for free (at least no license fees) but are also willing to share their experiences, share code improvements solving bugs or enhancing the product, and participate in its community. We did so too, especially in the form of writing Koha patches and participation in the Quality Assurance team by our own application manager (and author of this post). On a smaller scale, we sponsored the translation of Koha for the OPAC into Dutch (NL version for Netherlands); there is another Dutch version for Belgium too for similar reasons as having multiple translations for English, French or German etc.

In the last 10 years we have contributed over 1,000 patches, many smaller and some larger, in almost every corner of Koha. Often smaller bug fixes but also including the topics mentioned before, as well as things like uploading files to Koha, inventory tool improvements, a longer series of authority merging improvements (copying authority info to bibliographic records) in 2016/17, EU General Data Protection Regulation related changes for patron records in 2018/2019.

From the start of the development cycle for Koha 3.8 onwards in October 2011 until now, the Rijksmuseum also participated by allowing our application manager to join the release team of Koha as Quality Assurance (QA) team member (reaching this 17th term for the 20.05 version). During that time he tested, reviewed and signed off over 2,000 patches (not to mention the number of disapproved patches 😉. An activity that shows our appreciation for Koha but also benefits us too in the form of building knowledge and trust in the codebase as well as community ties.

A small peek into 2020
As we may expect, Koha 20.05 will bring us more in areas such as Accounts, Acquisition, Mana integration and Elasticsearch. On our side we hope to work on adding persistent identifiers to our own data for bibliographic and authority records. And enriching these records by matching with/linking to thesauri like Wikidata or VIAF. This will pave the way for further integration of our library data with our museum collection, and further sharing it as linked open data.

Hope that does not take another ten years 🙂 Just keep on the watch! Frans Banninck Cocq en Willem van Ruytenburch will certainly do just that.

Group picture KohaCon18


Dit bericht werd geplaatst in Collectie Automatisering. Bookmark de permalink .

Geef een reactie

Vul je gegevens in of klik op een icoon om in te loggen. logo

Je reageert onder je account. Log uit /  Bijwerken )

Google photo

Je reageert onder je Google account. Log uit /  Bijwerken )


Je reageert onder je Twitter account. Log uit /  Bijwerken )

Facebook foto

Je reageert onder je Facebook account. Log uit /  Bijwerken )

Verbinden met %s

Deze site gebruikt Akismet om spam te bestrijden. Ontdek hoe de data van je reactie verwerkt wordt.