Lees-Rijk API aggregator and search engine

Government

In 2014, Maykin Media won a tender issued by the Dutch national government. Task was to develop an aggregator, search engine and an API interface for Lees-Rijk. Lees-Rijk provides employees of the Dutch national government direct access to all available publications and literature. Through a search request one gets an overview of the freely available publications and ordering options, if a publication isn’t publicly available.

For Lees-Rijk, Maykin Media has developed an interface for unlocking sources of contracted publishers. The interface automatically aggregates available publications from these 18 publishers and offers an API that allows Lees-Rijk to browse quickly through all available publications. Additionally, the publicly available publications of seven government institutions are included in the interface. Individual departments also have the possibility to add their own publications to the database with a dedicated portal. This allows Lees-Rijk users to have access to approximately 200,000 licensed and freely available publications from 34 different sources, which the interface has completely browsed within 200 milliseconds.

The interface is made in Python and Django and has a direct link with ElasticSearch’s search engine. Through the Django environment, sources and individual products are accessible and you can manage the way data is aggregated. It was necessary to use a great variety of formats to create the links with publishers. Amongst the used formats are XML, JSON, CSV, XLS (X). In addition to this, data is aggregated through a series of different protocols and APIs. For some of the sources web scrapers (obviously with permission) are being used. Django Rest Framework is used for the communication between the API and Lees-Rijk.

The interface of Lees-Rijk was realized within three months and it was operational the first months of 2015. Maykin Media is still responsible for support, hosting and maintenance of the interface and connects new sources and publishers by Logius’ request.

• API, Webapplication

Logius, Rijksoverheid