Structure and features of Wikibase for DH

From DHWiki
Wikibase Logo

Introduction

General use case for digital humanities

You can understand Wikibase as a product including:

  • a linked data store,
  • an end user editing interface,
  • an end user interface for querying,
  • and a set of APIs based on open standards.

This combination constitutes a versatile product with a lower adoption curve for some users.

There are many benefits of being based on the MediaWiki framework, like:

  • There are more features available through existing MediaWiki extensions.[1]
  • Multilinguality: support for >600 languages and variants.[2]
  • Expandable with PHP extensions, Javascript accessories and web services APIs.
  • Implement two constraint systems: QualityConstrains[3] and Shape Expressions.[4][5][6]
  • In active development by Wikimedia Deutschland and others.
  • Free/Libre/Open Source software without intellectual property rights block risks.

Documentation for Wikibase users is already available in several disparate places; topics affecting all users such as hosting, installation, maintenance and backup of Wikibase instances, user management, interface customization through user scripts, etc., are covered to a certain extent. We aim to produce an overview of the available information sources as orientation to new users as well as to assist advanced users with use-case specific requirements.

Wikibase is an opensource linked open data software platform. It has been developed for running the Wikidata project, the world's biggest free knowledge graph. Now it’s available for other users, following two different technical approaches:

  • As a standalone installation, from source code or using the Wikibase Suite software bundle. Wikibase Suite[7] is in active development driven by Wikimedia Deutschland.[8]
  • As a cloud software as service, managed by the WBStack[9] platform. As of today, the most important WBStack service is Wikibase.Cloud,[10] offered by Wikimedia Deutschland.

In any case, a Wikibase install is a complex choreography of software artifacts and features can be extended through MediaWiki extensions, javascript gadgets, CSS gadgets[11] and external services. This extensibility can be limited if you are using a WBStack based service.

For the Digital Humanities, we can consider a general use case described in the diagram. Obviously your particular case can be a mix of a subset and superset set of that.

You should not confuse Wikibase with Semantic MediaWiki,[12] which is a different approach to linked open data over the MediaWiki framework.

Technical details

Software artifacts

The main software artifacts behind a Wikibase instance are:

MediaWiki The mature and proven content management system that runs the Wikipedias, and the framework for the specific LOD extensions. It is written in PHP.
Wikibase extensions Provides the LOD features. MediaWiki extensions are also written in PHP.
Other MediaWiki extensions Not LOD-specific, but required for an advanced user experience.
MariaDB The popular relational database management system (RDMS) hosting all the MediaWiki content.
Elasticsearch A very powerful search engine useful for faster and flexible text search, included transparently in the main UX. The Elasticsearch indexes are synchronized copies of the MariaDB contents, allowing faster text searches.
Blazegraph A very powerful triplestore and graph database, with advanced SPARQL features plus some extensions developed for Wikidata. Blazegraph data is propagated from MariaDB and structured as pure RDF according to the Wikibase ontology.
Wikidata Query UI The UI application for querying Blazegraph with the SPARQL language.
Wikibase essential software architecture

Also, you can find Wikibase bundled with other components like

  • Cradle, a forms based data entry tool, partially compatible with Shape Expressions [only available in Wikibase.cloud]
  • QuickStatements, a small language and tool for massive data uploads and edits using CSV files.
  • OpenRefine Wikibase, the reconciliation interface for the OpenRefine data cleanup and transformation tool.

With the exception of Blazegraph, all artifacts are in active development by Wikimedia Deutschland and others. Blazegraph is a functional database (able to support the heavy workloads of Wikidata), but development is discontinued. The good news is that there is a new opensource graph database implementing the RDF and SPARQL standards, QLever (Bast & Buchhold 2017),[13] which is faster than Blazegraph, and attracting a lot of interest in the Wikidata/Wikibase community as the sucessor of Blazegraph, due to be in active development (Bast et al. 2026).

Licensing

All the software related with Wikibase mentioned in this document is free/libre/open source software OSI approved. This reduces any potential IPR block risk. Also, all MediaWiki code, extensions included, are community-driven developed, which also contributes to reducing potential IPR block risks.

Software interfaces

Another great advantage of Wikibase is the availability of software interfaces for extensibility or interoperable applications. The case of Wikidata applications is excellent with dozens of existing examples. The main programming interfaces are:

  • The MediaWiki PHP API[14], for creation of new extensions.
  • The MediaWiki JavaScript API[15], for creating javascript addons.
  • And the Action API[16] and REST API[17] web services API.

To access the web API there are some mature language libraries available, specially:

  • WikibaseIntegrator[18], for Python, including the RDFLib package,[19]
  • Wikibase-sdk[20], a NodeJS package.

Also, all Wikibase entities can be accessed by web in Turtle format.[21]

Federation features

As of today, there are plans for federation in three contexts:

  • SPARQL federated queries
  • Wikibase ontology federation, composed by:
    • Wikibase federated statements
    • Wikibase federated properties

SPARQL federation, also referred as query federation, on Wikibase enables querying across multiple distributed linked datasets, allowing users to combine data from different Wikibase instances (such as Wikidata) or external Linked Open Data sources (like ….) in a single query without physically merging them. This is achieved using the SERVICE keyword in SPARQL, which instructs the query engine to retrieve data from a specified remote endpoint.

The primary benefit is overcoming data silos, creating a more interconnected and comprehensive knowledge graph. This federation capability enhances data analysis, entity linking, and contextual enrichment, supporting scalable knowledge graph applications while retaining the autonomy of each dataset. However, it requires careful query optimization to avoid timeouts or overloads, especially when endpoints vary in performance and availability.

The Wikibase Ontology federation[22] features are conceived as ways of sharing data and resources between Wikibase instances. By using data across multiple Wikibase instances you’ll save space, eliminate data redundancies and let faster data provision and curation. There are two types, federated values and federated properties.

Wikibase federated statements[23] are for easy data reuse for the object values in your instance. This feature is in active development[24] and it’s expected to be ready along 2026 for consuming bits from Wikidata into your Wikibase instance.[24]

Wikibase federated properties[25] are conceived for reuse of properties. The technical complexity is significantly bigger compared to the values feature, so it’s not expected to be available in any short term.

Multimedia support

The main use of Wikibase is for linked open data but its MediaWiki heritage supports multimedia files like:

  • graphics: JPEG, TIFF, SVG, PNG, etc.
  • video: all video formats supported by ffmpeg[26]
  • maps: GeoJSON[27]
  • sound: MP3, FLAC, MIDI, OGG, etc.
  • 3D modeling formats: STL[28] and glTF[29] (in development)
  • scanned text documents: DjVu[30] and PDF
  • the Timed Text subtitles system[31]

How multimedia is managed depends on the configuration of your instance. For example, in Wikibase.Cloud you only can use multimedia included in Wikimedia Commons,[32] and you can manage these resources as linked data using the commonsMedia Wikibase property datatype. In other configurations you can use local uploaded files as linked data using the localMedia Wikibase property datatype.

Wikibase hosting solutions

Previously we explained the technical approaches for hosting Wikibase. Here we get into operative practical details to help in the decision making when starting a Wikibase implementation.

Wikibase-cloud-started-sign-up
Welcome screen at Wikibase cloud main page.

Installing and maintaining a Wikibase instance can be technically demanding, so several solutions have emerged to simplify deployment and hosting:

  • Wikibase.Cloud[10] is a software as a service, based on WBStack,[33] offered free of charge by Wikimedia Deutschland[34] that provides cloud-hosted Wikibase instances, eliminating the need for complex setup. Everybody can have a user account with up to six Wikibase instances. Wikibase Cloud instances include a range of predefined extensions[35] without any user configurability. Please note the service is under terms of use[36] and a data hosting policy (in development).[37]
  • Wikibase Suite[38] is an open-source software Docker orchestration provided by Wikimedia Deutschland that bundles all necessary components to deploy a full-featured Wikibase environment in your server. It includes MediaWiki, Wikibase, Blazegraph (for SPARQL querying), QuickStatements, and other key services. It is ideal for institutions or technical users who want full control over their infrastructure — including additional extensions —, data, configuration, and security backups. Wikibase Suite is available as Docker image.
  • Self-hosted solutions: Very advanced users can choose for their own implementation from source code repositories or Docker images tailoring this and that according to their requirements. This approach is suitable for teams with great technical expertise on systems integration. This is valid for both standalone Wikibase and WBStack implementations.
  • Third-party hosting providers: Some organizations[39] offer Wikibase as a managed service, including setup, maintenance, and support. This can be a good option for institutions lacking in-house technical capacity. Hopefully the number will grow in the future. This is valid for both standalone Wikibase and WBStack implementations.

For most adopters the obvious alternatives are Wikibase Cloud and Wikibase Suite. The following tables compare the features available in either of the two solutions.

Comparison by software artifact included
software artifact Wikibase.Cloud Wikibase Suite
MediaWiki ✓ ✓
Wikibase extensions ✓ ✓
Other MediaWiki extensions ✓ ✓
MariaDB ✓ ✓
Elasticsearch ✓ ✓
Blazegraph ✓ ✓
Wikidata Query UI ✓ ✓
Cradle ✓ ✗
QuickStatements v2 ✓ ✓
OpenRefine Wikibase ✗ ✗
Comparison by Wikibase property datatype
datatype Wikibase.Cloud Wikibase Suite
commonsMedia      ✓ ✓
external-id ✓ ✓
geo-shape ✓ ✓
globe-coordinate ✓ ✓
monolingualtext ✓ ✓
quantity ✓ ✓
string ✓ ✓
tabular-data ✓ ✓
time ✓ ✓
url ✓ ✓
wikibase-item ✓ ✓
wikibase-property ✓ ✓
localMedia ✗ ✓
edtf ✓ ✓
musical-notation ✗ ✗
entity-schema ✓ ✓
math ✗ ✗
contentmath ✗ ✗
wikibase-lexeme ✓ ✗
wikibase-form ✓ ✗
wikibase-sense ✓ ✗
Comparison by installed MediaWiki extensions
extension rationale Wikibase.Cloud Wikibase Suite
3D Provides support for 3D file formats ✗ ✗
AdvancedSearch Easy access to advanced search capabilities on Special:Search ✓ ✗
Automated Values Automatic creation of Wikibase labels and aliases based on statement values ✗ ✗
CiteThisPage Adds a citation special page and toolbox link ✓ ✗
Citoid Provides access points between the Citoid service and MediaWiki ✗ ✗
Delete Batch Delete a batch of pages ✓ ✗
Nuke Gives administrators the ability to mass delete pages ✓ ✓
Discussion tools Tools to enhance discussion pages. ✗ ✗
EntitySchema Store and discuss Schemas for Wikibase entities ✓ ✓
Gadgets Allows users to select custom CSS and JavaScript plugins] in their preferences ✗ ✗
GeoData Adds the functionality of storing and retrieving geographic coordinates ✗ ✗
MediaSearch Media-focused way to display search results through the media search page. ✗ ✗
MobileFrontend Helps improve the mobile presence of a MediaWiki instance ✓ ✗
MultimediaViewer Expand thumbnails in a larger size in a fullscreen interface ✓ ✗
NativeSvgHandler Serves SVG images directly to clients ✗ ✗
OAuth Allows usage of OAuth 1.0a and OAuth 2.0 for API authorization ✓ ✓
PagedTiffHandler Driver for viewing TIFF files in image mode ✗ ✗
PageImages Collects information about images used on page (for thumbnails) ✓ ✗
PDF Handler Driver for viewing PDF files in image mode. ✗ ✗
Purge Adds a purge tab on all normal pages allowing for quick purging of the cache ✗ ✗
RevisionSlider Adds a slider interface to the diff view, so that you can easily move between revisions. ✓ ✗
Thanks Adds links for thanking users for edits, comments, etc. ✓ ✗
TimedMediaHandler Synchronized audio, video, and text controller, supporting WebM, Ogg Theora, Vorbis, and SRT formats ✗ ✗
UniversalLanguageSelector Gives the user several ways to select a language and to adjust language settings ✓ ✓
UploadWizard multimedia upload assistant ✗ ✗
UrlShortener URL shortener for arbitrary URLs ✗ ✗
VisualEditor Visual editor for MediaWiki ✓ ✓
WebAuthn Provides authentication support using WebAuthn protocol ✗ ✗
Wikibase EDTF Adds support for the EDTF standard to Wikibase via a dedicated data type ✓ ✓
Wikibase Export for exporting data from Wikibase as simple CSV via a special page called "WikibaseExport". ✗ ✗
Wikibase Import Allows importing data from another Wikibase instance ✗ ✗
Wikibase Local Media Adds support for local media files to Wikibase via a dedicated data type ✗ ✓
Wikibase RDF Wikibase extension that allows defining RDF mappings for Wikibase Entities ✗ ✗
WikibaseCirrusSearch Wikibase extension for using CirrusSearch functionality ✓ ✓
WikibaseClient Client for the Wikibase extension ✓ ✓
WikibaseInWikitext adds the "sparql" tag for rendering SPARQL ✓ ✗
SyntaxHighlight provides rich formatting of source code using the ‎<syntaxhighlight> tag. ✓ ✓
WikibaseLexeme provides improved modeling for lexical entities such as words and phrases. ✓ ✗
WikibaseLexemeCirrusSearch for using CirrusSearch functionality ✓ ✗
WikibaseManifest Combines metadata about a Wikibase installation exposing it as a simple API ✓ ✓
WikibaseMediaInfo adding a MediaInfo entity for handling structured data about multimedia files. ✗ ✗
WikibaseQualityConstraints Checks constraints on both items and properties and displays the results on a special page ✗ ✗
WikibaseRepository Structured data repository ✓ ✓
WikiLove Adds an interface for facilitating positive user feedback to user talk pages ✗ ✗
VipsScaler The VipsScaler extension creates JPG and PNG thumbnails faster and at low memory usage. ✗ ✗
PropertySuggester provide suggested properties when a user tries to add new statements to an item. ✗ ✗
wikibase-faceted-search Enhances MediaWiki's search with faceted search capabilities. Filter results based on instance type or statement values ✗ ✗
Delete Batch Delete a batch of pages ✓ ✗
WikibaseView View component for the Wikibase Repository ✗ ✗
Kartographer Allows maps to be added to the wiki pages ✓ ✗

Wikibase Ontology, the backbone data model

The Wikibase Ontology[40] is a hardcoded fundamental data model, which guarantees interoperability between different Wikibase instances on a basic level. As explained in detail in the Wikibase data model documentation[41], data about entities is structured in the layers labels, aliases, descriptions, and statements. In the edit interface, each of the three has its own place, as shown in the figure. Each entity has its own identifier in the /entity/ namespace of the wikibase instance (e.g. /entity/Q1, /entity/P1, etc.), and one corresponding Page URL, where the entity data is displayed and editable by a user, e.g. /wiki/Item:Q1.

Data model for Wikibase entities of type item.

Entity data in JSON and RDF format

The types of entities modeled in Wikibase are:

  • Items: Any concrete or abstract thing (apart from words or lexemes) to be described on the Wikibase, using properties, as subject in a semantic triple, or as value of a property with data type item. Items have numeral identifiers preceded by letter Q.
  • Properties: to be used as predicates in semantic triples. Each property has one datatype,[42] and a numeral identifier preceded by letter P.
  • Lexemes: Lexeme entities are used to describe words following Ontolex-Lemon. They have a numeral identifier preceded by letter L.
  • EntitySchemas:[43] Shape expressions used for performing data consistency checks (see Candela 2023), and as template for the form-based editing tool Cradle, etc. They have a numeral identifier preceded by letter E.
  • MediaInfo:[44] Entities for metadata management of locally hosted multimedia. They have a numeral identifier preceded by letter M.
JSON representation of the statement "Douglas Adams - instance of - human" in Wikidata[45]

Each type is implemented in a standardized namespace associated with an integer number.[46]

Wikibase structured data is stored internally as MediaWiki pages in a JSON format modeled compatible with RDF, Each entity includes all the assertions (labels, descriptions, and statements involving other entities or data values) stored as described in detail in the Wikibase documentation.[47] This JSON representation is the format required for upload or modification of entity data using the MediaWiki API. Also, each entity is exposed in several interoperability formats, including RDF, Turtle or JSON-LD, as HTML link tags.[48]

RDF properties used in Wikibase for basic entity description (item, property, EntitySchema[49]
JSON key RDF property
labels rdfs:label preferred label, one per language
descriptions schema:description short description, one per language
aliases skos:altLabel alternative labels, multiple per language
sitelinks schema:about (Wikidata only: Concept links from other Wikimedia projects)

In parallel to the JSON entity data, stored in a relational database, changes to entity data are propagated to a separate graph database, where the RDF representation of the structured data in Wikibase is stored. The RDF data can be accessed as entity data in Turtle format,[50] and as RDF graph through the graphical query service, or through the SPARQL endpoint.

RDF representation of a Wikibase statement

The Wikibase Ontology,[51] the data model underlying the RDF representation of Wikibase content, defines a number of classes and properties; many of these stem from W3C-recommended standard RDF vocabularies. Some parts of the data describing an entity will be represented using these properties (e.g., rdfs:label), while others will be represented using properties defined by the wikibase user (numeral identifiers preceded by letter P).

Wikibase statements include by default a mechanism for further describing the main claim of a statement using qualifiers, ranks, and references. A graphical model of a Wikibase statement's RDF representation is given in the figure, where "entity" represents an item, lexeme, sense, form, or property node, each of which has its own URI in the main entity namespace of the Wikibase instance, and where the blue-colored "value" nodes, depending on the property datatype, represent entities of the same Wikibase, or data values, including strings, external identifiers, date objects, globe coordinates, et cetera. Each statement node may be linked to several qualifier values, and to several reference nodes (in the RDF representation attached to the statement blank node using prov:wasDerivedFrom), which make up blocks of references. Ranks are used for annotating multiple statements for the same property with one of the three values normal, preferred, or deprecated.

Property data types

Wikibase implements standard property datatypes;[52][53] some of them included in the Wikibase extension, and others trough additional extensions. Each Wikibase property is associated to one datatype; when uploading data using the MediaWiki Action API, the value is validated according to the datatype: for example, only a valid URL will be accepted as value for a property with datatype URL.

Datatypes made available through additional extensions
datatype extension
localMedia Extension:Wikibase Local Media[54]
edtf Extension:Wikibase EDTF[55]
musical-notation Extension:Score[56]
entity-schema Extension:EntitySchema[57]
math Extension:Math[58]
contentmath Extension:MathSearch[59]
wikibase-lexeme Extension:WikibaseLexeme[60]
wikibase-form
wikibase-sense

Common in Extension:Wikibase[61] are the following:

  • commonsMedia, for linking to a file hosted on Wikimedia Commons (image/audio/video/document);
  • external-id, for identifiers linking to an external database, usually an authority;
  • geo-shape, for geographic shape definitions expressed in GeoJSON[27] data files;
  • globe-coordinate, for latitude/longitude coordinates on a specified celestial body (Earth, Mars, etc.), with format restricted to WGS84;[62]
  • monolingualtext, for plain text associated to a single Wikimedia language code;[63]
  • quantity, a numeric value optionally with unit (expressed with a Wikibase item) and bounds/precision;
  • string, for unstructured text (no language code, no formatting rules);
  • tabular-data, for data table files hosted in Wikimedia Commons and imported from spreadsheets (CSV/TSV/others);
  • time, for date/time values with precision and calendar model (Gregorian or Julian) similar to ISO 8601;[64]
  • url, for web links (IRI);
  • wikibase-item, a Wikibase item in the same Wikibase instance (Q…);
  • wikibase-property, a Wikibase Property in the same Wikibase instance (P…);

When choosing a hosting alternative you should check the availability of the extensions you need.

SPARQL Queries

Query Service

The above described RDF representation is stored in the Wikibase instance's graph database and can be queried using the SPARQL query language, either through a graphical interface, the Query Service,[65] or, programmatically, through the SPARQL endpoint. How to use the graphical Query Service is explained on the Wikidata Help pages.[66] These explanations hold through for any Wikibase, with the exception of pre-configured namespace prefixes, which may differ.

Federated queries

Federated queries are SPARQL queries that involve more than one Wikibase instance, or other RDF databases, at the same time.[67] Federations allows users to combine data from different source databases in a single query without physically merging them. This is achieved using the SERVICE keyword in SPARQL, which instructs the query engine to retrieve data from one or more specified remote endpoints.

The primary benefit is overcoming (redundant) data silos, creating a more interconnected and comprehensive knowledge graph. This federation capability enhances data analysis, entity linking, and contextual enrichment, supporting scalable knowledge graph applications while retaining the autonomy of each dataset. In many use cases, a federation of an own Wikibase with Wikidata will make sense, for example for person metadata: Details about persons change over time, and synchronizing updates between databases can be avoided if federating with a central database, which for persons would be Wikidata.

Query result visualisations

The Wikibase Query service offers a range of visualisation types for SPARQL query results.[68] By default, variables named in the SELECT expression of the query correspond to the columns of the table containing the result, which can be downloaded in several formats (JSON, CSV, HTML). In addition, the following visualisation options are available:

Visualisation Description Examples
Table Default tabular display of query results; supports download as CSV, JSON, or HTML.
Image Grid Displays images from Wikimedia Commons returned by the query in a grid layout. An image gallery of DARIAH-EU WG DHwiki members: Query
Map Plots geographic coordinates on an interactive map. Birthplaces of people naming streets in the city of Vitoria-Gasteiz: Query
Line Chart Visualises numeric or time-based data as a line chart. Amount of publications described on this Wikibase per year of publication: Query
Bar Chart Compares categorical or numeric values using bars. Chart showing the number of pages in the 30 biggest Wikibase Cloud instances: Query on Wikibase World.
Scatter Chart Displays relationships between two numeric or date variables. Example from Wikidata: Number of films by year and genre; Query on Wikidata
Area Chart Similar to a line chart, but the area under the line is filled. Example from Wikidata: Cancer deaths by year and cancer type; Query on Wikidata
Bubble Chart Scatter plot where bubble size represents a third numeric value. Most cited works by Koldo Mitxelena, a Basque scholar: Query
Tree Map Shows hierarchical data as nested rectangles sized by value. Example from Wikidata: Famous people eye and hair color; Query on Wikidata
Tree Displays hierarchical relationships in an expandable tree structure. Artworks in the ArtBase (Rhizome) with more than two variants: Query
Timeline Places events or entities along a chronological timeline. Timeline of music albums by Manu Chao and Mano Negra: Query
Dimensions Parallel-coordinates chart for comparing multiple variables at once. Example from Wikidata: Dimensions of elements; Query on Wikidata
Graph Interactive node-link graph showing relationships between entities. Citation relations: The work of Koldo Mitxelena, a Basque scholar, Query

Each visualization type requires specific variable names or data structures in the query results to render properly (for example, maps typically need coordinates, timelines need date/time values, etc.).[68] At Wikidata, there is a page with examples for all visualization types.[69]

Data re-use

In general, there is an ever-growing landscape of use cases that involve Wikidata content,[70] and excellent tools that reuse Wikidata content, e.g. Scholia,[71] but also involving other Wikibase instances. For the latter, refer to the following examples:

  • How to query a Wikibase through programmatically accessing its SPARQL endpoint is shown in a collection of jupyter notebooks, i.e. in a python environment, using data from this instance, DHwiki Wikibase.[72]
  • The INCEpTION text annotation tool[73] can be connected to Wikidata[74] or any other Wikibase for the retrieval of items for manually supervised entity matching, or of lexemes, senses and/or forms for dictionary linking.[75]
  • The OpenRefine application[76] can be connected not only to Wikidata but to any Wikibase.[77][78] OpenRefine is widely used for entity reconciliation (matching of literal values to Wikibase entities),[79] though it can also be used for data enrichment (i.e., for upload, see below).[80]

Data upload

QuickStatements

QuickStatements[81] is a widely used tool for batch editing Wikibase instances. It allows users to create, modify, or delete large numbers of statements, labels, or descriptions through a simple tabular syntax.

Data can be pasted directly into the interface or uploaded as a CSV file, making it well-suited for mass imports and structured changes. Once submitted, QuickStatements processes each instruction as a secure background batch job, applying changes to the Wikibase via the API.

Cradle

Cradle[82] is a tool for creating Wikibase items using forms, which are created according to EntitySchemas.[6]

It is particularly useful for Wikibase administrators who want to facilitate the reliable manual creation of large numbers of items while ensuring they adhere to predefined data models and remain consistent across the database.

In institutional environments where multidisciplinary teams—contributors often come with very different levels of technical expertise, cradle provides a clear, structured way to enter data that aligns with institutional standards, enabling librarians, scholars, catalogers, and other domain specialists to contribute meaningful structured data without the need for extensive training.

OpenRefine

OpenRefine[83] is an open-source desktop application for data cleanup and transformation to other formats, an activity commonly known as data wrangling. It is similar to spreadsheet applications, and can handle many data file formats like spreadsheet, etc, but it behaves more like a database. It can manage extremely big datasets and perform upload, edit and delete operations using the Wikibase web API. There are several applications available for this:

  • OpenRefine Wikibase,[78] for reconciliation of a tabular dataset with a Wikibase knowledge graph;
  • OpenRefine Wikibase Media,[84] for loading and editing batches of multimedia files to a Wikibase instance.
  • Albin Larsson's Reconcile Against any MediaWiki Instance reconciliation service for OpenRefine.[85]


Meiners & Bulle (2025) explain in detail the capabilities of the OpenRefine application, in the context a Cultural Heritage use case, from where we cite:[86]

Data conditioning and cleaning: e.g. normalising labels, cleaning and enrichments (like adding coordinates via API based on address data), and filtering candidates through facets;

Contextual reconciliation: matching not only by string similarity but by additional features such as location, inception date, geographic coordinates and subject header;

Clustering and validation: identifying near-duplicates through algorithms such as fingerprinting and Levenshtein distance, or by defining custom clustering rules that combine multiple criteria to reflect domain-specific matching logic. The latest version of OpenRefine extend these capabilities with flexible distance-based and bin-defined clustering functions, allowing users to group values dynamically based on expressions rather than fixed algorithms;

Schema alignment: structuring results for upload to Wikidata and Wikimedia Commons with clear provenance and license information as well as structured data;

Template export: transforming curated and reconciled data into structured output formats for external systems.

There are several OpenRefine tutorial videos available on Wikimedia Commons,[87] and on YouTube.[88]


MediaWiki API

The MediaWiki Action API is a underlying technical web service interface that enables programmatic access to any MediaWiki-powered site. It allows developers and tools to read, edit, and manage wiki content without using the standard web interface. The MediaWiki API provides a number of methods to interact with Wikibase.[89]

Through dedicated endpoints, users can send HTTP requests and retrieve page content, query metadata, upload files, modify pages, and perform administrative actions. The API supports multiple output formats (JSON, XML, PHP) and includes robust authentication mechanisms to ensure secure access.

This makes it essential for bots, automated maintenance tools, third-party applications, and integration services with external systems that need to interact with wikibase at scale. Software development packages like the python WikibaseIntegrator[18] or Wikibase-sdk[90] for javascript use this API. For example scripts that involve WikibaseIntegrator, refer to the jupyter notebooks provided by the developer,[91] or, for example, to a collection of scripts used for uploading data to a Wikibase Cloud instance (Lindemann 2025).

The API documentation[92] and versioning system help maintain backward compatibility while supporting the diverse ecosystem of tools built around MediaWiki platforms.

Wikibase REST API

The Wikibase REST API[93][94][95][17] is modern web service interface included in newer distributions of Wikibase, and is available in Wikibase Cloud instances.[96] It is designed to offer a simpler, more structured way to interact programmatically with Wikibase entity data compared to the traditional, more complex MediaWiki API, which is intended to eventually supersede. Today, most of the tooling relies on the MediaWiki Action API. The REST API is available in Wikidata since 2023/05,[97] in Wikibase Suite since 2025/03.[98] and in Wikibase Cloud since 2025/12.[99][100] The feature can be activated in the local configuration.[101] Lexeme users should consider the REST API for lexemes will not be available in the near future.[102]

User scripts

Having an account in a Wikibase instance allows many customizations to the user interface:[103] users can change their interface language (by default English), can change the languages they visualize in the termbox by adding a Babel to their user page,[104] and, most importantly, can change the aspect of the pages of the website, both in the graphical appearance and in the editing functions. These personalizations can be made through user scripts, whose code is entirely public; these scripts can be activated in the Gadgets section of the Preferences (if the Gadgets extension is used in the Wikibase instance[105] and the user scripts have been created in the namespace MediaWiki;[106] the gadgets available in the preferences are listed on the page SpecialGadgets[107]), or through the user subpages "/common.js".[108] and "/common.css"[109] Users can directly add lines of code in their common.js and common.css pages, or can import user scripts made by other users in their common.js using the importScript function. In Wikidata statistics about the most used user scripts are currently updated monthly,[110] as well as general lists of user scripts arranged by topic,[111] and specific lists of userscripts created by specific WikiProjects[112] or for specific events.[113] Gadgets from Wikidata can be, sometimes with adaptations, reused in other Wikibase instances. The main API for Wikibase are the Hooks JS[114].

Related presentation video

Load video
Vimeo
Talk at ADHO conference, Lisbon 2025. Slides available at https://doi.org/10.5281/zenodo.15874344

References

  1. ↑ See https://mediawiki.org/wiki/Category:Stable_extensions.
  2. ↑ See https://w.wiki/R$Mt.
  3. ↑ See https://www.mediawiki.org/wiki/Extension:WikibaseQualityConstraints).
  4. ↑ See https://addshore.com/2024/07/entityschema-and-the-entity-flip-flop/.
  5. ↑ See https://www.mediawiki.org/wiki/Extension:EntitySchema.
  6. ↑ 6.0 6.1 For Wikidata Entity Schemes, see https://wikidata.org/wiki/Wikidata:Schemas.
  7. ↑ See https://www.mediawiki.org/wiki/Wikibase/Docker.
  8. ↑ https://www.wikimedia.de/
  9. ↑ See https://www.wbstack.com/.
  10. ↑ 10.0 10.1 See https://wikibase.cloud.
  11. ↑ See examples at ENwiki.
  12. ↑ See https://www.semantic-mediawiki.org/.
  13. ↑ See https://qlever.dev/.
  14. ↑ See https://doc.wikimedia.org/mediawiki-core/master/php/.
  15. ↑ See https://doc.wikimedia.org/mediawiki-core/master/js/.
  16. ↑ See https://www.mediawiki.org/wiki/Special:MyLanguage/API:Action_API.
  17. ↑ 17.0 17.1 See https://www.wikidata.org/wiki/Wikidata:REST_API
  18. ↑ 18.0 18.1 See https://toolhub.wikimedia.org/tools/wikibaseintegrator.
  19. ↑ See https://github.com/RDFLib/rdflib.
  20. ↑ https://github.com/maxlath/wikibase-sdk
  21. ↑ See an example for an item at EntityData/Q130741564.ttl, and for a lexeme at EntityData/L1347.ttl.
  22. ↑ Wikibase/Ontology-Federation
  23. ↑ See https://phabricator.wikimedia.org/T406228.
  24. ↑ 24.0 24.1 See File:Wikibase Live Session November 2025 - Federated Values Demo.webm.
  25. ↑ See https://phabricator.wikimedia.org/project/view/4604/.
  26. ↑ See https://ffmpeg.org/.
  27. ↑ 27.0 27.1 See https://geojson.org/.
  28. ↑ See https://www.fabbers.com/tech/STL_Format.
  29. ↑ See https://github.com/KhronosGroup/glTF/.
  30. ↑ See Lizardtech DjVu Reference, DjVu v3.
  31. ↑ See https://commons.wikimedia.org/wiki/Commons:Timed_Text.
  32. ↑ See https://commons.wikimedia.org.
  33. ↑ See https://www.wbstack.com/.
  34. ↑ See https://wikimedia.de.
  35. ↑ For this instance, DHwiki Wikibase, you can access detailed information at Special:Version#mw-version-ext.
  36. ↑ See: https://www.wikibase.cloud/terms-of-use
  37. ↑ See Hosting_policy_draft_v0
  38. ↑ See https://wikiba.se.
  39. ↑ See https://meta.wikimedia.org/wiki/Wikibase/Consultants_and_Support_Providers.
  40. ↑ The Wikibase Ontology in OWL is available as TTL file at https://wikiba.se/ontology/.
  41. ↑ See https://www.mediawiki.org/wiki/Wikibase/DataModel/Primer.
  42. ↑ See the list of datatypes on the Wikidata help pages at https://www.wikidata.org/wiki/Help:Data_type
  43. ↑ See https://www.wikidata.org/wiki/Wikidata:WikiProject_Schemas and https://www.weso.es/WShEx/
  44. ↑ https://www.mediawiki.org/wiki/Extension:WikibaseMediaInfo
  45. ↑ Source: https://www.wikidata.org/wiki/Special:EntityData/Q42.json
  46. ↑ https://www.wikidata.org/wiki/Help:Namespaces
  47. ↑ See https://doc.wikimedia.org/Wikibase/master/php/docs_topics_json.html.
  48. ↑ See example https://www.wikidata.org/wiki/Special:EntityData/Q1631944.rdf
  49. ↑ For lexemes, see Lexemes chapter.
  50. ↑ See an example for an item at https://wikidata.org/wiki/Special:EntityData/Q42.ttl (Douglas Adams), and for a lexeme at https://wikidata.org/wiki/Special:EntityData/L1347.ttl.
  51. ↑ The Wikibase OWL ontology can be obtained as TTL file at http://wikiba.se/ontology.
  52. ↑ See https://www.mediawiki.org/wiki/Wikibase/DataModel#Datatypes_and_their_Values.
  53. ↑ See also https://www.wikidata.org/wiki/Help:Data_type.
  54. ↑ https://www.mediawiki.org/wiki/Extension:Wikibase_Local_Media
  55. ↑ https://www.mediawiki.org/wiki/Extension:Wikibase_EDTF
  56. ↑ https://www.mediawiki.org/wiki/Extension:Score
  57. ↑ https://www.mediawiki.org/wiki/Extension:EntitySchema
  58. ↑ https://www.mediawiki.org/wiki/Extension:Math
  59. ↑ https://www.mediawiki.org/wiki/Extension:MathSearch
  60. ↑ https://mediawiki.org/wiki/Extension:WikibaseLexeme
  61. ↑ https://www.mediawiki.org/wiki/Extension:Wikibase
  62. ↑ https://epsg.org/crs_4326/
  63. ↑ See https://www.wikidata.org/wiki/Help:Wikimedia_language_codes/lists/all.
  64. ↑ https://www.wikidata.org/wiki/Help:Dates#Time_datatype
  65. ↑ For example, the query service of this instance is available at https://dhwiki.wikibase.cloud/query/. Wikidata's query service is available at https://query.wikidata.org/; for Wikidata, the University of Freiburg provides, as an alternative, https://qlever.dev/wikidata. See also https://phabricator.wikimedia.org/tag/wikidata_query_ui/.
  66. ↑ See https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service.
  67. ↑ See https://wikidata.org/wiki/Wikidata:SPARQL_query_service/Federated_queries.
  68. ↑ 68.0 68.1 See https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/Wikidata_Query_Help/Result_Views.
  69. ↑ See https://www.wikidata.org/wiki/Wikidata:SPARQL_query_service/queries/examples/advanced.
  70. ↑ The 2025 Data Reuse Days page gives a good overview: https://www.wikidata.org/wiki/Event:Data_Reuse_Days_2025.
  71. ↑ See https://scholia.toolforge.org/.
  72. ↑ The notebooks collection is accessible at https://github.com/hibernator11/dhwiki-notebooks.
  73. ↑ See homepage at https://inception-project.github.io/.
  74. ↑ See https://inception-project.github.io/releases/39.2/docs/user-guide.html#knowledge_bases_in_getting_started.
  75. ↑ For linking text tokens to lexical entries, see https://github.com/inception-project/inception/discussions/4928.
  76. ↑ See homepage at https://openrefine.org/.
  77. ↑ See https://wetneb.github.io/new-openrefine-website/docs/manual/wikibase/configuration/.
  78. ↑ 78.0 78.1 https://gitlab.com/nfdi4culture/openrefine-reconciliation-services/openrefine-wikibase
  79. ↑ See https://openrefine.org/docs/manual/reconciling.
  80. ↑ See complete guidelines at https://en.wikiversity.org/wiki/OpenRefine_to_Wikibase:_Data_Upload_Pipeline.
  81. ↑ See https://wikidata.org/wiki/Help:QuickStatements; the tool for Wikidata is available at https://quickstatements.toolforge.org/; a rewritten version of QuickStatements is available at https://qs-dev.toolforge.org/ (the manual is https://meta.wikimedia.org/wiki/QuickStatements_3.0/Documentation).
  82. ↑ This page explains Cradle for Wikidata: https://wikidata.org/wiki/Wikidata:Cradle.
  83. ↑ See homepage ate http://openrefine.org/
  84. ↑ https://gitlab.com/nfdi4culture/openrefine-reconciliation-services/openrefine-wikibase-media
  85. ↑ See https://byabbe.se/2024/08/05/reconcile-against-any-mediawiki-instance.
  86. ↑ Source of the list: https://openhumanitiesdata.metajnl.com/articles/10.5334/johd.412#32-openrefine-as-game-master
  87. ↑ See the category page https://commons.wikimedia.org/wiki/Category:OpenRefine_video_tutorials.
  88. ↑ See https://www.youtube.com/results?search_query=open+refine+tutorial.
  89. ↑ For the API sandbox of this instance, see Special:ApiSandbox.
  90. ↑ See https://github.com/maxlath/wikibase-sdk.
  91. ↑ See https://github.com/LeMyst/WikibaseIntegrator/tree/master/notebooks.
  92. ↑ See https://www.mediawiki.org/wiki/API:Action_API.
  93. ↑ See https://tech-news.wikimedia.de/2023/09/07/starting-fresh-the-wikibase-rest-api/.
  94. ↑ See https://doc.wikimedia.org/Wikibase/master/php/repo_rest-api_README.html.
  95. ↑ See https://doc.wikimedia.org/Wikibase/master/js/rest-api/.
  96. ↑ For this instance, see Special:Version#mw-version-entrypoints.
  97. ↑ See the Wikidata-tech mail list.
  98. ↑ See: Wikibase Community User Group maillist
  99. ↑ See Wikibase Community User Group maillist.
  100. ↑ See https://phabricator.wikimedia.org/T362769
  101. ↑ See Wikibase: How to enable the API
  102. ↑ See Phabricator ticket T329096
  103. ↑ Cf. https://commons.wikimedia.org/wiki/File:NLL2024_-_Advantages_of_an_account_in_Wikidata.pdf for an overview.
  104. ↑ Cf. https://www.wikidata.org/wiki/Help:Babel.
  105. ↑ https://www.mediawiki.org/wiki/Extension:Gadgets.
  106. ↑ E.g. https://www.wikidata.org/wiki/MediaWiki:Gadget-Merge.js.
  107. ↑ In Wikidata https://www.wikidata.org/wiki/Special:Gadgets.
  108. ↑ E.g. https://www.wikidata.org/wiki/User:Magnus_Manske/common.js.
  109. ↑ E.g. https://www.wikidata.org/wiki/User:Magnus_Manske/common.css.
  110. ↑ https://wikidata.org/wiki/Wikidata:Database_reports/Gadget_usage_statistics.
  111. ↑ https://wikidata.org/wiki/Wikidata:Tools/Enhance_user_interface
  112. ↑ E.g. https://wikidata.org/wiki/Wikidata:WikiProject_Linked_Data_for_Production/Practical_Wikidata_for_Librarians.
  113. ↑ E.g. https://wikidata.org/wiki/Wikidata:WikiProject_Latvia/NLL_2024.
  114. ↑ Wikibase: Hooks JS
Retrieved from "https://dhwiki.wikibase.cloud/w/index.php?title=Structure_and_features_of_Wikibase_for_DH&oldid=3660"