Help:About data
Wikidata is a free knowledge base that can be read and edited by both humans and machines. It is just one of many wiki-based projects hosted and maintained by the Wikimedia Foundation, a free-content nonprofit organization probably best known for Wikipedia. Each of the Wikimedia Foundation's projects has its own focus—for example, Wikipedia is for encyclopedia content, Wikimedia Commons supports image and other media files, and Wiktionary provides lexical information about words like definitions and synonyms. The focus of Wikidata is structured data.
This page is intended as an overview of structured data. If you are already familiar with structured data, but want to learn more about its specific use on Wikidata, how to access the data on Wikidata, or how to contribute your own project's data to Wikidata, please jump ahead to the section about linking data.
Understanding Wikidata
[edit]Structured data refers to data that has been organized and is stored in a defined way, often with the intention to encode meaning and preserve the relationships between different data points within a dataset.
But what is data, anyway? And why should you care about structured data in particular?
Defining data
[edit]Big data, experimental data, open data, metadata—you may have encountered some or even all of these terms before.
Each term means something a little different but all are built on a common understanding of data and its potential for describing and improving our understanding of the world around us.
As an abstract concept, data can be thought of as a precursor to information, meaning that information can be inferred or derived from data.
This is because data when boiled down to its essence is simply a set of values about things. These values can be numeric or quantitative like a measurement or an amount. They can also be qualitative, like a description or a comparison. For example, we can say that "8,848 m (29,029 ft)" is a data value about the height of Mount Everest and that "red" is a data value about the color of a car.
As previously mentioned, information is not the same as data but is instead a product of the collection and analysis of data. For example, 8,848 (data) is a somewhat meaningless number on its own even if we know it's the height of a mountain; we only can say that Mount Everest is the highest mountain in the world at 8,848 m (information) if we are aware of standard measurements of height and once we know the height of other mountains. It becomes a lot easier to make such inferences, gain new insights and knowledge, and establish facts when data is structured—we will return to this idea later.
Where is data?
[edit]Data is all around us. There are many kinds of data sources, including financial, biological, and social data. Even this page has data! For example, it has a total word count, dates it was created and last revised, a topic and subject matter, a number of page views, and languages that the content is available in.
However, while everything is potentially a source of data, data that is not recorded and organized may as well not exist at all. Without an underlying structure, data appears meaningless and fails to provide useful information.
By organized, we mean categorized in a standard and unambiguous way. The organized and categorized data is what we refer to when we say structured data.
Where is structure?
[edit]On the web, structure reigns. Most websites are created using HTML, a markup language which provides the basic scaffolding, or structure, of a web page.
Markup languages are also used for tagging and describing page content so that search engines, bots, and applications like RSS feeds can easily process and "understand" it. For example, <title> tags tell machines what the name of a website is.
Instead of supporting the structure and common elements of a web page, Wikidata provides structure for all the information stored in Wikipedia, and on the other Wikimedia projects. Wikidata is based on the Mediawiki software as is any other Wikimedia project, extended by Wikibase, the software which powers Wikidata and is designed to manage large amounts of structured data. Structure is not directly added to the content of Wikipedia or other Wikimedia site pages, as in tables or lists, nor is any knowledge of markup languages, data schemas, object notation, or other special syntax required by Wikidata users; instead, data is added to and edited in Wikidata through user-friendly input forms.
All data stored on Wikidata can be used to generate all kinds of automated and up to date lists or tables or other structured pages in any Wikimedia site or elsewhere.
| Data for Mountains | ||
|---|---|---|
| Mountain | Property | Value |
| Mount Everest | height | 8,848 m |
| K2 | hauteur | 8,611 m |
| Kanchenjunga | height | 8,586 m |
| Lhotse | height | 27940 ft |
Structuring data
[edit]For an example on the importance of structure, let's look at Table 1. In this table we can see data for the four highest mountains on Earth. If we would like to know a particular piece of information, such as the height of the second highest mountain in the world, we should be able to look at the provided data and find out the correct value. However, only three of the four mountains have their data categorized as a height value, and only two of those three mountains have values in metres. While we know that height and hauteur (French for height) can be understood as equal to each other, and how to convert metres to feet or vice versa, a machine, such as a bot or a computer program may not.
It would be much easier for both humans and machines to process the information and answer the original question about the second highest mountain when all underlying data is recorded in a similar way even if the presentation differs.
Modeling data
[edit]Collections of structured data, like Wikidata, are organized according to a data model. Data models are machine-readable, meaning they can be understood by a computer. While computers are powerful, they are often not as smart as us when it comes to simple reasoning. For instance, in the example above, a machine would not be able to know that height and hauteur are the same unless they were explicitly told somehow that was the case.
-
item: Earth -
property: highest point -
value: Everest
| Data for Mountains | ||
|---|---|---|
| Mountain | Property | Value |
| Mount Everest | continent | Asia |
| K2 | continent | Asia |
| Kanchenjunga | continent | Asia |
| Lhotse | continent | Asia |
Data models vary based on the analysis needs, scope and conceptual framework of the dataset, and the technical requirements of a system. However, all data models typically will specify what kind of data can be supported by a system and what relationships between values can be understood and represented. For example, a data model could specify that height and hauteur be mapped to each other so that both terms represent one concept, or that measurements in feet be automatically converted into metres. The Wikidata data model shapes the way that data can be edited and added to the system by users. It is also a work in progress, with new data types being added to the model over time.
The data model also essentially translates human natural language patterns into something that can be processed by machines. For example, in English we might say:
- "Mount Everest is the highest mountain in the world"
This is also the raw, unstructured format of content currently on Wikipedia and all other Wikimedia sites.
On Wikidata, this would be represented by a statement, which consists of a property-value pair about an item, in this case Earth:
Earth (Q2) (item) → highest point (P610) (property) → Mount Everest (Q513) (value)
Additionally, Wikidata would also hold a statement about the item for Mount Everest (indicating it is a mountain):
Mount Everest (Q513) (item) → instance of (P31) (property) → mountain (Q8502) (value)
Note that because other items can be used as the values for statements, and all items have their own unique page on Wikidata, this means that all items in the system can be linked together through a series of statements. Because Wikidata uses a machine-readable format, this interlinking of data allows new relationships and connections to be discovered and processed by machines. For example, in Table 2 we see new data for our mountains, this time about their geographical location by continent but nothing about their heights. Assuming this continent data was linked to the mountain height data, we would feel more confident making predictions or drawing certain conclusions about it, like saying that Asia is home to the world's highest mountains.
Linking data
[edit]Besides being a collection of structured data, Wikidata also supports linked data. Linked data refers to the practice of publishing structured data so that it can be interlinked.
For Wikidata this means that volunteer-contributed data can also be linked to other datasets, databases, and data sources from all around the web and from diverse initiatives outside of the Wikimedia family. For example, Wikidata currently allows interlinking with datasets and databases as diverse as Google Books, Canmore (one of the Historic Environment Scotland databases), the Vatican Library, OmegaWiki, and MusicBrainz.
By following linked data principles and practices, Wikidata is also able to support and be used by other projects.
Linked data principles
[edit]Wikidata uses unique identifiers, or uniform resource identifiers (URIs), for all its items as per linked data standards.
While Wikidata uses a unique data model, its content can be exported in RDF, a widely used and standard format for linked data. In Wikidata terms, a statement is composed of an item and a property-value pair. For those familiar with linked data concepts, an item can be viewed as the subject part of a triplet; the property represents a triplet's predicate; and a value is used to express the object of a triplet.
However, Wikidata statements may also contain elements beyond the subject-predicate-object, such as references and qualifiers (for more information, see Help:Statements). This makes it complicated to fully represent Wikidata's content using the language of RDF—more information on these challenges can be found in the document "Introducing Wikidata to the Linked Data Web".
Contributing data
[edit]If you have datasets you would like to contribute to Wikidata, please see Wikidata:Data donation.
Accessing data
[edit]The data in Wikidata is published under the Creative Commons Public Domain Dedication 1.0, allowing the free reuse of the data. You can copy, modify, distribute and perform the data, even for commercial purposes, all without asking permission.
See Data access for details about the different ways to programmatically access Wikidata's data.
See also
[edit]For related pages, see:
For additional information and guidance, see:
- Project chat, for discussing all and any aspects of Wikidata
- Wikidata:Glossary, the glossary of terms used in this and other Help pages
- Help:FAQ, frequently asked questions asked and answered by the Wikidata community
- Help:Contents, the Help portal featuring all the documentation available for Wikidata