Loading data in turtle RDF format to TerminusDB

TerminusDB
January 30, 2020
Tutorials & Examples

TerminusDB loves RDFs! Although not an RDF DB, TerminusDB uses the W3C’s OWL language to define the schema of its databases, and OWL is normally encoded as RDFs.

This is a quick tutorial to show you how to load data in turtle RDF format into TerminusDB. This tutorial relates to an early version of TerminusDB, 1.1.

Loading data in turtle RDF format to TerminusDB is elegant and simple.

Just as a revision, when loading the csvs in the our tutorials, we use:

				
					const csv = WOQL.get(
    WOQL.as("Start station","v:Start_Station")
        .as("End station", "v:End_Station")
        .as("Start date", "v:Start_Time")
        .as("End date", "v:End_Time")
        .as("Duration", "v:Duration")
        .as("Start station number", "v:Start_ID")
        .as("End station number", "v:End_ID")
        .as("Bike number", "v:Bike")
        .as("Member type", "v:Member_Type")
).remote("https://terminusdb.com/t/data/bike_tutorial.csv")

together with wrangles and inserts to load the data:

				
					const inputs = WOQL.and(csv, …wrangles); 
WOQL.when(inputs, inserts)

So now, we want to load turtle RDF, so instead of the csv, we can do:

				
					WOQL.with("graph://temp", 
        WOQL.remote("my_url", 
            {"type":"turtle"}), 
        WOQL.quad(  "v:Subject", 
                    "v:Predicate", 
                    "v:Object", 
                    "graph://temp")
)

That will read all the turtle data from my_url into a temporary in-memory graph graph://temp then we can use WOQL.when() to insert the items in the temporary graph to the database.

Here is an example, we are going to load in all the skills in the Language Skills Collection data at ESCO. We downloaded and unzip the file and put the lauguage_skills_collection.ttl in our mounted volume.

Create a Schema for the Data

Now create a new database and make a query:

WOQL.when(true, WOQL.doctype("skill").label("Skills").description("Skills extracted from ESCO dataset")
)

Now click on the Schema button on the left, we can see that we have created this schema:

Load data to a temporary in-memory graph for inspection

Now make a new query:

WOQL.limit(50).with("graph://temp",
     WOQL.file("/app/local_files/language_skills_collection.ttl",
               {"type":"turtle"}),
     WOQL.quad("v:Subject",
               "v:Predicate",
               "v:Object",
               "graph://temp")
     )

We use limit(50) to preview the data in the turtle file:

You can also make other queries by replacing the WOQL.quad()` above with other queries. For example, we want to grab all the class object, properties of those objects and their types:

WOQL.limit(50).with("graph://temp",
     WOQL.file("/app/local_files/language_skills_collection.ttl",
               {"type":"turtle"}),
     WOQL.and(
          WOQL.quad("v:Class_Obj", "type", "v:Class", "graph://temp"),
          WOQL.quad("v:Class_Obj", "v:Property", "v:Prop_Obj", "graph://temp"),
          WOQL.quad("v:Prop_Obj", "type", "v:Prop_Type", "graph://temp")
         )
    )

And it will give you the following result:

Now if we want to have a closer look at all the property http://www.w3.org/2004/02/skos/core#prefLabel of http://data.europa.eu/esco/model#Skill:

WOQL.with("graph://temp", WOQL.file("/app/local_files/language_skills_collection.ttl", {"type":"turtle"}),
WOQL.and(
     WOQL.quad("v:Skill_obj", "type", "http://data.europa.eu/esco/model#Skill", "graph://temp"),
     WOQL.quad("v:Skill_obj", "http://www.w3.org/2004/02/skos/core#prefLabel", "v:Skills", "graph://temp")
    )
)

Which shows us all the skills in the data:

Insert data from in-memory graph to database

To make the insert query, we will use a WOQL.when() like we did for the csvs:

const inputs = WOQL.with("graph://temp", WOQL.file("/app/local_files/language_skills_collection.ttl", {"type":"turtle"}),
WOQL.and(
     WOQL.quad("v:Skill_obj", "type", "http://data.europa.eu/esco/model#Skill", "graph://temp"),
     WOQL.quad("v:Skill_obj", "http://www.w3.org/2004/02/skos/core#prefLabel", "v:Skills", "graph://temp")
    )
);
const inserts = WOQL.insert("v:Skill_obj", "skill").label("v:Skills");
WOQL.when(inputs, inserts)

If you see the blue box there you know you have your data in the database:

To be sure, let’s make a simple query to the database:

WOQL.limit(50).star()

Voila! We have all the skills:

This content is a little dated and applies to TerminusDB Version 1.1. We’ve moved on quite a bit since then, check out our docs site for the latest tutorials and guidance. TerminusDB & TerminusX which is version 10, combines the power of knowledge graphs, with the simplicity of documents. Build apps and insights with the safety of version control, combined with the ability to create a vibrant knowledge graph of connected JSON documents. Visit our home page for more info.