¶ DatabaseLayer_Server
Go REST API with CRUD operations for KNOX database. The database API receives triples from groups C and D via it's POST endpoint, and makes these triples available to the Functionality Layer through a GET endpoint.
The code can be found in this repository.
¶ Getting Started
- Get the environment variables. All groups store their environment variables in a file called
/srv/data/.secreton serverknox-func01@srv.aau.dk.- The environment variables needed for this API are the ones under
for database api on knox-kb01.srv.aau.dk.
- The environment variables needed for this API are the ones under
- Create a file called
.envin the root of this repository and paste the environment variables into it. - Make sure that you are connected to the internet via AAU's network, because the API accesses a Virtuoso that is hosted there.
- Run
docker-compose up --buildin the root of this repository.
¶ Endpoint Documentation
Can be found here.
¶ Generate new documentation based on code
¶ Prerequisites
- We use
go-swaggerto generate Open-API 2.0 spec files automatically from code. See go-swagger for more info. - To turn the spec file into markdown, we use widdershins.
¶ Generate new documentation
swagger generate spec -m -o ./swagger.yaml
widdershins swagger.yaml -o swagger.md
Remember to insert the new markdown file in the wiki manually using ctrl+a ctrl+c ctrl+v on windows.
¶ Database
¶ Creating a Virtuoso database
This can be done locally and on a server.
Pull the latest Docker image to your system by running the command docker pull openlink/virtuoso-opensource-7 in the command line.
(Optionally) check to see if you're running the latest version by using the command docker run openlink/virtuoso-opensource-7 version.
Create folder where you need it by using Mkdir knox_virtuoso_db and navigate to it by Cd knox_virtuoso_db.
Run the Docker container specifying a password for admin access and which ports to be used:
docker run --name knox_virtuoso_db --interactive --tty --env DBA_PASSWORD=*** --publish 1111:1111 --publish 8890:8890 --volume pwd:/database openlink virtuoso-opensource-7:latest
Opening http://localhost:8890 displays the Virtuoso Conducter.
To login as admin, use username and password to authenticate.
¶ Inserting Ontologies Into Virtuoso
Tunnel into the host server and cd into the volume directory /home/student.aau.dk/database.
Use Curl to install the ontology. This requires a download link for the ontology.
Use a file-transfer system to create a download link. Example: WeTransfer
The volume will build the ontology file in the /database folder in the Virtuoso image.
Use Virtuoso's conductor interface to bulk load the ontology file. This will be done by using the Interactive SQL.
First use ld_dir ('source-filename-or-directory', 'file name pattern', 'graph iri');
to load the file from the specific directory.
Execute the bulk load by using rdf_loader_run();.
For verification use SELECT * FROM DB.DBA.LOAD_LIST; This can be used to check the list of data sets registered for loading, and the graph IRIs into which they will be or have been loaded.
Remember to end with a checkpoint;. This command MUST be run to commit the bulk loaded data to the Virtuoso database file.
Full example:
¶ Set Up Virtuoso Authorization
To ensure that only the admin or autherized users can interact with the endpoint, it is necessary to configure the autherization on Virtuoso. This exmple will use the admin user, with Username and password to authenticate.
Start by configuring the settings in Virtuoso. The first thing to do is to ensure that the nobody user does not have any permissions. This user is the default for unauthorized access.
To change the permissions, open the Interactive SQL in the conductor and execute this function:
DB.DBA.RDF_DEFAULT_USER_PERMS_SET ('nobody', 0);'.
The integer is the permission bit and follows this structure:
The dba user will always have the proper permissions, but if another user is needed make sure to set proper permission bits by following the structure above. For example, if a user would need full permission, you would set the bit as 15(1+2+4+8).
To configure an endpoint to require authorization, go to the Web Application Server tab in the conductor. Then choose Virtual Domains & Directories. In there, open the folder with the correct port and find the endpoint you want authorization. The /sparql-auth/ endpoint uses digest auth by default, but this example wants basic auth.
To configure this, we edit the /sparql endpoint. In here, scroll down to Authentication Options and:
- select
basicin Method - select
SPARQL Endpointin Realm - Select
DB.DBA.HP_AUTH_SQL_USERin Authentication Function
Remember to save changes.
Now when sending requests to the /sparql/ endpoint, the user will need to set the username and password in the header of a request using basic auth.
¶ Watchtower
The server uses Watchtower to pull the latest updates of the repository to the server.
The follwing docker run command can be used to start Watchtower on the server.
docker run -d --name watchtower -v /var/run/docker.sock:/var/run/docker.sock containrrr/watchtower --interval 30
Be sure to select an appropriate interval for how often Watchtower should check for updates
To deploy the Docker image of the repository to the server a GitHub Action needs to be created. The GitHub workflow can be found as Test and deploy under the Actions of this repository. The GitHub Action will trigger on pull requests to the main branch, and a Docker image of the repository will be made and uploaded to the GitHub Containter Registry (ghcr.io). The image of the repository is built from the Dockerfile located in the repository.
¶ Authors
- Casper Bruun Christensen caschr21@student.aau.dk
- Emily Treadwell Pedersen emiped21@student.aau.dk
- Malthe Reipurth mreipu21@student.aau.dk
- Matthias Munch Jakobsen mattja21@student.aau.dk
- Moritz Marcus Hönscheidt mhoens21@student.aau.dk
- Rasmus Louie Jensen rjen20@student.aau.dk