DataShare Plugin

The purpose of this addon is to retrieve structured data from Confluence pages. To make this data consumable by third-party applications it is provided in JSON format. The addon subscribes to page create/update/delete events. When a page is saved the addon checks its contents for certain class names that identify objects and attributes. It then parses the page to create JSON that it stores in Confluence database.

The addon provides REST endpoints to third-party applications, system services for integration with other Confluence plugins, CQL function to filter using JSON data and other features described below. Please take a look at a 5-minute introduction video.

Using inline attributes

The most simple use case for this addon is to annotate a page with attributes to create a simple JSON object:

Inline Attribute

You create inline attributes the same way that you would create inline comments in Confluence. The plugin reuses some of the inline comments functionality, namely the Text Highlight, so for the plugin to work correctly Confluence Inline Comments plugin should be installed and enabled (which is by default).

To retrieve a page JSON you need to use REST endpoint provided by DataShare plugin: {CONFLUENCE_HOME}/rest/data-share/1.0/page/{pageId}. This is the most important resource that the plugin provides to third-party applications. For more details about the plugin’s REST API please refer to Using DataShare REST API section below.

Using Endpoint

The data that you define with inline attributes is immediately available. This may not be the case with heavy long-running operations, such as Confluence space import. That said, the plugin is very fast: in tests it processed a thousand of pages in a matter of seconds, thanks to efficient SAX-parser. It can be used with Confluence Data Center installations, though not ‘officially’ marked as DC-compatible (not yet).

Objects and attributes

JSON is made of objects and their attributes. The plugin provides annotation for both attributes and objects using custom HTML class names.

Attributes

An attribute uses two classes: attribute name and attribute type. Their respective classes are:

Class Role Example
dsattr-{name} Attribute name dsattr-first-name
dsattrtype-{type} Attribute type dsattrtype-page

Attribute Classes

When you create an inline attribute you define only dsattr-{name} class, but not an attribute type class. By default attribute type is considered to be ‘any type’, but a user can restrict attribute to be of a specific type:

  • text - plain text
  • page - Confluence page link
  • date - Confluence date label
  • user - Confluence user mention
  • link - email or weblink

An attribute type can also be an object type which is anything but these five predefined types. If an attribute is given an object type, than any other data but objects of that type will be discarded by page processor.

Objects

Objects are made of attributes.

Object

To define an object you need to assign class dsobject to some element that encapsulates attribute elements. A common use is to define table cell (<TD>) as an attribute, and table row (<TR>) or whole table (<TABLE>) as an object.

Object Classes

An object can have three classes: dsobject (fixed), object type and object id:

Class Role Example
dsobject Object marker dsobject
dsobjtype-{type} Object type dsobjtype-event
dsobjid-{id} Object identifier dsobjid-437876c7-27b9-3522-a6e9-8815cc66adf6

As mentioned earlier an object type is used by page processor in combination with attribute type: if object type matches attribute type than the object is accepted as an attribute value. If, on the other hand, object type does not match attribute type, the object is discarded.

Every object is given an id that should be unique across your Confluence system. Object ids are auto-generated and preserved during page edit/trash/restore operations. Third-party applications should use that id to uniquely identify your Confluence object.

Confluence page is considered to be a root object for other objects on page, so Confluence page id should be used to uniquely identify the page’s root data object.

Using templates

You can use DataShare plugin with table-based Confluence templates. Use table context menu’s Data Share button to invoke Data Share dialog.

DataShare Button

The dialog lets you edit Confluence <TABLE>, <TR> and <TD> class names directly. To define a ‘vertical form’ it is sufficient to set dsattr-{name} and optionally dsattrtype-{type} classes to table cells (<TD>):

Vertical Form

If data is supposed to be entered in a table rather than a simple form the procedure will be a bit more complex:

  • Define an attribute name and type on a table. Attribute type is required if you want to filter out irrelevant text in the table. For example, an attribute name could be ‘events’ and attribute type could be ‘event’. Do not pay attention to extra classes in the textarea at the bottom of the dialog - they are not relevant for your purpose.

Table Classes

  • Choose the last record in the table and define it to be an object (note the checkbox) of type ‘event’.

Row Classes

  • For each data cell in the row define an attribute name and optionally an attribute type.

Cell Classes

Save the page when you are done defining classes. The row that you marked as dsobject will be treated as ‘template row’. Use Confluence table menu’s Insert Above, Insert Below or copy-paste buttons to create copies of the row and fill in your data.

You can also use a Hidden Row plugin’s Add Row Button macro to hide the template row and thus prevent it from undesirable user modifications.

Using filters

DataShare plugin provides CQL-function pathDataFilter(...) that can be used to filter Confluence pages using data behind them. The function returns a list of page ids that match filter condition passed as a parameter to the function. For example

id in pathDataFilter("author = 'Mesilat Limited'")

will result in a list of pages with attribute author equal to text ‘Mesilat Limited’.

Please refer to Confluence documentation for information about using CQL.

Other valid filter expressions:

total = 129
/total = 129 or /book/author = 'John'
not total = 129 and (/book/author = 'John' or /book/author = 'Sonya')

The function supports the following operators:

Operator Usage
< less than
<= less than or equal
> greater than
>= grater than or equal
= equal
!=, <> not equal
~ case insensitive substring match
and logical ‘and’
or logical ‘or’
not negation
(, ) grouping operator

An example of using CQL-function from command line:

$ curl -u username:password https://conftest.mesilat.com/rest/api/content/search\?cql=id%20in%20pathDataFilter\(%22first-name%20%3D%20\'John\'%22\)
{
  "results": [
    {
      "id": "1769526",
      "type": "page",
      "status": "current",
      "title": "John Lennon",
      "restrictions": {},
      "_links": {
        "webui": "/display/ds/John+Lennon",
        "tinyui": "/x/NgAb",
        "self": "https://conftest.mesilat.com/rest/api/content/1769526"
      },
      "_expandable": {
        "container": "",
        "metadata": "",
        "extensions": "",
        "operations": "",
        "children": "",
        "history": "/rest/api/content/1769526/history",
        "ancestors": "",
        "body": "",
        "version": "",
        "descendants": "",
        "space": "/rest/api/space/ds"
      }
    }
  ],
  "start": 0,
  "limit": 25,
  "size": 1,
  "_links": {
    "self": "https://conftest.mesilat.com/rest/api/content/search?cql=id%20in%20pathDataFilter(%22first-name%20%3D%20%27John%27%22)",
    "base": "https://conftest.mesilat.com",
    "context": ""
  }
}

Using DataShare pages macro

DataShare pages macro is a table report based on data stored in Confluence database.

Report

The report takes three parameters: CQL, Columns and Captions. The report will print only records for pages that match the CQL filter provided. This can be a label filter, such as label = artist, a pathDataFilter() function-filter, for example id in pathDataFilter("author = 'Mesilat Limited'") or any other valid CQL expression.

Columns parameter is a comma-separated list of attributes to print. If no columns are specified the report will only print a single column with links to Confluence pages. The same will be true when Columns parameter value is set to ‘page’, so page is a reserved word that refers to Confluence page with data behind it.

Captions parameter is pretty self-explanatory: it list column captions, comma-separated.

Plugin integration

DataShare plugin provides events that could be used by third-party Confluence plugins to access data. The events are fired:

  • when page is saved
  • when page is trashed / restored
  • when inline attribute is defined on page
  • when Confluence space is imported
  • when REST API POST/PUT/DELETE methods are invoked
Class Description
DataObjectCreateEvent The event is fired when root data object is created
DataObjectUpdateEvent The event is fired when root data object is updated
DataObjectDeleteEvent The event is fired when root data object is deleted

A third-party plugin could implement data validation using event listeners and PageProcessingTask.addNotification(...) method:

@Named
public class DataValidator implements InitializingBean, DisposableBean {
    private static final String LABEL = "artist";
    @ComponentImport
    private final EventPublisher eventPublisher;

    @Override
    public void afterPropertiesSet() throws Exception {
        eventPublisher.register(this);
    }
    @Override
    public void destroy() throws Exception {
        eventPublisher.unregister(this);
    }

    @EventListener
    public void dataObjectCreateEvent(DataObjectCreateEvent event) {
    }
    @EventListener
    public void dataObjectUpdateEvent(DataObjectUpdateEvent event) {
        Page page = (Page) event.getSource();
        if (hasLabel(page)){
            if (event.getRootObject() != null
                && event.getRootObject().has("first-name")
            ) {
                if (!"John".equals(event.getRootObject()
                    .get("first-name").asText())
                ){
                    event.getTask().addNotification(
                        PageProcessingTask.Status.WARNING,
                        "The name was supposed to be John!"
                    );
                }
            }
        }
    }
    @EventListener
    public void dataObjectDeleteEvent(DataObjectDeleteEvent event) {
    }

    public DataValidator(EventPublisher eventPublisher) {
        this.eventPublisher = eventPublisher;
    }

    private boolean hasLabel(Page page){
        for (Label label: page.getLabels()){
            if (LABEL.equals(label.getName())){
                return true;
            }
        }
        return false;
    }
}

When a user tries to save invalid page he will get a warning message in Confluence:

Validation

Confluence Global Stylesheet could be used to modify the notification look and feel, for example:

div.com-mesilat-datashare-WARNING {
    background-color: #FFF0F5;
    margin-left: -50px;
    margin-right: -50px;
    margin-top: 16px;
    margin-bottom: -20px;
    padding-top: 20px;
    padding-bottom: 20px;
}

div.com-mesilat-datashare-WARNING ul {
    list-style: none;
}

div.com-mesilat-datashare-WARNING ul li {
    margin-bottom: 10px;
}

Validation

Using DataShare REST API

DataShare API

Method Endpoint Details
GET page/{id} Details…
GET pages?id={id}... Details…
POST pages Details…
DELETE page/{id} Details…
POST page/{id} Details…
PUT page/{id} Details…
GET find?q={text}... Details…
GET find2?q={text}... Details…
POST inline Details…

GET page

GET /rest/data-share/1.0/page/{id} Returns root data object for a page identified by id parameter.

$ curl -u username:password https://conftest.mesilat.com/rest/data-share/1.0/page/1769526
{
  "first-name" : "John",
  "last-name" : "Lennon",
  "birthdate" : "Oct 09, 1940",
  "instruments" : "guitar"
}

GET pages

GET /rest/data-share/1.0/pages?id={id}... Returns page details and root data objects for pages identified by id parameter(s).

$ curl -u username:password https://conftest.mesilat.com/rest/data-share/1.0/pages\?id=1769526\&id=1769528
[ {
  "id" : 1769526,
  "title" : "John Lennon",
  "href" : "https://conftest.mesilat.com/rest/data-share/1.0/page/1769526",
  "view" : "https://conftest.mesilat.com/pages/viewpage.action?pageId=1769526",
  "data" : {
    "first-name" : "John",
    "last-name" : "Lennon",
    "birthdate" : "Oct 09, 1940",
    "instruments" : "guitar"
  }
}, {
  "id" : 1769528,
  "title" : "Paul McCartney",
  "href" : "https://conftest.mesilat.com/rest/data-share/1.0/page/1769528",
  "view" : "https://conftest.mesilat.com/pages/viewpage.action?pageId=1769528",
  "data" : {
    "first-name" : "Paul",
    "last-name" : "McCartney",
    "birthdate" : "June 18, 1942",
    "instruments" : "bass guitar"
  }
} ]

POST pages

POST /rest/data-share/1.0/pages Returns page details and root data objects for pages identified by id parameter(s). Same as GET but without limitations of GET request URL (allows any number of ids).

$ curl -u username:password -H 'Content-Type: application/json' -d '[1769526,1769528]' https://conftest.mesilat.com/rest/data-share/1.0/pages
[ {
  "id" : 1769526,
  "title" : "John Lennon",
  "href" : "https://conftest.mesilat.com/rest/data-share/1.0/page/1769526",
  "view" : "https://conftest.mesilat.com/pages/viewpage.action?pageId=1769526",
  "data" : {
    "first-name" : "John",
    "last-name" : "Lennon",
    "birthdate" : "Oct 09, 1940",
    "instruments" : "guitar"
  }
}, {
  "id" : 1769528,
  "title" : "Paul McCartney",
  "href" : "https://conftest.mesilat.com/rest/data-share/1.0/page/1769528",
  "view" : "https://conftest.mesilat.com/pages/viewpage.action?pageId=1769528",
  "data" : {
    "first-name" : "Paul",
    "last-name" : "McCartney",
    "birthdate" : "June 18, 1942",
    "instruments" : "bass guitar"
  }
} ]

DELETE page

DELETE /rest/data-share/1.0/page/{id} Deletes root data object for a page identified by id parameter.

$ curl -u username:password -X DELETE https://conftest.mesilat.com/rest/data-share/1.0/page/1769526

POST page

POST /rest/data-share/1.0/page/{id} Overwrites root data object for a page identified by id parameter. Same as PUT.

curl -u username:password -H 'Content-Type: application/json' -d "{ \
  \"first-name\" : \"John\", \
  \"last-name\" : \"Lennon\", \
  \"birthdate\" : \"Oct 09, 1940\", \
  \"instruments\" : \"guitar\" \
}" https://conftest.mesilat.com/rest/data-share/1.0/page/1769526

PUT page

PUT /rest/data-share/1.0/page/{id} Overwrites root data object for a page identified by id parameter. Same as POST.

curl -u username:password -X PUT -H 'Content-Type: application/json' -d "{ \
  \"first-name\" : \"John\", \
  \"last-name\" : \"Lennon\", \
  \"birthdate\" : \"Oct 09, 1940\", \
  \"instruments\" : \"guitar\" \
}" https://conftest.mesilat.com/rest/data-share/1.0/page/1769526

GET find

GET /rest/data-share/1.0/find Find data objects given a text substring to search in page titles. Other parameters are:

  • label – page label or labels
  • limit – maximum number of pages to fetch
curl -u username:password https://conftest.mesilat.com/rest/data-share/1.0/find?label=artist\&limit=1\&q=john
[ {
  "id" : 1769526,
  "title" : "John Lennon",
  "href" : "https://conftest.mesilat.com/rest/data-share/1.0/page/1769526",
  "view" : "https://conftest.mesilat.com/pages/viewpage.action?pageId=1769526",
  "data" : {
    "first-name" : "John",
    "last-name" : "Lennon",
    "birthdate" : "Oct 09, 1940",
    "instruments" : "guitar"
  }
} ]

GET find2

GET /rest/data-share/1.0/find2 Find data objects given a text substring to search in page titles and data. Other parameters are:

  • label – page label or labels
  • limit – maximum number of pages to fetch
curl -u username:password https://conftest.mesilat.com/rest/data-share/1.0/find2?label=artist\&limit=1\&q=1942
[ {
  "id" : 1769528,
  "title" : "Paul McCartney",
  "href" : "https://conftest.mesilat.com/rest/data-share/1.0/page/1769528",
  "view" : "https://conftest.mesilat.com/pages/viewpage.action?pageId=1769528",
  "data" : {
    "first-name" : "Paul",
    "last-name" : "McCartney",
    "birthdate" : "June 18, 1942",
    "instruments" : "bass guitar"
  }
} ]

POST inline

POST /rest/data-share/1.0/inline Add data attribute to a root data object.

curl -u username:password -H 'Content-Type: application/json' -d "{ \
  \"attributeName\": \"isbn\", \
  \"index\": 0, \
  \"lastFetchTime\": \"1562516668518\", \
  \"numMatches\": 1, \
  \"pageId\": \"1769495\", \
  \"selectedText\": \"ISBN-0123456789\" \
}" https://conftest.mesilat.com/rest/data-share/1.0/inline

This addon is free under MIT license. Source code is available at GitHub. To report a bug please use the issue tracker