Print bookPrint book

Guide for Extensions

Issues like getting aligned with FIESTA-IoT’s semantic data models, testbed and resource registration, send data/observations and testbed management are explained in detail here.

Site: FIESTA-IoT Training Platform
Course: Guide for 3rd Parties
Book: Guide for Extensions
Printed by: Guest user
Date: Saturday, 21 October 2017, 7:25 PM

1 Getting aligned with FIESTA-IoT’s semantic data model

When it comes to put your testbed into the FIESTA-IoT federation, it is deemed necessary to get aligned to the data model (i.e. ontology) we have defined under the umbrella of the whole project. Otherwise, you will not be able to put your data into our repositories, since it will not pass the validation stage (nor the testbed registration module beforehand). As a direct consequence, as a testbed administrator, you will have to undertake the role of adapting your datasets into the semantically annotated formats that have been shaped up in our project.

Before going any further, we strongly recommend that you read through the ontology’s webpage the related deliverables (FIESTA-IoT D3.1, 2016) and (Project Deliverable D3.2) and the paper (Unified IoT Ontology to Enable Interoperability and Federation of Testbeds, 2016).

Moreover, we include the FIESTA-IoT ontology graph that depicts the ontologyy:

FIESTA-IOT ONTOLOGY

1.1 FIESTA-IoT ontology alignment phase

The FIESTA-IoT ontology is an outcome of a thorough analysis of all the different platforms that are part of the FIESTA-IoT’s initial testbed federation. These platforms are SmartSantander (Spain), Smart ICS (UK), Soundcity (France) and KETI (South Korea). Amongst the assets, where sensors stand out as the dominant realm, we have outlined the most appropriate means of how to describe a single resource, focusing on the essential aspect that must be defined for a complete minimal definition of a resource. These include: physical location, owner (testbed), sensor or sensors that are embedded in that concrete resource, their respective quantity kind and unit of measurement. Besides these key concepts, in the graph another set of concepts hold a secondary (optional) information, like VirtualEntity, Coverage or Source, just to name a few.

They are addressed to describe additional information about the resources, although they are not strictly necessary. Moreover, there is a Metadata class (right side of the figure), where any kind of information could be appended, thus opening the ontology to non-foreseen items (for instance, if a sensor follows a synchronous operation, we might want to represent its frequency or rate, said in other words, the time between consecutive observations).

Note that the current version of the ontology is clearly biased towards the sensor realm, as we only provide the concepts for ssn:SensingDevice. Although, we have introduced the concepts of iot-lite:ActuatingDevice and iot-lite:TagDevice, but we have not thoroughly dived into the details.

Together with the part in charge of the description of the resources, we have modelled how an observation harvested by one of these sensors would look like. Essentially, it answer the following questions: who (sensor that has actually measured the observation), where (physical location of the sensor), when (timestamp of the observation), what (quantity kind and unit of measurement of the observation) and last, and the most important, the value of the observation itself.

1.2 M3-lite taxonomy alignment phase

An essential part of the solution we have brought about is the gathering of the set of capabilities covered between the four heterogeneous testbeds, giving rise to a taxonomy that spans different types of sensing devices, all the quantity kinds, units of measurement and domains of interest of the different sensing devices (i.e. sensors).

As we have mentioned before, it is strongly recommended to go over this taxonomy and double check that all your assets actually match the concepts that we have already defined in the taxonomy. Despite covering a wide range of sensor types and domains of interest, you might have a subset that we have not contemplated so far. In such a case, the first thing you would have to do is to inform us, detailing the needs, in order to let us know your requirements. With this information, we would be able to modify the taxonomy and append needed concepts.

1.3 FIESTA-IoT Amendment proposals

As can be easily inferred from the two previous sections, the solution we propose for the semantic annotation of IoT resources might not fulfil everybody’s needs.


Recall that the ontology is a living entity and is prone to be modified during the project lifetime. As such, your testbed might have some elements that have not been envisaged or covered within a stable version of FIESTA-IoT ontology. In that case, we would need to upgrade it in order to suit them. Then, what you would have to do is to contact us in order to double check whether your proposed changes are worth modifying the picture or not. In essence, these changes might belong either to the ontology itself or to the m3-lite taxonomy, where these latter ones pertain to the concepts of quantity kinds, units of measurement or domains of interest.


The following list represents the fields that you have to fill for every element you want to bring to the ontology/taxonomy:

  • Need: Type of entity/class/concept you are about to define.
  • Legacy FIESTA-IoT: In case your proposal matches with any concept in FIESTA-IoT ontology, specify it.
  • Name: Your proposed concept name.
  • Comment: Provide a description of the concept that you are proposing.
  • SubclassOf: If the element is a subclass of any other, specify its parent. Otherwise, we assume that you inherit from ssn:SensingDevice (as most of the proposals will have to do with updates to the taxonomy) for the sensor, qu:QuantityKind for quantity kinds, qu:Units for Units and m3-lite:DomainOfInterest for domain of interest.
  • rdfs:isDefinedBy or rdfs:seeAlso. Provide any existing reference to your proposal in case you have borrowed the concept from another existing ontology.
  • Justification for addition: Provide justification on why do you need it and if there is a near match in the Taxonomy.



Apart from the list above, in case you do want to make any change to the ontology, you have to provide also a modified picture of it including your stuff and the way it is connected with the rest of the classes. This would help us identify and determine whether its addition can be integrated or not.

Last, but not least, this is an exercise that shall be done in the initial phase of your 6 months. Thus, we have set a deadline till which you can send us your proposals.

2 Best practices for producing valid the annotations

Below we describe some of the best practices that testbed owners should follow in order to create semantic annotation. Please note that the APIs provided in the section 1.6 already use/satisfy already use/satisfy the best practices. The best practices are:

  • Do not use blank nodes. Create individuals for concepts used.
  • Have meaningful URIs for Individuals.
  • Respect the cardinality provided within the ontology
  • Do not push unnecessary/fake/test data
    • Remove unused concepts. They might cause validation errors. Further, you are not obliged to provide certain tuples that have predicates such as http://www.w3.org/2002/07/owl#equivalentProperty,  http://www.w3.org/2002/07/owl#inverseOf
  • Make sure that annotations for observations are not repeated. Meaning you do not replicate information. This will unnecessarily use the network resources.
  • ssn:SensingDevice individuals should have identical URIs in observation graph and resource graph
  • In observation realm, uuse both ssn:madeObservation and ssn:observedBy. This also applies to all other properties that have inverse relationships.
  • Do not abuse iot-lite:Metadata by providing just any information about the sensing device.
  • Get the annotations validated first before pushing data to FIESTA-IoT repository by using certification portal or validation services.

3 Implementation of Testbed to FIESTA-IoT annotators

After having defined all the mappings between your platform’s assets (both in terms of resource descriptions and observations), including the possible modifications that may be brought about on the ontology, it is time to talk about the implementation of the annotator that would convert your datasets to that compliant with FIESTA-IoT ontology (data model).

For that, you can choose among a number of different RDF serialization the ones listed below (annotated according to the “Content-type” field of the HTTP header):

  • application/ld+json
  • application/n-quads
  • application/n-triples
  • application/n3
  • application/rdf+json
  • application/rdf+thrift
  • application/rdf+xml
  • application/trig
  • application/trix
  • application/trix+xml
  • application/turtle
  • application/x-trig
  • application/x-turtle
  • null/rdf
  • text/csv
  • text/n-quads
  • text/n3
  • text/nquads
  • text/plain
  • text/rdf+n3
  • text/trig
  • text/turtle

In order to show you an example of how to annotate a resource or an observation, we have included Appendix II where an illustrative example of one of the SmartSantander’s resources using json-ld as the serialization format is provided. Besides, you can dynamically query another Content-Type by means of the IoT-Registry API (FIESTA-IoT Consortium, 2016).

https://platform.fiesta-iot.eu/annotator/api/testbed

Method

Description

Request Body (example)

POST

Service to generate a semantically annotated testbed document (with or without subjacent resources)

{

"id": <Testbed IRI (string)>,
"devices": [
{
"id": <Device 1 unique ID (string)>,
"location": {
"lat": <Latitude (double)>,
"lon": <Longitude (double)>
},
"sensing_devices": [
{
"id": <Device 1 Sensor 1 unique ID (string)>,
"qk": <m3-lite’s QuantityKind (string)>,
"uom": <m3-lite’s Unit (string)>
},
{
"id": <Device 1 Sensor 2 unique ID (string)>,
"qk": <m3-lite’s QuantityKind (string)>,
"uom": <m3-lite’s Unit (string)> }
]
},
{
"id": <Device 2 Sensor 1 unique ID (string)>,
"location": {
"lat": <Latitude (double)>,
"lon": <Longitude (double)>
},
"type": "AirThermometer",
"qk": "TemperatureAmbient",
"uom": "DegreeCelsius"
}
]
}



In case case you want to check whether your annotations are compliant with our ontology, you can use the FIESTA-IoT Ontology Validator to receive a real-time validation report. Moreover, in case your annotated document is not a 100% match with the correct format, this report will not only return the binary correct/error output, but will also provide a detailed explanation of the actual source(s) of the error(s).

3.1 Annotator as a Service

Besides the option of designing and developing the whole annotator module on your own and deploying it at the tested level, we have developed APIs to ease the creation of annotations. These are the annotator-as-a-service APIs. Using the annotator-as-a-service APIs you can pass your legacy data in a defined way to the service. The service will return the semantically annotated FIESTA-IoT Ontology compliant annotations.

In a nutshell, you can generate annotated documents just by sending simple JSON objects to the corresponding endpoints that are resumed in following tables. However, note that the nature of these web services is to provide a generic way to annotate resources, so providing complex descriptions will not be appropriate through this approach. It is understood that for complex cases, you will have to carry out the annotator on your own.

Testbed description generation through the annotator service:

https://platform.fiesta-iot.eu/annotator/api/testbed

Method

Description

Request Body (example)

POST

Service to generate a semantically annotated testbed document (with or without subjacent resources)

{

  "id": <Testbed IRI (string)>,

  "devices": [

    {

      "id": <Device 1 unique ID (string)>,

      "location": {

        "lat": <Latitude (double)>,

        "lon": <Longitude (double)>

      },

      "sensing_devices": [

        {

          "id": <Device 1 Sensor 1 unique ID (string)>,

          "qk": <m3-lite’s QuantityKind (string)>,

          "uom": <m3-lite’s Unit (string)>

        },

        {

          "id": <Device 1 Sensor 2 unique ID (string)>,

          "qk": <m3-lite’s QuantityKind (string)>,

          "uom": <m3-lite’s Unit (string)>        }

      ]

    },

    {

      "id": <Device 2 Sensor 1 unique ID (string)>,

      "location": {

        "lat": <Latitude (double)>,

        "lon": <Longitude (double)>

      },

      "type": "AirThermometer",

      "qk": "TemperatureAmbient",

      "uom": "DegreeCelsius"

    }

  ]

}

Resource description generation through the annotator service:

https://platform.fiesta-iot.eu/utils/annotator/device

Method

Description

Request Body (example)

POST

Service to generate a semantically annotated resource document 

{

  "id": <Device i unique ID (string)>,

  "location": {

    "lat": <Latitude (double)>,

    "lon": <Longitude (double)>

  },

  "sensing_devices": [

    {

      "id": <Device i Sensor 1 unique ID (string)>,

      "qk": <m3-lite’s QuantityKind (string)>,

      "uom": <m3-lite’s Unit (string)>

    },

    {

      "id": <Device i Sensor 2 unique ID (string)>,

      "qk": <m3-lite’s QuantityKind (string)>,

      "uom": <m3-lite’s Unit (string)>

    }

  ]

}

Observation description generation through the annotator service:

https://platform.fiesta-iot.eu/annotator/api/observation

Method

Description

Request Body (example)

POST

Service to generate a semantically annotated observation document 

{

"observed_by": <Sensor unique ID (string)>,

  "location": {

    "lat": <Latitude (double)>,

    "lon": <Longitude (double)>

  },

  "quantity_kind": "TemperatureAmbient",

  "value": <Value>,

  "format": <Type (integer, float, etc.)>,

  "unit": <m3-lite’s Unit (string)>,

  "timestamp": <Date>

}

As for the response of these methods, the iot-registry module returns appropriate message along with the status code in the same formats as the ones enumerated in Section 1.5.

4 Certification Suite

In order to succeed your certification process and get a valid certificate, you need to prepare your FIESTA-IoT account and your testbed before you can start pushing data to our platform.

  1. certification process must be conducted with your FIESTA-IoT account. This account makes the proof of the identity of the testbed under certification. If you do not have one yet, please refer to section 3 to get one
  2. The output data must be semantically annotated using the FIESTA-IoT ontology. You need also to be aware of the serialization format of your semantic data (i.e. RDF/XML, JSON-LD) and be able to extract one sample of your data stream which constitute a stand-alone semantic-annotated piece of information (i.e. with requested prefix used in the annotation, complete data graph). This “stand-alone” sample need to pass the ontology validation to make your certification process progress
  3. You must implement at least one of the four TPS API methods and the endpoint must be accessible for FIESTA−IoT platform. This is crucial for FIESTA-IoT to get data from your testbed. For more information of the TPS, please refer to section 4.3 and the sample code.

4.1 Log in to the Certification Portal

Go to the certification portal. Choose ““OpenAM” as Authentication mode, and use your FIESTA-IoT OpenAM credentials to log in. If you don’t remember your password, please contact the FIESTA-IoT support team (oc-support@fiesta-iot.eu). The link “Forgot password” just under the “password” textfield only works for the authentication mode “Normal”. If you have forgotten the password you created when you signed up on the FIESTA-IoT Portal, you can also reset your password, but in this case you will have to do it on the FIESTA-IoT Portal “Forgot password” link.

4.2 Launch the Certification Process

Just after the log in, you are on your home page where historical test reports and certifications are displayed. To launch a certification process, just click on “Launch certification” in the menu and you should see the following web page.



On this page, the certification process is explained. Follow the instructions on the page. It contains 3 assessment/tests that you need to perform to get a valid certification for joining FIESTA-IoT.

  1. Self-assessment scorecard. This scorecard is a questionnaire covering several aspects of your testbed. Answer it regarding to your testbed.
  2. Ontology validator. You need to provide a sample of your annotation and validate it against the Fiesta-IoT ontology. More details will be given in the following section.
  3. Interoperability testing. According to the FIESTA-IoT TPS specification, your testbed needs to implement at least one TPS API methods to be able to provide your semantic data. Choose one among the four TPS API tests regarding to your implementation in the drop-down menu. More details will be given in the following section.


Use the ontology validator

In step 3 ontology validator, you can choose either upload your semantic annotation as a file, or copy-paste directly the annotation using “direct input”.



If you choose “Direct input”, you need to specify the serialization type using the drop-down menu.



On the next page, you need to choose only FIESTA-IoT as reference ontology, and check all the four types of errors.

Click on the blue button “Validate” to launch the validation



Perform the Interoperability test

In the drop-down list, there are four TPS interoperability tests. According to your implementation, choose one among them to be tested. For example, if you have implemented both getObservations() a,d getLastObservation() API, you can either choose “tps getobservations test” or “tps getlastobservation test”.

The textfields below are automatically adapted to the test you just chose. Following the tooltip next to each textfield title, you are invited to provide the information about the testbed to make the interoperability tests able to be executed on your testbeds. In principle, the tests will play the role of Fiesta−IoT platform and simulate the interactions between the testbed and FIESTA-IoT.




Get the certification

After having passed all the three assessment/tests, you are directed to the certification result page as shown in the following figure:



If you do not have all the three tests passed, i.e. if you see a red or yellow flag rather than a green one, you can still click the button to print the report, but it will not be considered as a valid certification to be able to join Fiesta−IoT platform. 

Once all the tests are passed, you need to click on the blue button “Print certification report” to get your certificate in PDF format and send it to the FIESTA-IoT support team (oc-support@fiesta-iot.eu). In this e-mail you can ask for your account to be upgraded to Testbed Admin so that you have the rights to continue with the registration of your testbed and resources. In this e-mail, apart from the PDF file with your certificate, you have to include the username of the account you want to be upgraded. Note that for the moment, there is a limit of one account with Testbed Admin rights per testbed.

5 Testbed & Resource registration

In this section, we introduce the User Interfaces that are provided by FIESTA-IoT to facilitate the testbed and resource registration process within the Fiesta−IoT platform

5.1 Testbed Registration

The Testbed Registration UI can be found at the FIESTA-IoT portal under the Testbed Provider Menu (see 1st Figure below) item named the “Register Testbed”. 

Register Testbed page (see 2nd Figure below) will show the list of registered testbeds and enable user to register a new testbed. The admin user can view all the registered testbeds, whereas the testbed providers only see their own testbeds that they have registered.

In principle, you should see an empty list the first time you enter into the portal and, once you have registered your testbed, you should see it on the list. As far as you are registering only one testbed, there should be only one row listed in the UI.

The user can register a new testbed by clicking the “Register new testbed” button, a register testbed page (see 3rd Figure);  a register testbed page will be shown where the user inputs testbed related information:

  • Input IRI. This has to be unique for each registered testbed. Note that this IRI is the one that you will have to use when creating your annotated resource descriptions for the ssn:Deployment individual. This is important since it is the way of linking your devices to your testbed. For example, if the IRI you provide in the form is: http://testbed.test.com#testTestbed, then your devices’ descriptions will have to include a statement like: http://testbed.test.com#device1 ssn:hasDeployment http://testbed.test.com#testTestbed.
  • Input Annotated Resource Observation. This is a sample observation as it results from your annotator. Although this has been already checked during the certification process, the testbed registration form is created to do on-line validation. You must specify the appropriate data format by selecting a format from combo-box list. Validate it by clicking Validate Observation button in order to enable the Save button at the bottom of the testbed registration form.
  • Input Annotated Resource Description. This is a sample resource description as it results from your annotator. Although this has been already checked during the certification process, the testbed registration form is created to do on-line validation. You must specify the appropriate data format by selecting a format from combo-box list. Validate it by clicking Validate Annotated Resource button in order to enable the Save button at the bottom of the testbed registration form.
  • Input latitude and longitude. These are the GPS coordinates of your testbed. They are mainly used for map representation of your testbed. Typically, core of the testbed if it is distributed among a large area (e.g. city-hall for smart city).
  • Input Name. Just for informational purposes. Human-friendly name to be used for identifying your testbed.

Other remaining textboxes are optional. However, some of them are still important for completing the testbed integration.

  • Input Get Last Observations URL, Get Observations URL, Push Last Observations URL and Push Observations URL. You have to specify the URL for the TPS interface(s) that you have developed for your testbed. Only one out of the four options is necessary. Identify the one that you have implemented and provide the corresponding endpoint. Those that are not provided will not be available for Testbed Provider Interface (TPI) Configuration.
  • Input Get API Key and Push API Key. In case your TPS endpoint is protected using an API Key, you have to provide it. This API key is the one that the Fiesta−IoT Platform will use when calling at your TPS. Depending on which of the four options (i.e. two Get-based or two Push-based) you have implemented you have to provide the corresponding API key(s). If your endpoint(s) is not protected, then leave the field blank.
  • Note that if you support both types of TPS, then you should provide the corresponding API key for each of them (if it is the same one, you have to copy it in both fields)

When the user clicks the Save button, if the registration process is successful the register testbed page will disappear and a success message is displayed along with the identifier of the registered testbed.

You can confirm that the registration has been correctly stored at the Fiesta−IoT Platform semantic repository by taking the following steps:

1. Since Fiesta−IoT Platform hides your testbed details, it gives your testbed a unique ID that you can get by making an HTTP call similar to the one below:

GET /iot-registry/api/utils/identifier/to_fiesta_iot?type=testbed&amp;value=http%3A%2F%2Ftestbed.test.com%23ucExperimenterUser HTTP/1.1

Host: platform.fiesta-iot.eu

iPlanetDirectoryPro: AQIC5wM2LY4SfcwEw_Jg.........

Note that a valid SSO Token has to be previously obtained and added to the request. Also note that the appropriate Host has to be used. The IRI you provided when registering your testbed has to be provided as the query parameter value. As it is shown in the example, special characters like colon, slash, hash, etc., which are prohibited for URLs, have to be transformed into its appropriate URL encoding.

2. Make the following HTTP Request (i.e. accessing your testbed URL as result of step 1):

GET 

/iot-registry/api/testbeds/1B2F37ZYQ5A4I5... HTTP/1.1

Host: platform.fiesta-iot.eu

iPlanetDirectoryPro: AQIC5wM2LY4SfcwEw_Jg.......

....

Note that a valid SSO Token has to be previously obtained and added to the request. Also note that the appropriate Host has to be used. The result should be an annotated document of your testbed as an ssn:Deployment node in RDF format.

5.2 Resource Registration

After registering a testbed with the specified IRI, a user can use this IRI for registering resources related to the testbed. Register Resources can be found at the Fiesta−IoT portal under the Testbed Provider Menu item named the “Register Resources”. Once the user chooses this option, resource registration can be done through three options (see following Figure):

  • Register Devices Manual,
  • Register Device by Text,
  • Register Devices by Upload.

5.3 Register Devices Manually

When registering devices manually, the user must select a testbed (IRI) for which he/she wants to register devices manually (see 1st Figure).

After selecting the registered testbed IRI, user clicks on ‘Register Devices manual’ button and a form is displayed (see 2nd Figure).

Where the different field to fill in are the following ones:

  • Sensor ID. This is the URI with which you internally identify the testbed devices.
  • Latitude and Longitude. These are the GPS coordinates of your device. If this is a mobile device, then use representative ones (e.g. bus depot for device installed on a bus).
  • QuantityKind. This is the phenomena observed by the sensor.
  • Unit of Measurement. This is the unit used for the sensor observations.

After this, users can click the Save button to complete the devices manual registration process. Upon success, the user will return back to the main options window of ‘Register Resources’ (see 3rd Figure below).

You can confirm that the sensor description has been correctly stored at the Fiesta−IoT Platform semantic repository by making the following HTTP Request:

GET 

/iot-registry/api/testbeds/1B2F37ZYQ.../resources HTTP/1.1

Host: platform.fiesta-iot.eu

iPlanetDirectoryPro: AQIC5wM2LY4SfcwEw_Jg.......

Use the path found when registering your testbed and the appropriate Host. Note that a valid SSO Token has to be previously obtained and added to the request.

The result should be a list of the Sensor IDs that you have already registered.

5.4 Register Devices by text

When registering devices by text, testbed providers can input an annotated document with the resource descriptions of as many devices as they need. Firstly, the RDF serialization format used must be specified. Then input the text of the annotated resource description(s) into the text-area (see 1st Figure).

After providing the text for registering the device(s), you can Validate Annotated Resource for double checking or go directly to the registration by clicking on the Save button.

Upon success, the user will return back to the main options window of ‘Register Resources’.

You can confirm that the sensor description(s) has been correctly stored at the Fiesta−IoT Platform semantic repository by making the following HTTP Request:

GET 

/iot-registry/api/testbeds/1B2F37ZYQ.../resources HTTP/1.1

Host: platform.fiesta-iot.eu

iPlanetDirectoryPro: AQIC5wM2LY4SfcwEw_Jg........

Use the path found when registering your testbed and the appropriate Host. Note that a valid SSO Token has to be previously obtained and added to the request.

The result should be a list of the Sensor IDs that you have already registered.

5.5 Register Devices by Upload

When registering devices by Upload, the user must select the format for serializing the annotated resource(s) description used within the document and then upload the file to register the resource(s) (see 1st Figure). The current max size of the file upload is set to 10MB.

The system will only register resources after validating the document. If the process was successful on save, the window register by upload will disappear and user will return to the Register Resources page with a green message.

You can confirm that the sensor description(s) has been correctly stored at the Fiesta−IoT Platform semantic repository by making the following HTTP Request:

GET 

/iot-registry/api/testbeds/1B2F37ZYQ.../resources HTTP/1.1

Host: platform.fiesta-iot.eu

iPlanetDirectoryPro: AQIC5wM2LY4SfcwEw_Jg.........

Use the path found when registering your testbed and the appropriate Host. Note that a valid SSO Token has to be previously obtained and added to the request.

The result should be a list of the Sensor IDs that you have already registered.

6 Testbed Provider Services (Testbed to FIESTA-IoT)

In this subchapter you can find relevant information about the four following topics:

  1. TPS API Definition and
  2. TPS Implementation (Wildfly Container)

6.1 TPS API Definition

The 1st Table below illustrates the main API primitives that support the Testbed Provider Services functionalities, while 2nd Table provides more details about each one of the functions that comprise the API.

List of primitives comprising the Testbed Provider Services API

<<interface>>

TpiServicesInterface

---

POST:getLastObservations (getLastObservationsPayload:JsonObject): 

String annotatedMeasurements,

POST:getObservations (getObservationsPayload:JsonObject):

String annotatedMeasurements,

POST:pushLastObservations (pushLastObservationsPayload:JsonObject):String

POST:stopPushOfObservations (stopPushOfObservationsPayload:JsonObject):String



A Testbed SHOULD implement methods of the Testbed Provider Services API as specified in 2nd Table below:

Testbed Provider Services API definition

The accepted content types (Content-Type) mentioned above are:
• application/ld+json
• application/n-quads
• application/n-triples
• application/n3
• application/rdf+json
• application/rdf+thrift
• application/rdf+xml
• application/trig
• application/trix
• application/trix+xml
• application/turtle
• application/x-trig
• application/x-turtle
• null/rdf
• text/csv
• text/n-quads
• text/n3
• text/nquads
• text/plain
• text/rdf+n3
• text/trig
• text/turtle



Object Definition

The analysis of the JSON objects that are introduced in the previous section is provided below:

Definition of getLastObservationsPayload object:

{

"sensorIDs": [<String> sensorIDs]

}

 

Example:

{

    "sensorIDs": [

        "http://testbed.test.com#sensing_device2",

        "http://testbed.test.com#sensing_device1"

    ]

}

 

Definition of getObservationsPayload object:

{

     "sensorIDs": [<String> sensorIDs],

     "startDate": <Date> startDate,

     "stopDate": <Date> stopDate

}

Note: the Date has ISO 8601 date and time format ("yyyy-MM-ddThh:mm:ss") in UTC and the start date precedes the stop date in time. If the startDate and the stopDate appear in other format, they have to be converted to the above one.

Definition of pushLastObservationsPayload object:

{

"sensorIDs": [<String> sensorIDs],

    "endpointURI": <String> theEndpointURI

}

 

Example:

{

    "sensorIDs": [

        "http://testbed.test.com#sensing_device2",

        "http://testbed.test.com#sensing_device1"

    ],

    "endpointURI": "https://platform.fiesta-iot.eu/dms/pushObservationsProxy"

}

Definition of stopPushOfObservations object:

{

    "sensorIDs": [<String> sensorIDs],

    "endpointURI": <String> theEndpointURI

}

 

Example:

{

    "sensorIDs": [

        "http://testbed.test.com#sensing_device2",

        "http://testbed.test.com#sensing_device1"

    ],

    "endpointURI": "https://platform.fiesta-iot.eu/dms/pushObservationsProxy"

}

6.2 TPS Implementation (Wildfly Container)

A TPS instance may be implemented in any language and or environment the Testbed provider chooses to as long as it follows the above API as a REST web service. Shall you choose to implement it in Java we provide a skeleton Java project in order to help you bootstrap the process. Information where this project can be found and to be used is provided in the following sections.



Code Availability and Structure

The Testbed Provider Interface components is offered at FIESTA-IoT GitHub repository, and more specifically under the testbed.tpi repository. The latest version of the components are under the “develop branch”.

The project is divided into two main packages. The “eu.fiestaiot.tpi.api.tps.rest” and “eu.fiestaiot.tpi.api.tps.impl.dataservices”:

1) eu.fiestaiot.tpi.api.tps.rest

Within this package, the “TpiApiTestbedProviderServices.java”[1]  class can be found which is the entry point for the Services described at the TPS API above. The main methods of interest are:

  • public Response getLastObservations(String getLastObservationsPayload)
  • public Response getObservations(String getObservationsPayload)
  • public Response pushLastObservations(String pushLastObservationsPayload)
  • public Response stopPushOfObservations(String stopPushOfObservationsPayload)

These methods analyse the payload and forwards it to the implementation of them, which resides in the following package.

2) eu.fiestaiot.tpi.api.tps.impl.dataservices

In this package, we have a list of the implementations classes of the different web services exposed from the TPS instance. The Testbed provider needs to implement at least one of them (or two of them in the case of pushLastObservations since it needs the stopPushOfObservations as well). The four classes are:

• GetLastObservationsImpl.java GetLastObservationsImpl.java [2]
• GetObservationsImpl.java GetObservationsImpl.java [3]
• PushLastObservationsImpl.java [4]
• StopPushOfObservationsImpl.java [5]



Push Observations Implementation Details

In order to enable the option of a Testbed to push its observations, Fiesta−IoT platform is exposing an interface that accept various annotated document types and is called “pushObservationsStreamProxy”. The endpoint URI of the “pushObservationsStreamProxy” is received within the payload of “pushLastObservations” as shown at the TPS API description above. As soon as the testbed has new data (or based on a specific schedule defined by the testbed) from the “pushLastObservations” received sensor ID list it should send the annotated measurements within the body of the message and the document type as the header of the message. An example endpoint would be:

https://{{domain}}/tpi.api.dms/rest/dataservices/pushObservationsStreamProxy

And the header should contain:

• Content-Type: DocumentType (for example: application/rdf+xml)
• iPlanetDirectoryPro : {{UserCookie}}


The accepted content types (Content-Type) are:

• application/ld+json
• application/n-quads
• application/n-triples
• application/n3
• application/rdf+json
• application/rdf+thrift
• application/rdf+xml
• application/trig
• application/trix
• application/trix+xml
• application/turtle
• application/x-trig
• application/x-turtle
• null/rdf
• text/csv
• text/n-quads
• text/n3
• text/nquads
• text/plain
• text/rdf+n3
• text/trig
• text/turtle

The UserCookie SHOULD be generated by using the credentials of the user as described above (3.2 Invoking the OpenAM Authentication API to obtain a SSO token). For Example by sending as a header the Username and Password:

  • X-OpenAM-Username : UserX
  • X-OpenAM-Password : PasswordOfUserX

To the following URL:

http://{{domain}}:8090/openam/json/authenticate

More details can be found in the "Invoking the OpenAM Authentication API to obtain a SSO token" section of the "FIESTA-IoT Security Framework" book.



System Requirements

The TPS component described above is deployed within a WildFly container and is using Maven as project management.

The prototype runs on Windows, Linux, Mac OS X, and Solaris. In order to run the prototype, you need to ensure that Java 8 and WildFly are installed and/or available on your system. In order to build the prototype you will also need Maven. Before attempting to deploy and run the prototype applications, make sure that you have started WildFly. More details about the specific versions of the tools and libraries that have been used for the development or that are required for the deployment and execution of the prototypes are given in the section below.



Install & Run

The prototype has been implemented as a Maven-based web application. Below WILDFLY_HOME indicates the root directory of the WildFly distribution, and PROJECT_HOME indicates the root directory of the project (tpi.api.tps).

In order to build the prototype, run the following command in PROJECT_HOME:

mvn clean package

Finally, in order to deploy the prototype, run the following command in PROJECT_HOME:

mvn wildfly:deploy

The last step assumes that WildFly is already running on the machine where you run the command.

The produced, from the build process above (mvn clean package), project can be found within PROJECT_HOME/target/ folder and is the tpi.api.tps.war file. Alternatively copy the produced (from the build process above) tpi.api.tps.war file from the target directory (PROJECT_HOME/target/), into the WILDFLY_HOME/standalone/deployments directory of the WildFly distribution,

in order to be automatically deployed.

If the deployment has been successfully completed, the implemented web services described in the section above will be exposed under the following URLs:

  • http://[HOST]:[PORT]/tpi.api.tps/rest/tps/getLastObservations
  • http://[HOST]:[PORT]/tpi.api.tps/rest/tps/getObservations
  • http://[HOST]:[PORT]/tpi.api.tps/rest/tps/pushLastObservations
  • http://[HOST]:[PORT]/tpi.api.tps/rest/tps/stopPushOfObservations

where [HOST] is the host and [PORT] the port that WildFly uses.





[1] /tpi.api.tps/src/main/java/eu/fiestaiot/tpi/api/tps/rest/TpiApiTestbedProviderServices.java

[2] /tpi.api.tps/src/main/java/eu/fiestaiot/tpi/api/tps/impl/dataservices/GetLastObservationsImpl.java

[3] /tpi.api.tps/src/main/java/eu/fiestaiot/tpi/api/tps/impl/dataservices/GetObservationsImpl.java

[4] /tpi.api.tps/src/main/java/eu/fiestaiot/tpi/api/tps/impl/dataservices/PushLastObservationsImpl.java

[5] /tpi.api.tps/src/main/java/eu/fiestaiot/tpi/api/tps/impl/dataservices/StopPushOfObservationsImpl.java

 

7 Testbed Configuration management

This section will guide you on how to enable the transmission of observations from your testbed to the FIESTA platform. The TPI configurator is a tool in the Fiesta−IoT Platform portal that is used for the configuration of observation retrieval from the testbed. This in turn controls the Data Management Service (DMS) that directly interacts with a testbeds Testbed Provider Service (TPS) on how they will interact with each other. This could be in the form of several patterns, mainly either through polling the testbed, or the testbed pushing to the platform. This will be dependent on what TPS interface(s) have been implemented by the testbed provider.

To start the process, the FIESTA-IoT web portal must be accessed by an authorised testbed provider. In the menu panel, the TPI configurator should be selected

In the TPI configurator page, there are 3 tabs:

  • Discovery – is used to select the resources of a testbed(s) that were registered in the IoT registry.
  • Schedule – is used to select the TPS interface for the testbed and define the schedule for the DMS to interact with it.
  • Status – this will list the current schedules that are being executed.

7.1 Discovery and Selection

In the Discovery tab, select the testbed that you would like the TPI/DMS to interact with.

As soon as the tool opens it identifies the user and automatically searches for Testbeds bound with the logged-in User ID. The available testbeds are listed at the Testbed dropdown list of the tool’s Discovery tab (see 2nd Figure below).

When a testbed is chosen, the registered resources appear at the resource list where the user can select in order to proceed with their data retrieval scheduling (see 2nd Figure above and 3rd Figure below) by hitting next.

The next step is to select the Resources that are to be scheduled. 

If all Resources are to be selected, make sure that you scroll through the whole list to make sure that all resources have been loaded into the list.

Use the arrowed buttons to move the Resources from the available list to the selected list. Once the Resources are selected, proceed to the Schedule tab.

7.2 Scheduling and Status

In this section, we are going to introduce the User Interface that is provided by Fiesta−IoT in order to facilitate the testbed provider to manage the testbed behaviour with the Fiesta−IoT platform. Fiesta−IoT offers a TPI configuration UI that enables the testbed provider to discover the semantically registered resources of his/her testbed, by utilizing the IoT-Registry services. Moreover, it manages the data retrieval process, by utilizing the TPI DMS services.

The TPI Configurator UI can be found at the Fiesta−IoT portal under the Testbed Provider Menu (see 1st figure) by clicking the “TPI Configurator”.

The Testbed provider can now define a specific schedule to retrieve their measurements. This is achieved by defining it in the “Schedule” tab as shown in the 2nd Figure. In the schedule tab the user can specify the properties of the schedule to be defined and these are:

  • a name for the schedule,
  • to choose the Testbed ‘s exposed service (TPS) (see 2nd Figure) that has already been defined in the Testbed registration process. This Testbed URI will expose one of the TPS services (i.e. getLastObservations, pushLastObservations, etc.),

  • In order for Fiesta−IoT DMS to access the Testbed URI above, if your testbed is protected with API Key, select the Security Key (see 3rd Figure), as you reported it in the Testbed registration process. If your testbed does not use an API Key for authorizing requests on your TPS, leave it blank.

  • To specify a timeschedule (if the get methodology for TPS has been implemented) for the data-retrieval (get) to be executed from the DMS to TPS component. This is achieved by identifying:
    • The start time (optional): which defines a historical date in order to adittionally retrieve historical measurements for a resource (i.e. to retrieve the data from last month set the start time date 30 days in the past). Note that this feature is not curently implemented.
    • The frequency and the time unit of it which defines the periodicity frequency of the measurements retrieval (get) from the Testbed’s TPS.
    • If push-based TPS method is selected (i.e. pushLastObservations), the Time Schedule options are not visible.

After completing the required flow and the definition of the requested properties, the testbed provider needs to hit the schedule button (see Figure above) to initiate both the equivalent service of the TPI DMS and the process of pushing data to Fiesta−IoT platform or retrieving data from the Testbed (depending on the TPS service implemented from the Testbed TPS).

This step will take the Testbed Provider to the status tab where an overview of all the scheduled jobs can be found. In case the Testbed Provider needs to stop a job, the job should be picked from the Status list and the stop button should be clicked . This stops the schedule for the specific job of retrieving measurements from TPS (in case the get method has been implemented) or instructs TPS to stop pushing measurements to DMS (in case the push methodology was chosen).