Guide for Experimenters

1 Experiment Management

1.5 Run/Execute an experiment

An experiment can be executed in many ways and FIESTA-IoT provides solutions for the execution of experiment for two categories of users (novice, advanced). Further, for the advanced user case FIESTA-IoT provides 2 solutions: one based on APIs of “Experiment execution Engine” (EEE) and another based on directly accessing IoT-Registry APIs. Novice experimenters are advised to use the method described in Section "Run/Execute an experiment (Novice Experimenter)".

As said experiment execution is handled by a component called “Experiment execution Engine” or EEE. This module uses and supports the experiment description written by an experimenter in the domain specific language (DSL) format specified by FIESTA-IoT (for reference on the DSL refer to Section 1.1 "Experiment DSL (FEDSpec)"). Amongst the available features in the DSL, in the current version, EEE supports only a few.  These include starting an experiment service (FISMO), pausing a FISMO, restarting a FISMO, subscribing to already existing and discoverable FISMOs, unsubscribing from subscribed FISMOs, and polling a FISMO (executing one time a FISMO on the Fiesta−IoT Platform).  The EEE specific APIs are available here for developers or experimenters for testing and more in-depth knowledge about specific APIs. Note that in case an experimenter wants to use the EEE API they should still upload the FEDSpec either using the ERM API (Section 0) or the ERM Client (Section 1.4). Nevertheless, experimenters can also use Experiment Management Console and perform actions on the FISMO. This option is to be used by novice experimenters.



Run/Execute an experiment (Novice Experimenter)

In order to execute an experiment using Experiment Management Console that is described by its FISMOs, the Experimenter first need to go to: https://platform.fiesta-iot.eu/experimentConsole/experimentConsole.jsp.

You can also use the cookie version of the console by just using the link above. Upon successful authentication list of experiments associated with the experimenter or the user is retrieved as shown in following figure. Note that this is also available via portal. The experimenter needs to go to the Experimenter Menu and click the “Experiment Management Console”.





From this view, experimenters can then select whichever experiment they want to work on from the list using the “SELECT” button next to each experiment.  Once a particular experiment is selected, this would open another UI  (as shown in 2nd Figure to 4th Figure, the entire UI is divided into 3 panes: Experiment Details, Associated FISMOs, and Subscription Pane) where experiment name, experiment description, a list of experiment Domain of Interest along with Associated FISMOs and available FISMOs for subscription is shown.

An eexperimenter can choose to update the metadata of the experiment that he/she has created using “EXPERIMENT EDITOR”. This will open the UI provided in Section Define an experiment through Domain Specification Language (DSL) where experimenter can resubmit their updated Experiments. Upon these resubmissions, a service in EEE is triggered that changes only the scheduling interval. If the scheduling interval is not changed nothing is updated on the EEE.

The “Associated FISMOs” pane shows the “meta” information about the FISMOs that are associated to a particular experiment. The “meta” information includes: if the FISMO was Owned or Subscribed within the frame of an experiment, the status of the FISMO (either NOT YET SCHEDULLED, NORMAL, PAUSED, etc.), past execution history and polling option. Once scheduled a “delete job” button will appear that will let experimenters delete any reference of a particular FISMO from EEE. Upon deleting the FISMO will not be executed any more. In order to check for the description of the FISMO, experimenter can click on the name. This will open a snackbar in the bottom of the page and will show the description of the selected FISMO.

Initially, all the FISMOs have the NOT YET SCHEDULLED status. If the experimenter wants to start the FISMO, they can switch the toggle button. Upon first toggle, the FISMO will be scheduled by the EEE with the NORMAL status. Another toggle would PAUSE the FISMO execution. Yet another toggle would restart the PAUSED FISMO. In order to successfully schedule the FISMO execution, the current version of the EEE supports all that is specified in Section "Define an experiment through Domain Specification Language (DSL)".

A sample of <fed:scheduling> is provided below:

<

fed:scheduling>

       <fed:startTime>2016-11-08T18:13:51.0Z</fed:startTime>

      <fed:Periodicity>600</fed:Periodicity>

      <fed:stopTime>2017-11-08T18:13:50.0Z</fed:stopTime>

</fed:scheduling>



The <fed:scheduling> would provide the EEE with the start date, end date and the periodicity of the FISMO execution. Thus making these attributes essential in the FISMO description. Once the schedule is set in the EEE, EEE provides a JOB ID that is used for internal purposes. This JOB ID is then provided with the status NORMAL. Upon the schedule, the <query> is read by the EEE from the FISMO description and is sent to FIESTA-IoT Meta-Cloud.

The Meta-Cloud executes the query and sends back the results to the EEE. The EEE stores the result internally and pings the location specified in the location specified by the <fed:experimentOutput> (<fed:experimentOutput location=“location”/>).  Upon success, the results are sent to the specified location and deleted from the internal repository. Currently, EEE assumes that the “location” here is a URL, where the specified credentials are granted to the EEE to write the results in a file. For reference and ease, a sample code that experimenters can execute on their server can be found in the following public repository.

It is thus noteworthy to state that currently EEE only supports one mechanism right now to send the information to the experimenter. Given the above, it is thus essential to specify <fed:scheduling> <experimentControl> attribute of FISMO, <query> under <prt:query-request> under <fed:queryControl> and <fed:experimentOutput  location=“location”>. 

If If the experimenter wants to just execute the FISMO and not wait for the EEE to trigger the execution of the FISMO, the experimenter can use POLL NOW. The POLL NOW will execute the <query> defined within the FISMO and would return the results to the same URL that is specified (i.e. the URL where results of scheduled execution are being sent).

Nonetheless, apart from the above functionality, an experimenter can also subscribe to an already existing FISMO (If there were no FISMOs available for subscription this part would be disabled). In case there are many FISMOs available, an experimenter can choose a particular FISMO from the dropdown list and provide the URL information (see 4th Figure). Note that as EEE only support URL, experimenters must specify a valid endpoint. Only after validating the experimenter’s URL the “SUBSCRIBE” button will be unlocked. The experimenter can currently only choose one FISMO at a time.

Once successfully subscribed, the list of Associated FISMOs is updated to show the subscriptions. Each new subscription would provide a new JOB ID where the status of the JOB would be NORMAL to the subscribed FISMO and its execution would begin as the schedule specified in the description of that particular subscribed FISMO (see 5th Figure).

Moreover, the URL specified in the FISMO will not be used. Instead, the URL specified by the subscriber would be used to forward the results. An experimenter, if wishes, can unsubscribe the subscribed FISMO by clicking “UNSUBSCRIBE”. This will delete the JOB associated from the EEE. Note that in the 5th Figure, the Name of 2 FISMOs is same. This can occur due to the fact that another Experimenter has named one of his FISMO exactly the same.

An experimenter is also given a capability to see the details of past executions of the “Associated FISMOs”. The details are provides in the form of a graph and contains information like how much time did it take to execute the FISMO and how much data was obtained from the Meta-Cloud. This graph however not show how much time did it take to execute the FISMO and how much data was obtained from the Meta-Cloud when the FISMO is polled.

In order to delete the experiment it is advised that experimenters first stop/delete the execution of any related FISMO objects on the EEE using the Experiment Management Console. Once this is done, they are advised to remove the experiment from the Experiment Registry Client. We acknowledge this workflow because this will give experimenters a view of what all services are running and if it is really required to remove them at all.



Run/Execute an experiment (Advanced Experimenter)

This method is an API based method. The workflow to model the execution of the experiment should essentially follow:

  • Use ERM API to Store the experiment
  • Use EEE API to schedule, poll, subscribe or monitor the experiment.
  • To update FEDSpec follow the same process as described above (delete the execution first)

Further, to help experimenters use the dynamic feature, we also provide an API. This API should only be used by advanced experimenters and thus is not available via Experiment Management Console. Advance experimenters will have to use the API to execute their dynamic queries. It is important to note that experimenters can use ERM Client to store the FEDSpec and then use the EEE APIs to execute their experiments. Nevertheless, the API for dynamic queries has a signature as described below:

API: /polling/dynamicPollForReport

Method: POST

Produces: application/json

Header Parameters:

  • fismoID: required, fismoID of the service. It is in string format
  • femoID: required, femoID of the experiment. It is in string format
  • iPlanetDirectoryPro: required, it is the access token. It is in string format

Query Parameters:

  • owner: default value is true, this is to identify if the fismo was subscribed or not
  • geoLatitude: default value is 0. However, experimenters need to set this as a double value but in a string format
  • geoLongitude: default value is 0. However, experimenters need to set this as a double value but in a string format
  • intervalNowToPast: time in millisecond. The format is int. Default value is 0
  • fromTime: it is a long value to represent milliseconds after epoch. Default value is 0
  • toTime: it is a long value to represent milliseconds after epoch. Default value is 0

Body Parameters:

  • This is a JSON object represented as a string. The default value is “{}”. However, experimenters need to set the key value pair depending on the query. A JSON object the experimenters need to set is

{
    "KATInput":{"Method":[""], "Parameters":[""]},
    "otherParameters":{<key>:<value>}
}

Here, KATInput reflects the input needed for the FIESTA-IoT Analytics Toolkit, while otherParameters reflect the dynamic attributes.

Response:

  • 200: A successful message looks like {"response":"Polled successfully", "jobID": <JOBID>}
  • 400: Some of the 400 responses include: {"response":"PersistenceException"}, {"response":"ImplementationException"} and error message in JSON format coming from ERM module.

Nevertheless, advanced experimenters can also create their own EEE like component and directly interact with the IoT-Registry. In this section we refrain ourselves in describing the IoT-Registry APIs. These APIs are described in Section 5.3.



Experiment Result Sending and Storage

Having executed the experiment via EEE, it is also essential that we describe the method for sending data to the experimenters. Currently there are many ways experimenters can get the result they intend. These are:

  • Install Experiment Data receiver and provide URLLOCATION for the /store API in the FEDSpec. The process to install and provided as a readme (See here).
  • In case your URLLOCATION was down for some reason, the EEE stores the un-send resultsets locally. The un-send reseultset can be downloaded by the Experimenter.
  • In case you are advance experimenter and not using EEE, you need to setup infrastructure to receive data from IoT-Registry. Remember, as the execution of the query might take time, you need to set Request Timeout property accordingly to a large number.



Experiment Data receiver

A sample code is provided for experimenters to receive data provided by EEE. This code is tested to run on Tomcat successfully.

To deploy, on the experimenter side following has to be done before the deployment

  • in the web.xml change the location entry for multipart-config with the desired location

<multipart-config>  <location>#LOCATION#</location>  </multipart-config>

  • In the 

src/eu/fiesta_iot/experimentServer/ExperimentServerService.java change the $(LOCATION) to match the location set in the web.xml

File file = new File("${LOCATION}", fileName)

Note that this location is the desired location where you want to store the received files.

  • Make sure that the #LOCATION# has read-write permissions to the Tomcat user and group (under the name and group Tomcat is running).
  • In the Tomcat server change the following line in the conf/content.xml

 <Context>      ...  </Context>

with the following

 <Context allowCasualMultipartParsing="true">      ...  </Context>

  • restart Tomcat server

Once the above is done, do the following

  • cd <PATH TO EXPERIMENTSERVER>
  • mvn clean install
  • cp <PATH TO EXPERIMENTSERVER>/target/ExperimentServer.war <PATH TO TOMCAT WEBAPPS>

 

Your service is running at http(s)://<HOST>:<PORT>/ExperimentServer/store/ and thus your URLLOCATION should be http(s)://<HOST>:<PORT>/ExperimentServer/store/

This will enable you to receive the resultsets that are generated after the execution of your FISMO. As also stated before the name of the file received follows a naming convention. Again it is:

String filename = JOBID.replace(“-”,””)+URLLOCATION.replace(":", "").replace("/", "_")+”_”+LONG_TIMESTAMP;

Note here JOBID is a UUID, URLLOCATION is the location that you provide and LONG_TIMESTAMP is a Time-Stamp in long (milliseconds after epoch).

If the experimenter is using HTTPS, then they should use LetsEncrypt certificate for this API/URL. All other certificates other than those available in default JVM configuration will fail. This is because of the JVM does not have all the certificates installed. However, if the URL is HTTP, it will pass through.



Experiment Result Storage

All resultsets that cannot be sent to the experimenters are stored in the Experiment Result Storage (ERS). ERS stores resultset as is and returns them when service is invoked. Upon a success the particular resultset is deleted from the store.

Experimenters need to use an ERS API to download the needed data. This API has a signature:

API: /experiment-result-store
Method: GET
Accepts: application/json
Produces: application/json

Header Parameters:

  • iPlanetDirectoryPro: required, it is the access token. It is in string format
  • femoID: required, femoID of the experiment. It is in string format
  • jobID: optional, JobID of the FISMO in the EEE

If both FEMOID and JobID are provided, then the corresponding FISMO results are returned.

If only the FEMOID is provided, then all FISMO execution results under that particular FEMO along with its corresponding job IDs are returned.

Response:

  • 200: A successful message contains following template:

{ "femoResults": [

   {   "jobid": "<JOBID>",

       "results": [                

               { "time": "<TIMESTAMP>",

             "result": "<RESULTSET>"

           }

        ]

     },……

]}

A sample looks like  

{ "femoResults": [

{

iot/resource/sc-sics-sp-002-power,http://smart-ics.ee.surrey.ac.uk/fiesta- iot/resource/sc-sics-sp-001-power\n0.0,1.432187701766452e- 14,1.9040324872321435e- 14\n0.00819672131148,10.483904864244515,12.66849135485158\n0.016393442623, 10.037536235817262,11.694920662095793\n0.0245901639344,8.274362461842944,1 0.787303593017295\n0.0327868852459,6.936821514862263,8.95507559616293\n0.0 409836065574,5.322912117131182,6.684190365119811\n0.0491803278689,4.082843 4819081725,4.156983856615794\n0.0573770491803,2.267728032490481,2.53056132 0212919\n0.0655737704918,0.6565507982225585,0.6050609198606894\n0.07377049 18033,1.3997255070742531,1.0799031869904239\n0.0819672131148,1.66570166045 3452,2.954284570154875\n0.0901639344262,1.6320456294790227,2.5005243782136 066\n0.0983606557377,1.5195656197778484,2.7019058846634727\n"
}
]
}
]}

  • 400: Some of the 400 responses include:

{"response":"<APPROPRIATE RESPONSE MESSAGE>"}