The Octave Developer Hub

Welcome to the Octave developer hub. You'll find comprehensive guides and documentation to help you start working with Octave as quickly as possible, as well as support if you get stuck. Let's jump right in!

Get Started

Universal Serial Parser (USP) Guide

There are many machines and sensors which can emit streams of data over a serial interface and also receive control commands over the same interface.

Octave includes a Universal Serial Parser (USP) service that accepts such data from a serial interface on an Octave edge device, and makes it available as Resources. The data can then be interpreted in an Edge Action script to extract valuable information. The service also provides the capability to send commands via this same serial interface.

The service is exposed as any other Octave Service (e.g., Modbus, CANopen , GPIO, etc.). The service receives a configuration from the cloud and exposes Resources to read and write data from the serial link.

🚧

Preview

This feature is in preview and subject to change. It is available in edge firmware 3.0.x and above.

We appreciate any feedback you may have when previewing this feature.

Interfaces Description

The following are the USP Resources exposed in Octave:

  • /usp/config : configuration of the service
  • /usp/value : decoded message according to the configuration
  • /usp/write : message to be sent over the serial interface

Configuration

The USP service is configured by modifying the JSON content in the /usp/config Resource:

{
  "uart": "UART0" | ... | "UARTn",   // String, Mandatory
  "frame_timeout": 0.5,              // Timeout in seconds. Must be set to a value greater than 0. After waiting this amount of time without completing a message, reset the accumulation state and the parser state machine
  "string" : true | false,             // Optional. Default value if false. If true, data format is an ASCII string (UTF-8), else format is byte array.
  
  // Framing of serial data: choose one of the 3 strategies:
  "framing": {
     "delimiter":[[13, 10], [32]],       // Accumulate bytes (in decimal) on the serial port until matching one of the specified separators
    // or
      "fixed": 34,// Accumulate the number of specified bytes
    // or
      "lenfield" : {"start":2, "size":2, "bigendian":true, "offset":0}, // Accumulate bytes based on a length field which define the number of byte to accumulate
  }
}

In addition, a UART must be assigned to the USP service in the /io/config Resource. This is done by setting the USP service to "own" the UART via the "own":"usp" field:

{
   "devs":[
      {
         "conf":[
            {
               "baud":"38400",
               "bits":"8",
               "flow":"N",
               "own":"usp",
               "pair":"N",
               "std":"232",
               "stop":"1",
               "type":"UART1",
               "wire":"2"
            }
         ],
         "type":"serial"
      }
   ]
}

Three strategies are available to adapt to the way messages are framed on the serial link:

  • delimiter: the service looks for one of the matching delimiters defined in the configuration. A delimiter is the decimal representation of 1 to 4 bytes. All the data read on the serial link is accumulated in a buffer, and when the delimiter is found, the service writes the content of the buffer to the variable: /usp/value. The delimiter is included in the data reported. This strategy is convenient for text-based protocols where protocol data units (PDUs) are delimited by carriage return, or NFC reader messages terminating by 0x03. The following rules are in place based on the string mode:

    • If string mode is set to false, the delimiter method allows a maximum of 512 delimiters of 4 bytes (max) to be set at one time.
    • If string mode is set to true, only the first delimiter is taken into account with a maximum of 4 bytes.
  • fixed: the service reads a fixed number of bytes from the serial link as defined in the configuration, and stores the data in a buffer. Once that number of bytes have been read, the service writes the content of the buffer to /usp/value.

  • lenfield: this configuration is used to match protocols providing the length of the frame at a fixed position. The configuration must indicate:

    • the position of the length field in the frame (start)
    • the size of the length field (size)
    • the encoding of the length field (big endian (set to true or false))
    • the offset (offset), which is a value to be added to the read length field, in order to add trailing bytes (e.g., CRC bytes).

Once the number "length field value" + "offset" is reached, the service writes the content of the buffer to the variable /usp/value.

Sending Data

In order to send data, you must write a JSON document to the /usp/write Resource.

The data entry in the JSON document contains the bytes that are to be sent on the serial link.

Example 1 - Basic Data Transmission With Integers:

{
    "data":[1,6,145],       //bytes to send
    "answer_timeout":0.2      //(optional) timeout in seconds for the response frame
}

The data must be formatted as an integer array of up to 8192 elements, where each integer value represents a byte (0 through 255).

The answer_timeout entry is optional. It allows the user to configure a timeout between the end of the sent frame and the start of the next received frame.

Example 2: Transmission of Characters when String Mode is Enabled:

// "string" mode set to true :
{
    "data": "AT+ATI8",      //bytes to send
    "answer_timeout":0.2    //(optional) timeout in seconds for the next received frame
}

In this case, the data must be a UTF-8-encoded string containing up to a maximum of 8192 characters including the delimiter.

📘

Note

When string mode is set to true, and the delimiter setting is used (which should be the common case) , the delimiter will be automatically appended to the data written.

Receiving Data

When data bytes are received they are parsed (or counted) according to the framing strategy.

When a frame is considered complete, it is pushed to the /usp/value Resource.

If the frame_timeout time has elapsed after the first received byte without having completed the frame, the received bytes get pushed in the Resource, along with a timeout error.

Example 1 - Basic Example of Receiving Integer Data

{
    "data": [1,6,145],          //bytes received
    "error": "frame_timeout"    //error information, absent if no error. Possible values are "answer_timeout" or "frame_timeout"
}

Example 2 - Receiving Character Data When String Mode is Enabled

// "string" mode set to true :
{
    "data": "OK",               //bytes received
    "error": "frame_timeout"    //error information, absent if no error. Possible values are "answer_timeout" or "frame_timeout"
}

📘

Note

When string mode is set to true, and the delimiter setting is used (which should be the common case), the delimiter will be automatically removed from the data received.

Tutorial

In this tutorial, we will emulate the asset sending data over USP. For this purpose, we will use a development PC, a USB-to-UART (PC-to-Octave mangOH) connection, and a Python script to generate the serial data.

In this example, we will use the delimiter strategy of extracting frames over USP:

  • the simulated asset will be transmitting "PAYLOADX" as a string, where "X" is the end-of-frame separator
  • we will therefore configure USP in delimiter mode, with "X" as a delimiter

🚧

Preview

This feature is in preview and subject to change. It is available in edge firmware 3.0.x and above.

We appreciate any feedback you may have when previewing this feature.

Preparing Your USB-to-UART Setup

This tutorial uses a mangOH Red connected to your development PC over a USB-to-UART bridge, via a USB-to-serial cable.

You can easily adapt this tutorial for a mangOH Yellow or FX30S (the only difference is that you will map the UART to the IoT Connector or to the main RS-232 port of the FX30S).

The example performs the following:

  • creates a UART connection using Octave's southbound USP
  • pushes data periodically to the USP Resource

In order to set up this test environment, see ORP USB to UART setup.

Follow all the steps prior to launching the ORP Python script. The principles are mostly the same but we will run another dedicated script in this section.

USP Configuration

The Service is configured by writing a JSON document to the /usp/config Resource. Here we will select 'delimiter' and define it as "X"; 88 is the decimal representation of our "X" ascii separator. USP is mapped to UART1 which we will configure to USP in the section below.

{
  "frame_timeout": 0.5,
  "framing": {
    "delimiter": [
      [88]
    ]
  },
  "uart": "UART1"
}

We also need to assign the UART to be owned by the usp Service in the /io/config Resource by setting "own":"usp".

We map "usp" to "UART1", and are using the Raspberry Pi connector on the MangOH Red:

{
   "devs":[
      {
      "conf": [
        {
          "baud": "115200",
          "bits": "8",
          "flow": "N",
          "own": "usp",
          "pair": "N",
          "routing": "RPI",
          "std": "232",
          "stop": "1",
          "type": "UART1",
          "wire": "2"
        }
      ],
      "type": "serial"
    }
   ]
}

Observe Received Data


In order to report the received data to the cloud, we are going to configure an Observation on the /usp/value Resource.

Navigate to the Observations screen, and create a Cloud Stream Observation on that Resource.

Launch the Python Script to Send Data from the Simulated Asset over USP

  1. Copy the following code to a new .py file on your development machine and name it usp_sample.py:
import os

from serial import Serial

import string
from time import sleep


DEV = os.getenv('DEV', '/dev/ttyS0')

# Create serial object and use it to create SbSerial connection
s = Serial(port='COM16')
s.baudrate=115200


# Run Forever
while True:
    try:
       sleep(5)
       s.write(b'PAYLOADX')
       print ("Sent 'PAYLOADX' over UART")
    except KeyboardInterrupt:
        exit(0)
  1. Modify the serial port and baud rate settings in the script to match the USP serial configuration on your Octave-enabled device and save the script:
s = Serial(port='COM16')
s.baudrate=115200
  1. Open a terminal window on your computer and install pySerial:
python -m pip install pyserial
  1. Navigate to the location where you saved the usp_sample.py script and run it:

python ./usp_sample.py

The output should look as follows:

\Python27> python .\usp_sample.py
Sent 'PAYLOADX' over UART
Sent 'PAYLOADX' over UART
Sent 'PAYLOADX' over UART
Sent 'PAYLOADX' over UART
Sent 'PAYLOADX' over UART
Sent 'PAYLOADX' over UART
Sent 'PAYLOADX' over UART
Sent 'PAYLOADX' over UART

View the Received Data

The Cloud Stream Observation you have created reports events in a usp stream. This is what the received data should look like:

{
  "elems": {
    "usp": {
      "data": [
        80, // decimal ascii for P
        65, // decimal ascii for A
        89, // decimal ascii for Y
        76, // decimal ascii for L
        79, // decimal ascii for O
        65, // decimal ascii for A
        68, // decimal ascii for D
        88 // decimal ascii for X
      ]
    }
  }
}

You have now set up a simple asset-to-cloud data collection over USP.

Decode the USP Data Locally and Send it to the Cloud

In order to parse the ASCII locally on the edge side before sending it to the cloud, you can do the following:

  • create (or modify) the Observation on the usp/value Resource and route it to an Edge Action
  • create an Edge Action with the following code; the USP data will be sent as a string to the cloud in the device/:default Stream
function(event) {

    var array = event.value.data;
    var data = array.splice(-1); // remove bytes at the end of the frame (one per delimiter byte)

    var parsed = array.map(function(elem){return String.fromCharCode(elem)}).join('')
    //console.log(parsed);

return {
  "cl://": [{ "usp_data": parsed}]};
}

You can see the parsed data pushed periodically in the device/:default Stream:

"elems": {
    "usp_data": "PAYLOAD"
  }

Updated 15 days ago

Universal Serial Parser (USP) Guide


Suggested Edits are limited on API Reference Pages

You can only suggest edits to Markdown body content, but not to the API spec.