Thingworx and EMS

[wwang-21]

     Hi,everyboby.I have completed the Raspberry Pi Quickstart documentation.

     I'm able to use EMS to send raspberry voltage and other data uploaded to the thingworx platform.   Now I want to read the data on the sensor that placed on the raspberry through use of EMS,and then  will be the sensor data uploaded to the Thingworx platform .

Can you recommend several learning documents or examples?

Thanks!

[supandey]

Hi Wang, reading data from the sensor won't be that complicated it would require you to use the library for that sensor which if you don't have it already you will have to search/Google it and using that library you'll gather the data and push it to ThingWorx. For e.g. i used the REST API together with Python to deliver the payload but you can do the same with EMS, below code collected and pushed the data every 5 seconds from the sensor and then to the ThingWorx.

#!/usr/bin/python
import sys
import time
import json
import logging
import requests
import Adafruit_DHT
# enabling the logging
try:
    import http.client as http_client
except ImportError:
    #Python2
    import httplib as http_client
http_client.HTTPConnection.debuglevel=1
# defining the sensor type and the pin on which the data needs to be collected
sensor = 11
pin = 4
var = 1
#Webserver URL in TW
Webhost = "http://tw713:8080/Thingworx/Things/TempAndHumThing/Properties/*"
App_Key = "<appKey>" 
while var == 1:
 hum, temp = Adafruit_DHT.read_retry(sensor, pin)
 print('Temp={0:0.1f} *C  Hum={1:0.1f}%'.format(temp, hum))
 #ThingName = "TempAndHumThing"
 headers = {'Content-Type': 'application/json', 'appKey': App_Key}
 payload = {'temp': temp, 'hum': hum}
 response = requests.put(Webhost, headers=headers,json=payload, verify=True)
 time.sleep(5) # sleep for 5 seconds

There's another great reference/guide you could use from one of the community Blog post Using the C SDK to Deliver Data to ThingWorx from a Raspberry PI

Hope this would help.

[wwang-21]

Hi,Sushant

    Thanks for your help!

    I want to send the data of the read sensor to the thingworx platform via EMS this connection.Can I use lua to read sensor data? Now I have speed sensors, ultrasonic sensors and so on. I want to measure the displacement of the ultrasonic sensor through the EMS upload to the platform for observation.What language is used to read the data on the sensor is better? And then in the Template folder under the Template.lua set properties and services.

[wwang-21]

Hi,Sushant,

     About Edge MicroServer(EMS)&LuaScriptResource(LUA) have more information or helpful examples for?

With your best regards!

[supandey]

Wang, reading through your responses i feel you are looking for more examples on how lua can read your sensor data directly, is that correct? BTW just to put it out here so you are aware that LSR is optional with EMS, you can use EMS standalone too. I'm not aware if we have any examples on how to read the data from your specific sensor using Lua. Have you already tried searching for the available libraries for your sensor that can be easily integrated with lua?

Could you share the exact details of your sensor like the model, type etc.. which you are hooking up to Arduino board?

[wwang-21]

Hi,Sushant,

   Just as you say, I have HC-SR04 UItrasonic senor .I want to send the data to thingworx Platform through Edge MicroServer(EMS)&LuaScriptResource(LUA). My EMS version is 5.3.2.

With your best regards!

[supandey]

Would this GitHub - sza2/node_hcsr04: Distance measurement example for ESP8266/NodeMCU using HC-SR04 ultrasonic sensor help? I believe with this you can get going with reading data within your existing LSR setup together with EMS

[dbujoreanu]

Hi Sushant, Wang,

The example Sushant shared seems close to connecting directly from Lua to a development board, however the board is not exactly Arduino, it's Node MCU which is based on the board that comes as a WiFi module for Arduino - ESP8266. It's not a big difference but it's not the same as a regular Arduino board. Interfacing Arduino boards (Nano, Uno, Mini etc.) with ESP8266 takes a bit of work to make them talk to each other, but again there's plenty of examples on how to do that. So if you don't mind using the ESP8266 and an Arduino board then what Sushant shared is a good way to start.

Regards,
Denis

[dbujoreanu]

Hi Wang,

Using the Sense HAT is not limited or specific to Thingworx, basically once you have your Pi and Sense HAT hooked and configured you could use the Sense HAT from anywhere using the existing APIs to read/write to/from it. The Thingworx tutorial on getting the basic information from the Raspberry Pi (i.e. CPU freq&temp) which you've already mentioned as having done successfully can be expanded to read Sense HAT specific sensor information. The key idea is that you can call from your Pi LUA script, i.e. a PiTemplate.lua script to read basic Pi information, other Python scripts that read sensor data from the Sense HAT and then supply thier results back to set Thingworx thing properties values. You can also send from Thingworx strings to be displayed on the Sense HAT LED matrix by defining a remote service on the Pi thing and call it with the string to be displayed. The most common operation however would be to read the Sense HAT's sensor data such as  position, humidity and pressure. I have used these resources to learn about reading the Sense HAT sensor data:

https://www.raspberrypi.org/learning/getting-started-with-the-sense-hat/

there is also a specific Astro Pi related page you can refer to:

https://astro-pi.org/get-involved/program-the-sense-hat/python/

There is of course a fully fledged API documentation page available for the Sense HAT:

https://pythonhosted.org/sense-hat/

What I found most useful were the examples from the getting started with the sense hat. I started from the existing LUA example.lua template and built a Pi specific one to include services and properties to read Sense HAT sensor data.

Just as an example, my PiTemplate.lua script looks like this:

module ("templates.PiTemplate", thingworx.template.extend)

properties.cpu_temperature  =     { baseType="NUMBER",  pushType="ALWAYS", value=0 }
properties.cpu_freq  =     { baseType="NUMBER",  pushType="ALWAYS", value=0 }
properties.cpu_volt  =     { baseType="NUMBER",  pushType="ALWAYS", value=0 }

properties.temperature = {baseType="NUMBER", pushType="ALWAYS", value=0}
properties.pressure = {baseType="NUMBER", pushType="ALWAYS", value=0}
properties.humidity = {baseType="NUMBER", pushType="ALWAYS", value=0}

properties.yaw = {baseType="NUMBER", pushType="ALWAYS", value=0}
properties.roll = {baseType="NUMBER", pushType="ALWAYS", value=0}
properties.pitch = {baseType="NUMBER", pushType="ALWAYS", value=0}

serviceDefinitions.GetSystemProperties(
output { baseType="BOOLEAN", description="" },
description { "updates properties" }
)

services.GetSystemProperties = function(me, headers, query, data)
log.trace("[PiTemplate]","########### in GetSystemProperties#############")
queryHardware()
queryOrientation()
queryEnvirons()
return 200, true
end

function queryOrientation()

local f = io.popen("python /home/pi/twx/microserver/etc/custom/templates/get_pitch.py")
local s = f:read("*a")
properties.pitch.value = s
f:close()
log.debug("[PiTemplate]", string.format("Pitch: %.2f", properties.pitch.value))

local f = io.popen("python etc/custom/templates/get_roll.py")
local s = f:read("*a")
properties.roll.value = s
f:close()
log.debug("[PiTemplate]", string.format("Roll: %.2f", properties.roll.value))

local f = io.popen("python etc/custom/templates/get_yaw.py")
local s = f:read("*a")
properties.yaw.value = s
f:close()
log.debug("[PiTemplate]", string.format("Yaw: %.2f", properties.yaw.value))

end

function queryEnvirons()

local f = io.popen("python etc/custom/templates/get_humidity.py")
local s = f:read("*a")
properties.humidity.value = s
f:close()
log.debug("[PiTemplate]", string.format("Humidity: %5.2f", properties.humidity.value))

local f = io.popen("python etc/custom/templates/get_pressure.py")
local s = f:read("*a")
properties.pressure.value = s
f:close()
log.debug("[PiTemplate]", string.format("Pressure: %6.1f", properties.pressure.value))

local f = io.popen("python etc/custom/templates/get_temperature.py")
local s = f:read("*a")
properties.temperature.value = s
f:close()
log.debug("[PiTemplate]", string.format("Temperature: %4.1f", properties.temperature.value))

end

function queryHardware()
local tempCmd = io.popen("vcgencmd measure_temp")
local freqCmd = io.popen("vcgencmd measure_clock arm")
local voltCmd = io.popen("vcgencmd measure_volts core")
local s = tempCmd:read("*a")
s = string.match(s,"temp=(%d+\.%d+)");
log.debug("[PiTemplate]",string.format("temp %.1f",s))
properties.cpu_temperature.value = s
s = freqCmd:read("*a")
log.debug("[PiTemplate]",string.format("raw freq %s",s))
s = string.match(s,"frequency%(45%)=(%d+)");
s = s/1000000
log.debug("[PiTemplate]",string.format("scaled freq %d",s))
properties.cpu_freq.value = s
s = voltCmd:read("*a")
log.debug("[PiTemplate]",string.format("raw volts %s", s))
s = string.match(s,"volt=(%d+\.%d+)");
log.debug("[PiTemplate]",string.format("scaled volts %.1f", s))
properties.cpu_volt.value = s
end

tasks.refreshProperties = function(me)
log.trace("[PiTemplate]","~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In tasks.refreshProperties~~~~~~~~~~~~~ ")
queryHardware()
queryOrientation()
queryEnvirons()
end

The python scripts for doing the actual reading look like this example:

def get_pitch():

    from sense_hat import SenseHat

    sense = SenseHat()
    sense.clear()

    o = sense.get_orientation()
    pitch = round(o["pitch"],1)
    return pitch
    
print("{0}". format(get_pitch()))

And they all follow a similar approach, defined in the Astro Pi and the Raspberry Pi Sense Hat web pages mentioned above.

One mention I'd like to make is on how to get location information while using the Pi board. The Sense HAT has no such feature but given that the Raspberry Pi has several USB ports, inserting a USB GPS dongle is pretty straight forward while connecting and reading location information from the dongle is not a trivial task but neither is it very difficult given the amount of resources already available about connecting and reading NMEA data from a USB GPS dongle.

Hope this info helps.

Regards,

Denis

[wwang-21]

Hi,Denis

     Thank you for your helpful assistance.This is a very useful example.

     If you want to read any of the sensors attached to the raspberry pi , and then send it to the Thingworx platform via EMS and LSR. First you need to read the data on the sensor with the python script, and then set the property/services in the Template.lua.  Is that right?

     Is it possible to read the data on the sensor directly with lua ，without a python script?

Yours sincerely! 

[supandey]

Wang, what sort of sensor do you have? I've not done that myself so not sure if you can read it, probably it'll just be about finding the libraries within lua that can do that for you, so that you can directly read from the sensor . Though i feel (may be just my opinion here) python is better equipped with quite a view libraries with useful functionalities. 

[dbujoreanu]

Hi Wang,

As far as I could find, the Sense Hat can be queried using the Python scripts and libraries that come already compiled and packaged for it so if for example you'd attach to the GPIO port some other sensors then you won't necessarily need to use the Python libraries, they may not even exist for your particular sensor type and make, however given that Raspberry Pi does not explicitly prevent you from reading/writing to specific pins on the GPIO using any method, bash scripting or C compiled code for example, I don't see why you should be limited to calling Python from Lua. So far however the preferred approach with Raspberry Pi is to go for the Python scripting and existing libraries.

I used this approach of calling Python scripts from Lua since the Sense Hat already had came with the building blocks I needed to interact with the sensors so it didn't make much sense (no pun intended) to reinvent all the libraries that are already written and work quite well for the Sense HAT just so I don't make an outbound call to a Python script and have to read back its results. It actually helped to separate sensor specific code and Thingworx specific code.

I've also done some similar work with Arduino, if you're familiar with it, and at least in my opinion the Raspberry Pi and its approach to interact with add-ons and other boards is somehow at a higher level, not as low level as you have for Arduino, which mainly uses embedded C libraries for all of the sensors and other things you can connect to it. The Python scripting that is preferred in Raspberry Pi can be a bit of a put-off at first but once you go past Python's quirks it's easy to get along.

One other aspect I see is that  the EMS is biased in the sense that it has to use Lua scripting, the reasons are more or less valid depending on whose point of view you look from, but at least for me, having to call a third party script from Lua and pass parameters then process results is not a major setback. I also wanted to have GPS capabilities for the Pi so I attached a USB GPS dongle to one of my Raspberry Pi USB ports. Since this USB GPS dongle doesn't have any Python scripting already built into the Raspberry Pi libraries I had to devise a way to get the location information from the dongle in some other ways. I ended up using Python as well, not for some specific limitation but since all other scripts were in Python I didn't see a reason why change the approach, so after some digging, I had a script that can be called from Lua and it will return the location as recorded by the GPS dongle. It's not a perfect combination but it's consistent and works quite well.

Coming back to the Lua and Python requirement, this I think is of concern for the current setup, i.e. using Raspberry Pi with Sense HAT, however if you're planning to go onto a much broader scope, having Lua as the center point, "standard" language in which all Thingworx platform related code is done is a good approach -  you only need to worry about how your specific sensor/thing/device can be called from the OS and pass on the results or receive input parameters. Also considering that you may have to integrate devices from different vendors using different modes to interact with them makes a "standard" language for the sensors unfeasible.

Hope this helps and if you need some extra info about connecting the GPS dongle let me know, I can share the code I used.

Regards,

Denis

[wwang-21]

Hi,Denis,

     Thank you for your help.I don't have sense HAT this module and I am using Arduino's development board . Now I want to read the data on the ultrasonic sensor through EMS to the thingworx platform . You also said that EMS can only use lua scripting , I shouldn't use lua scripting to read data?  Can I read it in other languages?

Yours sincerely!

[BryanK]

Hi,

Thanks for this example it was exactly what I was looking for.
Next step would be to send data back to the Sense hat. If I get it right I will add to this.

Regards,

[rsingh-8]

Hi Wang, Aanjan Ravi​,

I am trying Raspberry Pi Quickstart documentation and stuck at Step 10(Run LSR). I am using EMS provided in the guide in beginning steps.

When i am running LSR it starts but it is giving error as

[INFO ] 2017-03-06 17:20:46,870 PiThing: ------------------------------------------------

[INFO ] 2017-03-06 17:20:48,838 PiThing: -- Starting script --------------------------

[INFO ] 2017-03-06 17:20:48,838 PiThing: Registering core callback handler

[INFO ] 2017-03-06 17:20:48,838 PiThing: Starting main loop

[INFO ] 2017-03-06 17:20:48,838 PiThing: Calling lifecycle start listeners.

[INFO ] 2017-03-06 17:20:48,838 shapes.propsubscribe: Initialized

[INFO ] 2017-03-06 17:20:48,839 thingworx.handler: Creating a new handler.

[WARN ] 2017-03-06 17:20:48,943 HttpClient::processReq: Authentication requested but no credentials supplied

[INFO ] 2017-03-06 17:20:48,944 PiThing: Error occured while accessing EMS. Checking isConnected.

[WARN ] 2017-03-06 17:20:49, 46 HttpClient::processReq: Authentication requested but no credentials supplied

[INFO ] 2017-03-06 17:20:49, 46 PiThing: MicroServer is now available.

[INFO ] 2017-03-06 17:20:49, 46 PiThing: EMS is available: true, online: false

config.json

{

"ws_servers": [{

"host": "guragidev01.strykercorp.com",

"port": 8081

}],

"http_server":  {

    "host": "10.117.255.20",

    "port": 8081

},

"appKey": "213abab3-a234-4357-a1f7-49e59d6ac8db",

"logger":   {

    "level":    "DEBUG"

},

"auto_bind":    [{

        "name": "PiThing",

        "gateway":  false

    }],

"certificates": {

    "validate": false,

    "allow_self_signed":    true

},

"ws_connection":    {

    "encryption":   "none",

    "verbose":  true,

    "msg_timeout":  1000

}

}

PiTemplate.lua

module("templates.PiTemplate", thingworx.template.extend)

properties.cpu_temperature = { baseType="NUMBER", pushType="ALWAYS", value=0 }

properties.cpu_freq = { baseType="NUMBER", pushType="ALWAYS", value=0 }

properties.cpu_volt = { baseType="NUMBER", pushType="ALWAYS", value=0 }

serviceDefinitions.GetSystemProperties(

output { baseType="BOOLEAN", description="" },

description { "updates properties" }

)

-- service input parameters

-- me: table that refers to the Thing

-- headers: table of HTTP headers

-- query: query parameters from the HTTP request

-- data: lua table containing the parameters to the service call.

services.GetSystemProperties = function(me, headers, query, data)

log.trace("[PiTemplate]","########### in GetSystemProperties#############")

queryHardware()

--  if properties are successfully updated, return HTTP 200 code with a true service return value

return 200, true

end

function queryHardware()

-- use the vcgencmd shell command to get raspberry pi system values and assign to variables

-- measure_temp returns value in Celsius

-- measure_clock arm returns value in Hertz

-- measure_volts returns balue in Volts

local tempCmd = io.popen("vcgencmd measure_temp")

local freqCmd = io.popen("vcgencmd measure_clock arm")

local voltCmd = io.popen("vcgencmd measure_volts core")

--   set property temperature

local s = tempCmd:read("*a")

s = string.match(s,"temp=(%d+\.%d+)");

log.debug("[PiTemplate]",string.format("temp %.1f",s))

properties.cpu_temperature.value = s

--  set property frequency

s = freqCmd:read("*a")

log.debug("[PiTemplate]",string.format("raw freq %s",s))

s = string.match(s,"frequency%(45%)=(%d+)");

s = s/1000000

log.debug("[PiTemplate]",string.format("scaled freq %d",s))

properties.cpu_freq.value = s

--  set property volts

s = voltCmd:read("*a")

log.debug("[PiTemplate]",string.format("raw volts %s", s))

s = string.match(s,"volt=(%d+\.%d+)");

log.debug("[PiTemplate]",string.format("scaled volts %.1f", s))

properties.cpu_volt.value = s

end

tasks.refreshProperties = function(me)

log.trace("[PiTemplate]","~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ In tasks.refreshProperties~~~~~~~~~~~~~ ")

queryHardware()

end

Config.lua file

scripts.log_level = "INFO"

scripts.PiThing = {

    file = "thing.lua",

    template = "PiTemplate",

    scanrate = 1000,

    taskrate = 3000

}

scripts.rap_host = "guragidev01.strykercorp.com"

scripts.rap_port = 8081

Since it worked for you can you please help what is the issue.

Is my below code ok?  When i tried using 127.0.0.1 is was not even connecting to EMS.

scripts.rap_host = "guragidev01.strykercorp.com"

scripts.rap_port = 8081

What is difference between WS or Http? I am new to Thingworx so excuse me for this basic question.

[rsingh-8]

Its working now. I was giving wrong http details. Thanks!!

[Navku]

Hi Ravindra singh,

I am getting the same error as above.

Can you guide me how to give those details correctly?

Thanks,

Naveen Kumar

[Gowresh_R]

Were you able to solve your problem?

[abarki]

Hi BryanK,

If Gowresh_R response answered your question? kindly mark the reply as the Accepted Solution, this will benefit others in the Community. 

Thank you,

abarki