Posting Property Updates from a Raspberry PI with Python Using REST

[#1]

Recently a customer from the ThingWorx Academic Program sent in a sample program they were having problems with. They were trying to post data from a Raspberry PI using Python to their ThingWorx server. It turns out that their program did work just fine and was also a great example of posting data from a PI using REST.

Here is how to set up this example.

1. Import the attached "Things_TempAndHumidityThing.xml" entity file.

2. from the PI run 'sudo pip install requests'

3. from the PI run 'sudo pip install logging'

4. from the PI run 'sudo pip install http_client'

5. Create a python file call test.py that contains this example code:

#!/usr/bin/python

import requests

import json

import logging

import sys

# These two lines enable debugging at httplib level (requests->urllib3->http.client)

# You will see the REQUEST, including HEADERS and DATA, and RESPONSE with HEADERS but without DATA.

# The only thing missing will be the response.body which is not logged.

try:

    import http.client as http_client

except ImportError:

    # Python 2

    import httplib as http_client

http_client.HTTPConnection.debuglevel = 1

# You must initialize logging, otherwise you'll not see debug output.

logging.basicConfig()

logging.getLogger().setLevel(logging.DEBUG)

requests_log = logging.getLogger("requests.packages.urllib3")

requests_log.setLevel(logging.DEBUG)

requests_log.propagate = True

#NYP Webserver URL in Thingworx

NYP_Webhost = sys.argv[1]

App_Key = sys.argv[2]

ThingName = 'TempAndHumidityThing'

headers = { 'Content-Type': 'application/json', 'appKey': App_Key }

payload = { 'Prop_Temperature': 45, 'Prop_Humidity': 33 }

response = requests.put(NYP_Webhost + '/Thingworx/Things/' + ThingName + '/Properties/*', headers=headers, json=payload, verify=False)

6. From the command line run, './test.py http://twhome:8080 e9274d87-58aa-4d60-b27f-e67962f3e5c4' except substitute your server and your app key.

7. A successful response should look like:

INFO:requests.packages.urllib3.connectionpool:Starting new HTTP connection (1): twhome

send: 'PUT /Thingworx/Things/TempAndHumidityThing/Properties/* HTTP/1.1\r\nHost: twhome:8080\r\nappKey: e9274d87-58aa-4d60-b27f-e67962f3e5c4\r\nContent-Length: 45\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.8.1\r\nConnection: keep-alive\r\nContent-Type: application/json\r\n\r\n{"Prop_Temperature": 45, "Prop_Humidity": 33}'

reply: 'HTTP/1.1 200 OK\r\n'

header: Server: Apache-Coyote/1.1

header: Set-Cookie: JSESSIONID=E7436D2E6AE81C84EC197D406E7E365A; Path=/Thingworx/; HttpOnly

header: Expires: 0

header: Cache-Control: no-store, no-cache

header: Cache-Control: post-check=0, pre-check=0

header: Pragma: no-cache

header: Content-Type: text/html;charset=UTF-8

header: Transfer-Encoding: chunked

header: Date: Mon, 09 Nov 2015 12:39:24 GMT

DEBUG:requests.packages.urllib3.connectionpool:"PUT /Thingworx/Things/TempAndHumidityThing/Properties/* HTTP/1.1" 200 None

My thanks to the customer who sent in the simple example.

[billrei]

If you are trying to post data via REST you should look into using MQTT. Here is another article I wrote that talks about MQTT and Arduino but it may still be relevant. Delivering Arduino Collected Data to ThingWorx using MQTT

[pnali]

hello,

I have tried the above code for connecting to thingworx.For localhost the url is working fine and data is displayed on thingworx but when tried running python code in my system and push data to other remote IP address, error message is displayed saying 403 forbidden error.

From command line i have typed following command:

python test.py http://remote-ip-address:80 (appkey)

Can you suggest me where I am getting wrong

[billrei]

There are a few possibilities.

1. Is your thingworx server on port 80 or port 8080. The 403 error may be coming from a server other than ThingWorx

2. You many not have an appKey or your appKey may not be present or have the privileges needed on the server you are trying to connect to.

Please attach your console output so I can be more specific.

[pedwards]

Hi Bill,

Thanks for posting your code it's been really useful. I'm not having a similar issue as Preethi in that I was able to get my Raspberry Pi to make restful posts to a local Thingworx server I had running in a VM. I've now created a Thingworx Foundation server running in the cloud and I'm getting a '405' error - not sure what I'm doing wrong, I thin I've got the setup indentitcal but obviously not! Any help would be appreciated, thanks Pete

Console output:

INFO:urllib3.connectionpool:Starting new HTTP connection (1): 52.204.140.39

send: 'PUT //Thingworx/Things/PressureTemperatureThing/Properties/* HTTP/1.1\r\nHost: 52.204.140.39\r\nappKey: <my-app-key>\r\nContent-Length: 35\r\nAccept-Encoding: gzip, deflate\r\nAccept: */*\r\nUser-Agent: python-requests/2.4.3 CPython/2.7.9 Linux/4.4.11-v7+\r\nConnection: keep-alive\r\nContent-Type: application/json\r\n\r\n{"Pressure": 23, "Temperature": 42}'

reply: 'HTTP/1.1 405 Method Not Allowed\r\n'

header: Server: Apache-Coyote/1.1

header: Expires: 0

header: Cache-Control: no-store, no-cache

header: Cache-Control: post-check=0, pre-check=0

header: Pragma: no-cache

header: Content-Type: text/html;charset=UTF-8

header: Transfer-Encoding: chunked

header: Date: Mon, 27 Jun 2016 07:36:35 GMT

DEBUG:urllib3.connectionpool:"PUT //Thingworx/Things/PressureTemperatureThing/Properties/* HTTP/1.1" 405 None

>>>

[billrei]

Is there anything in your application or security logs on your server?

Right now it still looks like your server may be configured to not permit this call.

[pedwards]

Hi Bill,

Thanks for getting back to me. Quite strange - I'm logged in as 'Administrator' but I'm unable to access the logs....

I've checked on my local VM and I can't access them there either - am I missing something here?

Pete

[billrei]

Yes. that is strange.

We need to pull in whoever is managing your system.

Do you know who that is?

[gherr-2]

Hi. I am getting an error at step 6.

[billrei]

It looks like you are trying to connect on thingworx on your local machine. There is an error in your connection URL. Try http://localhost:80/Thingworx . Also try http://localhost:8080/Thingworx as this is the default port used when you first run ThingWorx.

[gherr-2]

Thank you for your help. I replaced localhost with the IP address of my local machine and that worked! Thanks Bill Reichardt​

[sdhawane]

Hi Bill,

Thanks for posting this useful article and code.

I am trying to post data from PI to thingworx(Version7.2) server on VM.

I have followed the above mentioned process and I was succesfull in sending data to local thingworx server with the help of Postman.

But i am facing two issues.

1.(Issue on Rapberry PI)

  I have tried below steps.

  2. from the PI run 'sudo pip install requests' -(Result: Successful)

  3. from the PI run 'sudo pip install logging' -(Result: Successful)

  4. from the PI run 'sudo pip install http_client' --(Result: Failure)

  When I am trying to execute step 4 , I am getting one issue. Please find screenshot (fig 1.1)

Further I tried executing python code on PI, its giving error as Permission denied.

  Please find screenshot (fig 1.2)

2. (Issue while posting data on VM using REST call and Postman)

  When I am trying to push data on VM, I am getting issue as 403 Forbidden.

  And on thingworx log I can see "Error occurred while validating HTTP header: cookie" this error.

  Please find screenshot (fig 1.3)

Please help if I am missing something here

Thanks in advance

Thanks,

Sachin Dhawane.

[ppradhan-2]

Hi Bill ,

       i have tried what you explained in this blog but getting an error

Thanks & regards

        Pravin

[sdhawane]

Try this approach. Add this line to python script, this line will push the data to Thingworx.

r = requests.put("http://localhost:8080/Thingworx/Things/TempAndHumidityThing/Properties/(name_of_the_property&remove_brackets)?method=put&value=45", auth=('your_username', 'your_password'),headers=headers)     

for ex.

r = requests.put("http://localhost:8080/Thingworx/Things/TempAndHumidityThing/Properties/PropTemp?method=put&value=45", auth=('abc', 'xyz'),headers=headers)     

you can also print the response code to verify whether data sent or not. below is the python script to print the status

print "Response code {0} ".format(r.status_code)

Thanks,

Sachin Dhawane.

[ds-21]

Hi sachin Dhawane,

  for permission denied. use  Sudo bash or sudo su (which makes you to execute command in root mode)

regards

Dinesh

[ppradhan-2]

Thanks sachin for reply

      i tried the same but its giving the error code 400, how to solve it? is it bcoz of server problm?

[sdhawane]

Hi,

few hack trics,

lets see whether it works out or not.

1. Try using IP/hostname instead of localhost

2. Make sure Thingworx server is up and running

3. Make sure that there exist such property in Thing as what we said in URl(here in our case "PropTemp" property should present in Thing TempAndHumidityThing)

4. Put headers in python script file as below

headers ={'Content-Type': 'application/xml'}

It will be great if you could share your python script( only if possible).

[ppradhan-2]

import requests

import json

import logging

import sys

# These two lines enable debugging at httplib level (requests->urllib3->http.client)

# You will see the REQUEST, including HEADERS and DATA, and RESPONSE with HEADERS but without DATA.

# The only thing missing will be the response.body which is not logged.

try:

    import http.client as http_client

except ImportError:

    # Python 2

    import httplib as http_client

http_client.HTTPConnection.debuglevel = 1

# You must initialize logging, otherwise you'll not see debug output.

logging.basicConfig()

logging.getLogger().setLevel(logging.DEBUG)

requests_log = logging.getLogger("requests.packages.urllib3")

requests_log.setLevel(logging.DEBUG)

requests_log.propagate = True

#NYP Webserver URL in Thingworx

NYP_Webhost = "wss://Thingworx/WS"

App_Key = "041403e1-1c3d-4849-b133-52b3b1f4a7c5"

ThingName = 'TempAndHumidityThing'

headers = { 'Content-Type': 'application/json', 'appKey': App_Key }

payload = { 'Prop_Temperature': 45, 'Prop_Humidity': 33 }

#response = requests.put('http://localhost:8080/Thingworx/Things/' + ThingName + '/Properties/*', headers=headers, json=payload, verify=False)

r = requests.put("http://localhost:8080/Thingworx/Things/TempAndHumidityThing/Properties/PropTemp?method=put&value=45", auth=('Administrator', 'admin'),headers=headers)

print "Response code {0} ".format(r.status_code)

[ppradhan-2]

I have tried all possibility's/tricks you mentioned before this but take a look at the code it is same as bill's code "test.py"

[sdhawane]

Hi,

I hope you are executing python script on Raspberry pie. And if you are running script on Pie then you must have to provide hostname or IP instead of localhost

e.g

r = requests.put("http://ppradhan.itc.net:8080/Thingworx/Things/TempAndHumidityThing/Properties/PropTemp?method=put&value=45", auth=('Administrator', 'admin'),headers=headers)

And update header also in your file. keep it as below

headers ={'Content-Type': 'application/xml'}

If it worked out then OK, otherwise will need to investigate more.

[Hanandan]

Hi just a quick question, does thingworx have or support pyhton SDK?