Where does API GET request fetches data ?

[Lo_ry96]

After some time being working with Thingworx APIs I would like to know a bit more about their internal functionality.

When i make a get request , how Thingworx handles this ? at which level? if anyone can explain the logics of how this works internally , it would be of a good help 

Many thanks 

[ShirishMorkhade]

Hi @Lo_ry96 ,

I believe if you can share an example of what GET request you're referring to, then me be it will help the audience here to relate and understand your requirement/use case.

Regards,

Shirish

[Lo_ry96]

hello, 

yes sure. I want to fetch data from Thingworx via API requests like this : GET https://<mythingworxplateform.com>/Thingworx/Things/<my_machine>/Properties/

Now, as i'm in corporate environment , my API requests are blocked from the proxy. So i want to know exactly the mechanism of Thingworx API and what access should be given in FW or other so that i can ask IT team to do the needed things so that i can execute my requests normally. 

I hope that things are more clear. 

Best regards, 

[nmutter]

The REST API is just leveraging normal https protocol - in your sample URL you host it on the default https port 443 (as it is not specified). So you would need to allow access for https requests for your instance <mythingworxplatform.com>.

Depending on what you want to do with the API you will do GET, POST, PUT, DELETE requests. Also see related docs to TWX REST API: https://support.ptc.com/help/thingworx/platform/r9.5/en/#page/ThingWorx/Help/REST_API/UpdatingDeletingandExecutingThroughtheAPI.html# 

[VladimirRosu]

Hi @Lo_ry96 ,

I remember I replied to this question in another forum post, but I do not remember where exactly.

In that reply I wrote that the inner workings of the ThingWorx API are not intended to be documented, since they are PTC IP.

Their behavior, however, is documented, so that you know which API to use when you need to (you would need however to have a level of training to understand the APIs, as there is a certain level of complexity to them).

I would still want to make sure I understand your situation.

I saw you wrote that "your API requests are blocked from the proxy". Is it that your ThingWorx inbound API requests (that you originate via browser or another client) are blocked by the proxy? Or is it that the ThingWorx outbound requests are blocked (the requests that ThingWorx might do depending on the API you're calling) ?

In first scenario, you, as an user, should have unlimited access to ThingWorx's API, since without a full access the Composer itself would not work successfully. Another way of saying is that especially for you, as a Developer, there should not be any level of restriction by the proxy.

You mentioned the request "GET .../Things/[machine]/Properties" . That's a request that you should never need to whitelist on its own - meaning you should have a request to the IT to whitelist anything (*) to /Thingworx.

In the second scenario, ThingWorx would need unlimited access to its internal Database server (which can be any of the DB servers supported by ThingWorx for it to work, it's typically either PostgreSQL or MSSQL/Azure SQL). That connection is done via JDBC, which effectively means a pool of TCP connections. That's it for the basics. If you're using InfluxDB you'd need to whitelist the connectivity from ThingWorx to InfluxDB (which is REST on the port that Influx is setup). The same applies for any other system you're connecting to ThingWorx.

Since I'm not sure what direction you're speaking about, I will take the example of the same "GET .../Things/[machine]/Properties". Just in case it's useful: that request does NOT trigger any third party calls (via JDBC or other protocols) to third party systems. But, even if it would trigger, it should not make any difference, because ThingWorx MUST have access to its own DB server to work.

[Lo_ry96]

Hi,

Many thanks on the detailed answer. My use case is the first scenario you described. I want to fetch data from Thingworx and push it to InfluxDB so it's an inbound API request.

As for the proxy , I'm getting following error which made think that proxy is blocking the request: (by the way if you know more about this error , i'm happy to get your feedback about this..) 

ProxyError: HTTPSConnectionPool(host='my_platform_name.com', port=443): Max retries exceeded with url: /Thingworx/Things/Machine_name/Properties/ (Caused by ProxyError('Unable to connect to proxy', OSError('Tunnel connection failed: 504 28; Operation timed out')))

To be more specific, i'm making the GET request from vscode server hosted on Docker (corporate infrastructure that i don't know all of its details).

When i make the same request from my local PC , it works fine!

[VladimirRosu · Accepted solution]

Understood. In this case you must reach out to the internal IT security team in your company and ask them to whitelist the domain name+base url (eg: mythingworxserver/Thingworx/*) for the machine where you're executing those requests (that docker infrastructure).

You can ask them to whitelist "/Thingworx/Things/Machine_name/Properties/" but only if you're 100% sure that's the only request you'll ever make from that machine - up to you to decide.

The error you see can be caused by many things, I believe probably by the fact the VSCode inside that Docker is not able to reach out to the platform. That's 100% IT security job. ( It says timed out, but the issue is that for that endpoint you should have almost instant response back. 504 is gateway timeout error, which means that from proxy's perspective the TW server is not replying).

Note that once this issue is fixed, then maybe we can have a separate discussion on which API to choose to push data to InfluxDB (in this approach you'll loose data if the rate of accessing that endpoint is higher than the rate data is sent to ThingWorx).

[Lo_ry96]

Many thanks for the detailed answer. Yes it is in IT central team hands. I will reach out when this issue solved from my side. 

Thanks again 

Best regards, 

[VladimirRosu]

Sure @Lo_ry96 .

If you have time, when you manage to fix this issue please let us know if you managed to do that successfully (I'm also curious whether this worked and if IT presented any objections).

[Lo_ry96]

Hi , so after many discussions with IT guys , only the HTTP flow is open. Get request with http throws a 504 error. 

Is there a way to ping thingworx with HTTP instead of HTTPS ? 

Many thanks 

[VladimirRosu]

Hi @Lo_ry96 ,

Can you confirm what exactly did they enable? Was it that they enabled HTTP traffic only, regardless of port, or did they enable the capability to only connect to port 80 (regardless of the protocol)?

In any case: if they only accept HTTP protocol, then you need to configure the ThingWorx tomcat server to expose an HTTP port. There's plenty documentation about that on internet....BUT it is very strange however to do this configuration, as from a security best practice doing this exposes ThingWorx to severe security risks - anybody can then intercept your traffic, even if you're in an internal network.

If they're an IT security team, they will know this, so if I would be you, I would very much want to double check with them this setup...very strange (to only allow HTTP requests from that Docker). There must be an element missing here, probably something lost in the communication chain.