IoT & Connectivity Tips

1. Use Postman or any other software for Rest Api call to the ThingWorx. 2. Create a query in Postman with following parameters: Type: POST URL: https://<IP>:<PORT>/Thingworx/Users/<UserName>/Services/AssignNewPassword <IP>: IP of the server where ThingWorx is installed. <PORT>: Port on which ThingWorx is running (if required). <UserName>: User Name of the user whom Password is to be reset. Headers: appkey : Your Administrator App key or App key of user having Permission for AssignNewPassword Service for the user. Content-Type: application/json Body: {     "newPassword":"NewPasswordHere",     "newPasswordConfirm":"NewPasswordHere" } 3. Send the Query. 4. Login using new Password.

This has been moved to its new home in the Augmented Reality Category in the PTC Community.

Timers and schedulers can be useful tool in a Thingworx application.  Their only purpose, of course, is to create events that can be used by the platform to perform and number of tasks.  These can range from, requesting data from an edge device, to doing calculations for alerts, to running archive functions for data.  Sounds like a simple enough process.  Then why do most platform performance issues seem to come from these two simple templates? It all has to do with how the event is subscribed to and how the platform needs to process events and subscriptions.  The tasks of handling MOST events and their related subscription logic is in the EventProcessingSubsystem.  You can see the metrics of this via the Monitoring -> Subsystems menu in Composer.  This will show you how many events have been processed and how many events are waiting in queue to be processed, along with some other settings.  You can often identify issues with Timers and Schedulers here, you will see the number of queued events climb and the number of processed events stagnate. But why!?  Shouldn't this multi-threaded processing take care of all of that.  Most times it can easily do this but when you suddenly flood it with transaction all trying to access the same resources and the same time it can grind to a halt. This typically occurs when you create a timer/scheduler and subscribe to it's event at a template level.  To illustrate this lets look at an example of what might occur.  In this scenario let's imagine we have 1,000 edge devices that we must pull data from.  We only need to get this information every 5 minutes.  When we retrieve it we must lookup some data mapping from a DataTable and store the data in a Stream.  At the 5 minute interval the timer fires it's event.  Suddenly all at once the EventProcessingSubsystem get 1000 events.  This by itself is not a problem, but it will concurrently try to process as many as it can to be efficient.  So we now have multiple transactions all trying to query a single DataTable all at once.  In order to read this table the database (no matter which back end persistence provider) will lock parts or all of the table (depending on the query).  As you can probably guess things begin to slow down because each transaction has the lock while many others are trying to acquire one.  This happens over and over until all 1,000 transactions are complete.  In the mean time we are also doing other commands in the subscription and writing Stream entries to the same database inside the same transactions.  Additionally remember all of these transactions and data they access must be held in memory while they are running.  You also will see a memory spike and depending on resource can run into a problem here as well. Regular events can easily be part of any use case, so how would that work!  The trick to know here comes in two parts.  First, any event a Thing raises can be subscribed to on that same Thing.  When you do this the subscription transaction does not go into the EventProcessingSubsystem.  It will execute on the threads already open in memory for that Thing.  So subscribing to a timer event on the Timer Thing that raised the event will not flood the subsystem. In the previous example, how would you go about polling all of these Things.  Simple, you take the exact logic you would have executed on the template subscription and move it to the timer subscription.  To keep the context of the Thing, use the GetImplimentingThings service for the template to retrieve the list of all 1,000 Things created based on it.  Then loop through these things and execute the logic.  This also means that all of the DataTable queries and logic will be executed sequentially so the database locking issue goes away as well.  Memory issues decrease also because the allocated memory for the quries is either reused or can be clean during garbage collection since the use of the variable that held the result is reallocated on each loop. Overall it is best not to use Timers and Schedulers whenever possible.  Use data triggered events, UI interactions or Rest API calls to initiate transactions whenever possible.  It lowers the overall risk of flooding the system with recourse demands, from processor, to memory, to threads, to database.  Sometimes, though, they are needed.  Follow the basic guides in logic here and things should run smoothly!

Embedded databases come with the installation of the ThingWorx Platform No additional installation or configuration is required for embedded databases Read about the various benefits and pitfalls of embedded versus external below Database Options H2 RDBMS (relational database management system), written in Java Has a small memory footprint Embedded into ThingWorx for easy installation Not as robust as other database options Not scalable in production environments (unless used alongside a separate, external database for stream, value stream, and other data) ​ See KCS Article CS243975 for further reading on the use of external databases Meant to be used for quick deployments and testing environments PostgreSQL ORDBMS (object-relational database management system), written in C PostgreSQL is the ThingWorx recommended database for production systems More Robust External database installed separately from ThingWorx Beneficial because external databases can be specifically configured for use in production, while embedded databases cannot Able to efficiently handle larger amounts of data and store more data without affecting ThingWorx system performance Greater Stability Recover from data corruptions more easily by accessing the database from an external application (separate from ThingWorx) using simple SQL statements Easier to back-up the database in case of issues (further reading in KCS Article CS246598) Less risky and simpler upgrade procedure, which occurs "in-place" Instead of exporting and importing data and entities, a simple schema update allows these to automatically persist into the new version If ThingworxStorage folder is accidentally deleted, entities and data are secure in the external database More Secure HA (High Availability) allows for multiple server instances at different locations in the network Assists in time of failover, i.e. if one server fails, the other can immediately take over Secures the data and prevents further data loss in the event of a failure Customizable security settings and complex password requirements Fewer security vulnerabilities than other databases Because Postgres is an external database, it can be harder to install Follow the steps in the installation guide closely See KCS Articles CS235937 and CS230085 for troubleshooting and help with installation and configuration Hana RDBMS (relational database management system) In-memory, column based data storage For more information on this database, please see the Getting Started with SAP HANA Guide Neo4J GDBMS (graph database management system), written in Java Data is not easily accessed by external applications, and CQL must be used instead of SQL, making recovery from corruptions very difficult Embedded database with limited configuration options Known to have issues with deadlocks Deprecated in version 7.0 (related KCS Article: CS228537) For full installation steps for H2 and PostgreSQL, see the ThingWorx Installation Guide

The ThingWorx Platform is fully exposed using the REST API including every property, service, subsystem, and function.  This means that a remote device can integrate with ThingWorx by sending correctly formatted Hyper Text Transfer Protocol (HTTP) requests. Such an application could alter thing properties, execute services, and more. To help you get started using the REST API for connecting your edge devices to ThingWorx, our ThingWorx developers put together a few resources on the Developer Portal: New to developing with ThingWorx? Use our REST API Quickstart guide that explains how to: create your first Thing, add a property to your Thing, then send and retrieve data. Advanced ThingWorx user? This new REST API how-to series features instructions on how to use REST API for many common tasks, incl. a troubleshooting section. Use ThingWorx frequently but haven’t learned the syntax by heart? We got you covered. The REST API cheat sheet gives details of the most frequently used REST API commands.

Validator widgets provide an easy way to evaluate simple expressions and allow users to see different outcomes in a Mashup. Using a validator is fairly intuitive for simple expressions, such as is my field blank? But if we need to evaluate a more complex scenario based on multiple parameters, then we can user our validator with a backing service that will perform more complex analytics. To show how services can work in conjunction with a validator widget, let’s consider a slightly more complicated scenario such as: A web form needs to validate that the zip or postal code entered by the user is consistent with the country the user selected. Let’s go through the steps of validating our form: Create a service with two input parameters. Our service will use regex to validate a postal code based on the user’s country.  Here’s some sample code we could use on an example Thing: //Input parameters: Country and PostalCode (strings) //Country-based regular expressions: var reCAD = /^[ABCEGHJKLMNPRSTVXY]{1}\d{1}[A-Z]{1} *\d{1}[A-Z]{1}\d{1}$/; var reUS = /^\d{5}(-\d{4})?$/; var reUK = /^[A-Za-z]{1,2}[\d]{1,2}([A-Za-z])?\s?[\d][A-Za-z]{2}$/; var search = ""; //Validate based on Country: if (Country==="CAD")                search = reCAD.exec(PostalCode); else if (Country==="US")                search = reUS.exec(PostalCode); else if (Country==="UK")                search = reUK.exec(PostalCode); (search == null) ? result = false: result = true; Set up a simple mashup to collect the parameters and pass them into our service Add a validator widget Configure the validator widget to parse the output of the service. ServiceInvokeComplete on the service should trigger the evaluation and pass the result of the service into a new parameter on the widget called ServiceResult (Boolean). The expression for the evaluator would then be: ServiceResult? true:false Based on the output of the validator, provide a message confirming the postal code is valid or invalid Add a button to activate the service and the validator evaluation Of course, in addition to providing a message we can also use the results of the validator to activate additional services (such as writing the results of the form to the database). For an example you can import into your ThingWorx environment, please see the attached .zip file which contains a sample mashup and a test thing with a validator service.

Here is a sample to run ConvertJSON just for test 1. Create a DataShape 2. There are 4 input for service ConvertJSON fieldMap (The structure of infotable in the json row) json (json content)   { "rows":[         {             "email":"example1@ptc.com"         },         {             "name":"Lily",             "email":"example2@ptc.com"         }     ] } rowPath (json rows that indicate table information) rows dataShape (infotable dataShape) UserEmail

One of the issues we have encountered recently is the fact that we could not establish a VNC Remote session. The edge was located outside of the internal network where the Tomcat was hosted, and all access to the instance was through an Apache reverse proxy. The EMS was able to connect successfully to the Server, because the Apache had correctly setup the Websocket forwarding through the following directive: ProxyPass "/Thingworx/WS/"  "wss://192.168.0.2/Thingworx/WS" However, we saw that tunnels immediately closed after creation and as a result (or so we thought), we could not connect from the HTML5 VNC viewer. More diagnostics revealed that you need to have ProxyPass directives for the following: -the EMS will make calls to another WS endpoint, called WSTunnelServer. After you setup this, the EMS will be able to create tunnels to the server. -the HTML5 VNC page will make a "websocket" call to yet another WS endpoint, called WSTunnelClient. Only at this step you have the ability to successfully use tunnels through a reverse proxy. Hope it helps!

Hello, Since there have been discussions regarding SNMP capabilities when using the ThingWorx platform, I have made a guide on how you can manage SNMP content with the help of an EMS. Library used: SNMP4J - http://www.snmp4j.org/ Purpose: trapping SNMP events from an edge network, through a JAVA SDK EdgeMicroServer implementation then forwarding the trap information to the ThingWorx server. Background: There are devices, like network elements (routers, switches) that raise SNMP traps in external networks. Usually there are third party systems that collect this information, and then you can query them, but if you want to catch directly the trap, you can use this starter-kit implementation. Attached to this blog post you can find an archive containing the source code files and the entities that you will need for running the example, and a document with information on how to setup and run and the thought process behind the project. Regards, Andrei Valeanu

There are some scenarios where you don't necessarily want to connect to your corporate mail server, or a public mail server like gmail - e.g. when testing a new function that possibly spams the official mail servers - or the mail server is not yet available. In such a scenario it might be a good idea to use a custom, private mail server to be able to send and receive emails locally on a test- or development-environment.   In this post I will show how to use the hMailServer and setup the ThingWorx mail extension to send emails. This post will concentrate on installing and deploying within a Windows environment. More specifically on a Windows 2012 R2 server virtual machine.   Installing hMailServer   Download and install the ​.NET Framework 3.5 (includes .NET 2.0 and 3.0)​. In Windows Server 2012 R2 open the Server Manager and in the Configuration add roles and features.​ Click through the "Role-based or feature-based installation" steps and install the ".NET Framework 3.5 Features" in case they are not already installed.   Download ​hMailServer​ via https://www.hmailserver.com/download As always: the latest version is more stable while the beta versions might provide more functionality and additional bug fixes. This post is based on version 5.6.6-B2383. Functionality and how-to-clicks might change in other versions.   Note: The Microsoft .NET Framework is required for this installation. In case the .NET installation fails by installing it with the hMailServer framework, it's best to cancel the installation and install the required .NET Framework manually instead of the automatic download and installation offered by hMailServer. In case of such a failure it's best to play it safe and uninstall the mail server again, install the .NET framework manually and then re-install hMailServer. (Any left-over directories should be deleted before re-installing)   For the installation, choose your path and a ​full​ installation. Use the built-in database engine​, set a password for the administrative user and install.   Configuring hMailServer   The hMailServer Administrator opens automatically after the installation - if not you will find it in the Start menu. Connect to the default instance on the localhost. The password is the one set up during the installation process.   ​Add a domain​ (e.g. mycompany.com) and ​save​ it. The domain will specify the domain of the mail-addresses e.g. user@domain (me@mycompany.com).   In the domain add an account​. Specify the address (e.g. noreply) and set a password (e.g. ts). ​Save​ the new account.   The default port used for SMTP is 25​. For POP3 it's ​110​. This is configured under Settings > Advanced > TCP/IP ports​ Ensure the ports for SMPT and POP3 are not blocked by a firewall in case you run into issues later on.   This setup should *usually* work. However there might be hostname specific SMTP issues. In case something happens / or to avoid errors in the first place, go to Settings > Protocols > SMTP > Delivery of e-mail​ and specific the ​Local host name​. This should be the fully qualified hostname of the server (e.g. myserver.this.company.com).   Test hMailServer via telnet   Note: telnet needs to be installed for this test - in case it's not installed, Google can help.​​​   Open a command line window and execute: telnet <yourhostname> 25 This will open a connection to the SMTP port of the hMailServer. Manual commands can be send to test if the basic send functions are working. The following structure can be used for testing - it holds manual input and responses.   Username and password need to be Base64 encoded. See https://www.base64encode.org/ for Base64 conversions. (Tip: only text, don't add additional spaces or line breaks - otherwise the hash will be quite different!)   Command / Response Description 220 <HOSTNAME> ESMTP Connected to host HELO mycompany.com Connect with domain as defined in hMailServer 250 Hello. Connected AUTH LOGIN Login as authenticated user 334 VXNlcm5hbWU6 Base64 for "Username:" bm9yZXBseUBteWNvbXBhbnkuY29t Base64 for "noreply@mycompany.com" 334 UGFzc3dvcmQ6 Base64 for "Password:" dHM= Base64 for "ts" 235 authenticated. Authentication successful MAIL FROM: noreply@mycompany.com Sender address 250 OK   RCPT TO: <your real mail address> To address 250 OK   DATA ​Body​ 354 OK, send.   Subject: sending mail via telnet ​Subject ​line   Blank line to indicate end of subject just a simple test! ​Content​ . . indicates the end of mail 250 Queued (10.969 seconds) Mail queued and sent with duration QUIT Log off telnet 221 goodbye   Connection to host lost. Log off confirmed     Configuring ThingWorx   Download and configure the mail extension   Download the MAIL EXTENSION from the ThingWorx Marketplace https://marketplace.thingworx.com/Items/mail-extension   In ThingWorx, click Import / Export > Extensions > Import​, choose the downloaded .zip file and import it. The Composer should be refreshed to reflect the changes introduced by the extension.   The Extension created a new Thing Template: MailServer​   Create a new ​Thing​ based on the MailServer Template​. In its configuration adjust the servername and port to match the hMailServer configuration, e.g. localhost and port 25. Change the Mail User Account and Password to the authentication user (e.g. noreply@mycompany.com / ts). ​Save​ the configuration to persist the changes.   In any Thing, create a new Service to send mails and notifications. Insert a snippet based on Entities > <yourMailThing> > Send Message​ Call the service manually for an initial functional test. It should look similar to this... but parameters need to be adjusted to your environment:   var params = {   cc: undefined /* STRING */,   bcc: undefined /* STRING */,   subject: "sending email via ThingWorx" /* STRING */,   from: "noreply@mycompany.com" /* STRING */,   to: "<your real mail address>" /* STRING */,   body: "just a simple test!" /* HTML */ };   // no return Things["<yourMailThing>"].SendMessage(params);   Check your mailbox for incoming messages!   What next?   The mail server can also be used to receive emails. So instead of sending mails to your regular mail address and risking a ton of spam (depending on your services and frequency of sending automated emails), you could also configure a local Outlook / Thunderbird / etc. installation and send mails directly to the noreply@mycompany.com address. Those mails can then be downloaded from hMailServer via POP3.   With this the whole send AND receive mechanism is contained within a single (virtual) machine.

This has been moved to its new home in the Augmented Reality Category in the PTC Community.

Hi everyone, As everyone knows already, the main way to define Properties inside the EMS Java SDK is to use annotations at the beginning of the VirtualThing class implementation. There are some use-cases when we need to define those properties dynamically, at runtime, like for example when we use a VirtualThing to push a sensor's data from a Device Cloud to the ThingWorx server, for multiple customers. In this case, the number properties differ based on customers, and due to the large number of variations, we need to be able to define programmatically the Properties themselves. The following code will do just that: for (int i = 0; i < int_PropertiesLength; i++) {     Node nNode = device_Properties.item(i);     PropertyDefinition pd;     AspectCollection aspects = new AspectCollection();     if (NumberUtils.isNumber(str_NodeValue))     {         pd = new PropertyDefinition(nNode.getNodeName(), " ", BaseTypes.NUMBER);     }     else if (str_NodeValue=="true"|str_NodeValue=="false")     {         pd = new PropertyDefinition(nNode.getNodeName(), " ", BaseTypes.BOOLEAN);     }     else     pd = new PropertyDefinition(nNode.getNodeName(), " ", BaseTypes.STRING);     aspects.put(Aspects.ASPECT_DATACHANGETYPE,    new StringPrimitive(DataChangeType.VALUE.name()));     //Add the dataChangeThreshold aspect     aspects.put(Aspects.ASPECT_DATACHANGETHRESHOLD, new NumberPrimitive(0.0));     //Add the cacheTime aspect     aspects.put(Aspects.ASPECT_CACHETIME, new IntegerPrimitive(0));     //Add the isPersistent aspect     aspects.put(Aspects.ASPECT_ISPERSISTENT, new BooleanPrimitive(false));     //Add the isReadOnly aspect     aspects.put(Aspects.ASPECT_ISREADONLY, new BooleanPrimitive(true));     //Add the pushType aspect     aspects.put("pushType", new StringPrimitive(DataChangeType.ALWAYS.name()));     aspects.put(Aspects.ASPECT_ISLOGGED,new BooleanPrimitive(true));     //Add the defaultValue aspect if needed...     //aspects.put(Aspects.ASPECT_DEFAULTVALUE, new BooleanPrimitive(true));     pd.setAspects(aspects);     super.defineProperty(pd); }  //you need to comment initializeFromAnnotations() and use instead the initialize() in order for this to work. //super.initializeFromAnnotations();   super.initialize(); Please put this code in the Constructor method of your VirtualThing extending implementation. It needs to be run exactly once, at any instance creation. This method relies on the manual discovery of the sensor properties that you will do before this. Depending on the implementation you can either do the discovery of the properties here in this method (too slow), or you can pass it as a parameter to the constructor (better). Hope it helps!

Concepts of Anomaly Detection used in ThingWatcher ThingWatcher is based on anomaly detection with the normal distribution. What does that mean? Actually,  normally distributed metrics follow a set of probabilistic rules. Upcoming values who follow those rules are recognized as being “normal” or “usual”. Whereas value who break those rules are recognized as being unusual. What is a normal distribution? A normal distribution is a very common probability distribution. In real life, the normal distribution approximates many natural phenomena. A data set is known as “normally distributed” when most of the data aggregate around it's mean, in a symmetric way. Also, it's extreme values get less and less likely to appear. Example When a factory is making 1 kg sugar bags it doesn’t always produce exactly 1 kg. In reality, it is around 1 kg. Most of the time very close to 1 kg and very rarely far from 1 kg. Indeed, the production of 1 kg sugar bag follows a normal distribution. Mathematical rules When a metric appears to be normally distributed it follows some interesting law. As does the sugar bag example. The mean and the median are the same. Both are equal to 1000. It’s because of  the perfectly symmetric “bell-shape” It is the standard deviation called sigma σ that defines how the normal distribution is spread around the mean. In this example σ = 20 68% of all values fall between [mean-σ; mean+σ] For the sugar bag [980; 1020] 95% of all values fall between [mean-2*σ; mean+2*σ] For the sugar bag [960; 1040] 99,7% of all values fall between [mean-3*σ; mean+3*σ] For the sugar bag [940; 1060] The last 3 rules are also known as the 68–95–99.7 rule also called the three-sigma rule of thumb When the rules get broken: it’s an anomaly As previously stated, When a system has been proven normally distributed, it follows a set of rules. Those rules become the model representing the normal behavior of the metric. Under normal conditions, upcoming values will match the normal distribution and the model will be followed. But what happens when the rules get broken? This is when things turn different as something unusual is happening. In theory, in a normal distribution, no values are impossible. If the weights of the bags of sugar were really distributed, we would probably find a bag of sugar of 860 g every billion products. In reality, we approximate this sugar bag example as normally distributed. Also, almost impossible value are approximated as impossible Techniques of Anomaly Detection Technique n°1: outlier value An almost impossible value could be considered as an anomaly. When the value deviates too much from the mean, let’s say by ± 4σ, then we can consider this almost impossible value as an anomaly. (This limit can also be calculated using the percentile). Sugar bags who weigh less than 920 g or more than 1080 g are considered anomalous. Chances are, there is a problem in the production chain. This provides a simple way to define maximum and minimum thresholds. Technique 2: detecting change in the normal distribution Technique n°2 can detect unusual distribution fast, using only some points. But it can’t detect anomalies who move from one sigma σ to another in a usual manner. To detect this kind of anomaly we use a “window” of n last elements. If the mean and standard derivation of this window change too much from usual then we can deduce an anomaly. Using a big window with a lot of values is more stable, but it requires more time to detect the anomaly. The bigger the window is the more stable it becomes. But it would require more time to detect the anomaly as it needs to aggregate more values for the detection.

In this particular scenario, the server is experiencing a severe performance drop.The first step to check first is the overall state of the server -- CPU consumption, memory, disk I/O. Not seeing anything unusual there, the second step is to check the Thingworx condition through the status tool available with the Tomcat manager. Per the observation: Despite 12 GB of memory being allocated, only 1 GB is in use. Large number of threads currently running on the server is experiencing long run times (up to 35 minutes) Checking Tomcat configuration didn't show any errors or potential causes of the problem, thus moving onto the second bullet, the threads need to be analyzed. That thread has been running 200,936 milliseconds -- more than 3 minutes for an operation that should take less than a second. Also, it's noted that there were 93 busy threads running concurrently. Causes: Concurrency on writing session variable values to the server. The threads are kept alive and blocking the system. Tracing the issue back to the piece of code in the service recently included in the application, the problem has been solved by adding an IF condition in order to perform Session variable values update only when needed. In result, the update only happens once a shift. Conclusion: Using Tomcat to view mashup generated threads helped identify the service involved in the root cause. Modification required to resolve was a small code change to reduce the frequency of the session variable update.

This has been moved to its new home in the Augmented Reality Category in the PTC Community.

Hi everybody, In this blogpost I want to share with you my local ThingWorx installation, with some optimizations that I did for local development. -use the -XX:+UseConcMarkSweepGC . This uses the older Garbage Collector from the JVM, instead of the newer G1GC recommended by the ThingWorx Installation guide since version 7.2. The advantage of ConcMarkSweepGC is that the startup time is faster and the total memory footprint of the Tomcat is far lower than G1GC. -use -agentlib:jdwp=transport=dt_socket,address=1049,server=y,suspend=n. This allows using your Java IDE of choice to connect directly to the Tomcat server, then debugging your Extension code, or even the ThingWorx code using the Eclipse Class Decompilers for example. Please modify the 1049 to your port of choice for exposing the server debugging port. -use -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=60000 -Dcom.sun.management.jmxremote.ssl=false                  -Dcom.sun.management.jmxremote.authenticate=false           This sets up the server to allow JMX monitoring. I usually use VisualVM from the JDK bin folder, but you can use any JMX monitoring tool.           This uses no Authentication, no SSL and uses port 6000 - modify if you need. I usually startup Tomcat manually from a folder via startup.bat, and the setenv.bat looks like: set JAVA_HOME=C:\Program Files\Java\jdk1.8.0_102 set JRE_HOME=C:\Program Files\Java\jdk1.8.0_102 set THINGWORX_PLATFORM_SETTINGS=D:\Work\servers\apache-tomcat-8.0.33 // this is where the platform-settings.json file is located set CATALINA_OPTS=-d64 -XX:+UseNUMA -XX:+UseConcMarkSweepGC -Dfile.encoding=UTF-8 -agentlib:jdwp=transport=dt_socket,address=1049,server=y,suspend=n -Dcom.sun.management.jmxremote -Dcom.sun.management.jmxremote.port=60000 -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.authenticate=false In this mode I can look at any errors in almost real time from the console and it makes killing the server for Java Extension reload a breeze -> Ctrl+C Please don't hesitate to provide feedback on this document, I certainly welcome it. Be warned: THESE ARE NOT PRODUCTION SETTINGS. Best regards, Vladimir

This has been moved to its new home in the Augmented Reality Category in the PTC Community.

This is an example for setting up remote desktop and file transfer for an asset in Thingworx Utilities using the Java Edge SDK. Step 1.   EMS Configuration ClientConfigurator config = new ClientConfigurator(); // application key RemoteAccessThingKey String appKey = "s2ad46d04-5907-4182-88c2-0aad284f902c"; config.setAppKey(appKey); // Thingworx server Uri config.setUri("wss://10.128.49.63:8445//Thingworx/WS"); config.ignoreSSLErrors(true); config.setReconnectInterval(15); SecurityClaims claims = SecurityClaims.fromAppKey(appKey); config.setSecurityClaims(claims);      // enable tunnels for the EMS config.tunnelsEnabled(true); // initialize a virtual thing with identifier PTCDemoRemoteAccessThing VirtualThing myThing = new VirtualThing(ThingName, "PTCDemoRemoteAccessThing", "PTCDemoRemoteAccessThing", client); **// for the file transfer functionality FileTransferVirtualThing myThing = new FileTransferVirtualThing(ThingName, "PTCDemoRemoteAccessThing", "PTCDemoRemoteAccessThing", client); myThing.addVirtualDirectory("AssetRepo",  "E:/AssetRepo"); Step 2.   Install TightVNC on the asset ( http://www.tightvnc.com/download.php ) Step 3.   Go to Thingworx Composer, search for thing PTCDemoRemoteAccessThing                 a.   pair  PTCDemoRemoteAccessThing with identifier PTCDemoRemoteAccessThing                 b.   go to PTCDemoRemoteAccessThing Configuration  and add a tunnel name: vnc host: asset IP port: 5900 (this is the default port for VNC servers; it can be changed)                 c.   go to PTCDemoRemoteAccessThing Properties and set the vncPassword Troubleshooting                 If the VNC server is on the same machine with the Thingworx server, check “Allow loopback connections” from Access Control tab in TightVNC Server Configuration.

In this Blog, we will share some light about Gradient boost, which is a default algorithm in our Analytics platform. We will touch on: 1) The main purpose of Gradient boost and how the technique works. 2) We will look at advantages and constraint. 3) Last some “nice to know” tips when working with Gradient. Gradient boost is a machine learning technique which main purpose is to help weak prediction models become stronger. Gradient boost works by building one tree at a time, and correct errors made by previously tree. The theory support reweights of edges which allows badly weight edges to get reweighted. For example the misclassified gain weight and those weights which are classified correctly, lose weight. It is kind of the same strategy when dealing with stocks; you balance the investment between bonds and share. An analog could also be done to illnesses; If a doctor informs that you have a rare disease, you want to make sure to get a few more opinions from other doctors, You will evaluate all the information to make a more correct decision about how to cure yourself. Why use gradient boost: - Gradient boost provides the user with a powerful tool to boost/improve weak prediction models. - Gradient boost works well with regression and classification problems, therefore Decision tree can benefit from applying gradient boost. - Gradient bo​ost is known in the industry, to be one of the best techniques to use when dealing with model improvement. - Gradient boost uses stagewise fashion, in this way each time it adjust a tree, it does not go back and readjust when dealing with the next tree. As with all machine learning algorithms gradient boost also have some constraint: - There is a change of overfitting. “Nice to know” tips: - A natural way to reduce this risk of overfitting would be to monitor and adjust the iterations. - The depth of the tree might have an influence on the prediction error, observe what happens if the depth is a stump/1 level deep.

This Blog presents a simple Java utility to validate the deployment of ThingWatcher. It is important to note that the utility used is not a real life situation, the intent was to keep it as simple as possible in order to achieve its aim: validation of the deployment. An understanding of Java IDE (such as Eclipse) is necessary in order to run the utility with relevant dependency and classpath setup. Those are beyond the scope of this posting. We will cover the following points: Pre-Requisites Using the sample utility Code walk through Validate training job creation Validate model job creation Update for ThingWorx Analytics 8.0 Pre-requisites A strict adherence to the ThingWatcher deployment guide is recommended in order to first deploy training and model microservices as well as to familiarize yourself with ThingWatcher APIs. Prior to testing ThingWatcher, both the training and model microservices should be up and running The media for ThingWatcher (including model and training micro-service) should be downloaded from PTC Software Download page . The commands to deploy the micro-services will vary depending on the platform used and are presented in the ThingWatcher deployment guide. As a reference example, on Windows the command will be similar to the following: Start Docker: Start > Program > Docker > Docker Quick Start Terminal Load model micro service tar $ docker load < "D:\PTC\MED-61147-CD-522_F000_ThingWorx-Analytics-ThingWatcher-52-2\components\ModelService\ModelService\model-service.tar"     3. Install model service: $ docker run -d -p 8080:8080 -v '/d/TWatcherStorage/model:/data/models' -v '/d/TWatcherStorage/db:/tmp/' twxml/model-service:1.0 -Dfile.storage.path=/data/models -jar maven/model-1.0.jar server maven/standalone-evaluator.yml     4. Load training micro service tar file                         $ docker load < "D:\PTC\MED-61147-CD-522_F000_ThingWorx-Analytics-ThingWatcher-52-2\components\TrainingService\TrainingService\training-service.tar"     5. Install training service                         $ docker run -d -p 8090:8080  twxml/training-service:1.0.0  -Dmodel.destination.uri=model://192.168.99.100:8080/models -jar maven/training-standalone-1.0.0-bin.jar server /maven/training-standalone-single.yml Note: the -Dmodel.destination.uri points here to the model micro-service host. To find the ip address, enter docker-machine ip on the model micro-service docker machine.     6. Validate micro-services deployment: Execute docker ps  and confirmed that both services are up, as in the following example: CONTAINER ID        IMAGE                          COMMAND                      CREATED            STATUS              PORTS NAMES 5b6a29b95611        twxml/training-service:1.0.0  "java -Dmodel.destina"  13 days ago        Up 44 minutes      8081/tcp, 0.0.0.0:8090->8080/tcp  modest_albattani 8c13c0bc910e        twxml/model-service:1.0        "java -Dfile.storage."      2 weeks ago        Up 44 minutes      0.0.0.0:8080->8080/tcp, 8081/tcp  thirsty_ptolemy   Using the sample utility Download the attachment Main.java Import Main.java into Eclipse (or IDE of choice) with the ThingWatcher dependencies added in classpath. Update the trainingBaseURI (see below) to points to the training micro-services. The utility should be ready to execute. Code walk through The code declares a thingwatcher in the following snippet: ThingWatcher thingwatcher = new ThingWatcherBuilder() .certainty(90.0) .trainingDataDuration(60) .trainingDataDurationUnit(DurationUnit.SECOND) .trainingBaseURI("http://192.168.99.100:8090/training") .getThingWatcher(); In the above code it is important to update the trainingBaseURI argument with the correct ip address and port for the training micro-service host. The code then loops 10000 times and sends a new value, which simulates the sensor data, at a simulated 100 ms interval. The value is computed as Math.sin(i) for the whole calibrating phase and most of the monitoring phase too. We artificially introduce an anomaly by sending a value of Math.incremetExact(i) between the 9000 th and 9900 th iterations. During the Monitoring phase, the code logs the value, the anomalous status and the thingwatcher state. It is advised to save the output to a file in order to review the logging once the utility has run. In Eclipse this can be done by selecting the Main.java with right mouse button > Run As… > Run Configuration > Common and tick Output File under the Standard Input and Output, and specify a location for the output file. A review of the output log file will shows that somewhere between timestamp 900000 and 990000, the isAnomalousValue is true. Note that this does not starts and ends exactly at 900000 and 990000, as ThingWatcher needs a few occurrences before reporting it as anomaly. Sample output indicating an anomalous state: [main] INFO com.thingworx.analytics.demo.Main - Value = 901700,9017.0,-9016.403802019577 [main] INFO com.thingworx.analytics.demo.Main - isAnomalousValue = true [main] INFO com.thingworx.analytics.demo.Main - ThingWatcherStat = MONITORING As part of validating the successful deployment of ThingWatcher, it is recommended to validate the correct creation of a training and model job. Validate training job creation In order to validate the successful creation of a training job, execute a GET request to the training micro service : http://192.168.99.100:8090/training (update the ip address to the one on your system) This should return a COMPLETED job whose body starts with something similar to: Validate model job creation In order to validate the successful creation of a model job, execute a GET request to http://192.168.99.100:8080/models (update the ip address to the one on your system) to see all the models that have been created. For example: Alternatively, click (or use) the URI reported in the training job output, here http://192.168.99.100:8080/models/6/pmml.xml, to see the complete model definition. The output will be similar to: When this sample test runs correctly, the ThingWatcher deployment has been validated. Update for ThingWorx Analytics 8.0 Deploying the microservices, see Video Link : 1937 Updated Java code: see Does anyone know how to use java api to achieve anomaly detection with Thingwatcher8.0? To Note: The utility provided is for testing purpose only. The code does not represent any kind of best practice and is not meant to be a perfect java coding example. It is provided as is with no guarantee.

First of all wishing everyone a blessed 2017 So here is a little something that hopefully can be helpful for all you Thingworx developers! This is a 'Remote Monitoring Application Starter' Mainly this is created around Best Practices for Security and provides a lot of powerful Modeling and Mashup techniques. Also has some cool Dashboard techniques Everything is documented in accompanying documents also in the zip (sorry went through a few steps to get this up properly. Install instructions: Thingworx Remote Monitoring Starter Application – Installation Guide Files All files needed are in a Folder called: RemoteMonitoringStarter, this is an Export to ThingworxStorage Extensions Not included, but the application uses the GoogleWidgetsExtension (Google Map) Steps Import Google Map extension. Place RemoteMonitoringStarter folder in the ThingworxStorage exports folder. From Thingworx do an Import from ThingworxStorage – Include Data, Use Default Persistence Provider, do NOT ignore Subsystems. After the import has finished, go to Organizations and open Everyone. In the Organization remove Users from the Everyone organization unit. Go to DataTables and open PTC.RemoteMonitoring.Simulation.DT Go to Services and execute SetSimulationValues Go to the UserManagementSubsystem In the Configuration section add PTC.RemoteMonitoring.Session.TS to the Session. Note: This step may already be done. Note: Screen shots provided at the end. Account Passwords FullAdmin/FullAdmin All other users have a password of: password. NOTE: You may have to Reset your Administrator password using the FullAdmin account. I also recommend changing the passwords after installing.