Load Testing through C SDK Remote Device Simulation in ThingWorx

Load Testing through C SDK

Remote Device Simulation in ThingWorx

As discussed in the EDC's previous article, load or stress testing a ThingWorx application is very important to the application development process and comes highly recommended by PTC best practices. This article will show how to do stress testing using the ThingWorx C SDK at the Edge side. Attached to this article is a download containing a generic C SDK application and accompanying simulator software written in python. This article will discuss how to unpack everything and move it to the right location on a Linux machine (Ubuntu 16.04 was used in this tutorial and sudo privileges will be necessary). To make this a true test of the Edge software, modify the C SDK code provided or substitute in any custom code used in the Edge devices which connect to the actual application.

It is assumed that ThingWorx is already installed and configured correctly. Anaconda will be downloaded and installed as a part of this tutorial. Note that the simulator only logs at the "error" level on the SDK side, and the data log has been disabled entirely to save resources. For any questions on this tutorial, reach out to the author Desheng Xu from the EDC team (@DeShengXu).

Background:

Within ThingWorx, most things represent remote devices located at the Edge. These are pieces of physical equipment which are out in the field and which connect and transmit information to the ThingWorx Platform. Each remote device can have many properties, which can be bound to local properties. In the image below, the example property "Pressure" is bound to the local property "Pressure". The last column indicates whether the property value should be stored in a time series database when the value changes. Only "Pressure" and "TotalFlow" are stored in this way. 

A good stress test will have many properties receiving updates simultaneously, so for this test, more properties will be added. An example shown here has 5 integers, 3 numbers, 2 strings, and 1 sin signal property.

Installation:

1. Download Python 3 if it isn't already installed
   
2. Download Anaconda version 5.2
   Sometimes managing multiple Python environments is hard on Linux, especially in Ubuntu and when using an Azure VM. Anaconda is a very convenient way to manage it. Some commands which may help to download Anaconda are provided here, but this is not a comprehensive tutorial for Anaconda installation and configuration.
   1. Download Anaconda
      

      curl -O https://repo.anaconda.com/archive/Anaconda3-5.2.0-Linux-x86_64.sh

   2.  Install Anaconda (this may take 10+ minutes, depending on the hardware and network specifications)
      

      bash Anaconda3-5.2.0-Linux-x86_64.sh​

   3. To activate the Anaconda installation, load the new PATH environment variable which was added by the Anaconda installer into the current shell session with the following command:

      source ~/.bashrc​

   4. Create an environment for stress testing. Let's name this environment as "stress"
      

      conda create -n stress python=3.7​

   5. Activate "stress" environment every time you need to use simulator.py
      

      source activate stress​

3.  Install the required Python modules
   Certain modules are needed in the Python environment in order to run the simulator.py  file: psutil, requests.
   Use the following commands to install these (if using Anaconda as installed above):
   

   conda install -n stress -c anaconda psutil 
   conda install -n stress -c anaconda requests​

4. Unpack the download attached here called csim.zip
   
   1. Unzip csim.zip  and move it into the /opt  folder (if another folder is used, remember to change the page in the simulator.json  file later)
      
   2. Assign your current user full access to this folder (this command assumes the current user is called ubuntu ) :

      sudo chown -R ubuntu:ubuntu /opt/csim

       

5. Move the C SDK source folder to the lib  folder
   1. Use the following command: 

      sudo mv /opt/csim/csdkbuild/libtwCSdk.so.2.2.4 /usr/lib​

   2. You may have to also grant a+x permissions to all files in this folder
      
      
6. Update the configuration file for the simulator
   1. Open /opt/csim/simulator.json  (or whatever path is used instead)
      
   2. Edit this file to meet your environment needs, based on the information below
      
      
      
7. Familiarize yourself with the simulator.py file and its options
   Use the following command to get option information:

   python simulator.py --help​

   

Set-Up Test Scenario:

1. Plan your test
   Each simulator instance will have 8 remote properties by default (as shown in the picture in the Background section). More properties can be added for stress test purposes in the simulator.json  file.
   
   For the simulator to run 1k writes per second to a time series database, use the following configuration information (note that for this test, a machine with 4 cores and 16G of memory was used. Greater hardware specifications may be required for a larger test):
   1. Forget about the default 8 properties, which have random update patterns and result in difficult results to check later. Instead, create "canary properties" for each thing (where canary refers to the nature of a thing to notify others of danger, in the same way canaries were used in mine shafts)
   2. Add 25 properties for each thing:
      1. 10 integer properties
      2. 5 number properties
      3. 5 string properties
      4. 5 sin properties (signals)
   3. Set the scan rate to 5000 ms, making it so that each of these 25 properties will update every 5 seconds. To get a writes per second rate of 1k, we therefore need 200 devices in total, which is specified by the start and end number lines of the configuration file
   4. The simulator.json  file should look like this:

      Canary_Int: 10 
      Canary_Num: 5 
      Canary_Str: 5 
      Canary_Sin: 5 
      Start_Number: 1 
      End_Number: 200​

2. Run the simulator
   1. Enter the /opt/csim  folder, and execute the following command:

      python simulator.py ./simulator.json -i​

   2. You should be able to see a screen like this:
      
   3. Go to ThingWorx to check if there is a dummy thing (under Remote Things in the Monitoring section):
      
      This indicates that the simulator is running correctly and connected to ThingWorx
      
      
3. Create a Value Stream and point it at the target database
   
   
   
4. Create a new thing and call it "SimulatorDummyThing"
   
   Once this is created successfully and saved, a message should pop up to say that the device was successfully connected
   
   
5. Bind the remote properties to the new thing
   1. Click the "Properties and Alerts" tab
      
   2. Click "Manage Bindings"
      
   3. Click "Add all properties"
      
   4. Click "Done" and then "Save"
   5. The properties should begin updating immediately (every 5 seconds), so click "Refresh" to check
      
      
      
6. Create a Thing Template from this thing
   1. Click the "More" drop-down and select "Create ThingTemplate"
      
   2. Give the template a name (ensure it matches what is defined in the simulator.json  file) and save it
      
   3. Go back and delete the dummy thing created in Step 4, as now we no longer need it
      
7. Clean up the simulator
   1. Use the following command:

      python simulator.py ./simulator.json -k​

   2. Output will look like this:
      
      
      
8. Create 200 things in ThingWorx for the stress test
   1. Verify the information in the simulator.json  file (especially the start and end numbers) is correct
      
   2. Use the following command to create all things:

      python simulator.py ./simulator.json -c​

   3. The output will look like this:
      
   4. Verify the things have also been created in ThingWorx:
      
   5. Now you are ready for the stress test

Run Stress Test:

1. Use the following command to start your test:

   python simulator.py ./simulator.json -l​

   or

   python simulator.py ./simulator.json --launch

2. The output in the simulator will look like this:
   
3. Monitor the Value Stream writing status in the Monitoring section of ThingWorx:
   

Stop and Clean Up:

1. Use the following command to stop running all instances:

   python simulator.py ./simulator -k​

2. If you want to clean up all created dummy things, then use this command:

   python simulator.py ./simulator -d​

3. To re-initiate the test at a later date, just repeat the steps in the "Run Stress Test" section above, or re-configure the test by reviewing the steps in the "Set-Up Test Scenario" section

That concludes the tutorial on how to use the C SDK in a stress or load test of a ThingWorx application. Be sure to modify the created Thing Template (created in step 6 of the "Set-Up Test Scenario" section) with any business logic required, for instance events and alerts, to ensure a proper test of the application.