Showing results for 
Search instead for 
Did you mean: 
Showing results for 
Search instead for 
Did you mean: 

Load Testing through C SDK Remote Device Simulation in ThingWorx

Regular Member

Load Testing through C SDK Remote Device Simulation in ThingWorx


As discussed in the EDC's previous article, load or stress testing a ThingWorx application is very important to the application development process and comes highly recommended by PTC best practices. This article will show how to do stress testing using the ThingWorx C SDK at the Edge side. Attached to this article is a download containing a generic C SDK application and accompanying simulator software written in python. This article will discuss how to unpack everything and move it to the right location on a Linux machine (Ubuntu 16.04 was used in this tutorial and sudo privileges will be necessary). To make this a true test of the Edge software, modify the C SDK code provided or substitute in any custom code used in the Edge devices which connect to the actual application.


It is assumed that ThingWorx is already installed and configured correctly. Anaconda will be downloaded and installed as a part of this tutorial. Note that the simulator only logs at the "error" level on the SDK side, and the data log has been disabled entirely to save resources. For any questions on this tutorial, reach out to the author Desheng Xu from the EDC team (@DeShengXu).



Within ThingWorx, most things represent remote devices located at the Edge. These are pieces of physical equipment which are out in the field and which connect and transmit information to the ThingWorx Platform. Each remote device can have many properties, which can be bound to local properties. In the image below, the example property "Pressure" is bound to the local property "Pressure". The last column indicates whether the property value should be stored in a time series database when the value changes. Only "Pressure" and "TotalFlow" are stored in this way. 


A good stress test will have many properties receiving updates simultaneously, so for this test, more properties will be added. An example shown here has 5 integers, 3 numbers, 2 strings, and 1 sin signal property.




  1. Download Python 3 if it isn't already installed
  2. Download Anaconda version 5.2
    Sometimes managing multiple Python environments is hard on Linux, especially in Ubuntu and when using an Azure VM. Anaconda is a very convenient way to manage it. Some commands which may help to download Anaconda are provided here, but this is not a comprehensive tutorial for Anaconda installation and configuration.
    1. Download Anaconda
      curl -O
    2.  Install Anaconda (this may take 10+ minutes, depending on the hardware and network specifications)
    3. To activate the Anaconda installation, load the new PATH environment variable which was added by the Anaconda installer into the current shell session with the following command:
      source ~/.bashrc​
    4. Create an environment for stress testing. Let's name this environment as "stress"
      conda create -n stress python=3.7​
    5. Activate "stress" environment every time you need to use
      source activate stress​
  3.  Install the required Python modules
    Certain modules are needed in the Python environment in order to run the  file: psutil, requests.
    Use the following commands to install these (if using Anaconda as installed above):
    conda install -n stress -c anaconda psutil 
    conda install -n stress -c anaconda requests​
  4. Unpack the download attached here called
    1. Unzip  and move it into the /opt  folder (if another folder is used, remember to change the page in the simulator.json  file later)
    2. Assign your current user full access to this folder (this command assumes the current user is called ubuntu ) :
      sudo chown -R ubuntu:ubuntu /opt/csim


  5. Move the C SDK source folder to the lib  folder
    1. Use the following command: 
      sudo mv /opt/csim/csdkbuild/ /usr/lib​
    2. You may have to also grant a+x permissions to all files in this folder

  6. Update the configuration file for the simulator
    1. Open /opt/csim/simulator.json  (or whatever path is used instead)
    2. Edit this file to meet your environment needs, based on the information below

  7. Familiarize yourself with the file and its options
    Use the following command to get option information:
    python --help​



Set-Up Test Scenario:

  1. Plan your test
    Each simulator instance will have 8 remote properties by default (as shown in the picture in the Background section). More properties can be added for stress test purposes in the simulator.json  file.

    For the simulator to run 1k writes per second to a time series database, use the following configuration information (note that for this test, a machine with 4 cores and 16G of memory was used. Greater hardware specifications may be required for a larger test):
    1. Forget about the default 8 properties, which have random update patterns and result in difficult results to check later. Instead, create "canary properties" for each thing (where canary refers to the nature of a thing to notify others of danger, in the same way canaries were used in mine shafts)
    2. Add 25 properties for each thing:
      1. 10 integer properties
      2. 5 number properties
      3. 5 string properties
      4. 5 sin properties (signals)
    3. Set the scan rate to 5000 ms, making it so that each of these 25 properties will update every 5 seconds. To get a writes per second rate of 1k, we therefore need 200 devices in total, which is specified by the start and end number lines of the configuration file
    4. The simulator.json  file should look like this:
      Canary_Int: 10 
      Canary_Num: 5 
      Canary_Str: 5 
      Canary_Sin: 5 
      Start_Number: 1 
      End_Number: 200​
  2. Run the simulator
    1. Enter the /opt/csim  folder, and execute the following command:
      python ./simulator.json -i​
    2. You should be able to see a screen like this:
    3. Go to ThingWorx to check if there is a dummy thing (under Remote Things in the Monitoring section):
      This indicates that the simulator is running correctly and connected to ThingWorx

  3. Create a Value Stream and point it at the target database

  4. Create a new thing and call it "SimulatorDummyThing"
    Once this is created successfully and saved, a message should pop up to say that the device was successfully connected

  5. Bind the remote properties to the new thing
    1. Click the "Properties and Alerts" tab
    2. Click "Manage Bindings"
    3. Click "Add all properties"
    4. Click "Done" and then "Save"
    5. The properties should begin updating immediately (every 5 seconds), so click "Refresh" to check

  6. Create a Thing Template from this thing
    1. Click the "More" drop-down and select "Create ThingTemplate"
    2. Give the template a name (ensure it matches what is defined in the simulator.json  file) and save it
    3. Go back and delete the dummy thing created in Step 4, as now we no longer need it
  7. Clean up the simulator
    1. Use the following command:
      python ./simulator.json -k​
    2. Output will look like this:

  8. Create 200 things in ThingWorx for the stress test
    1. Verify the information in the simulator.json  file (especially the start and end numbers) is correct
    2. Use the following command to create all things:
      python ./simulator.json -c​
    3. The output will look like this:
    4. Verify the things have also been created in ThingWorx:
    5. Now you are ready for the stress test


Run Stress Test:

  1. Use the following command to start your test:
    python ./simulator.json -l​


    python ./simulator.json --launch
  2. The output in the simulator will look like this:
  3. Monitor the Value Stream writing status in the Monitoring section of ThingWorx:


Stop and Clean Up:

  1. Use the following command to stop running all instances:
    python ./simulator -k​
  2. If you want to clean up all created dummy things, then use this command:
    python ./simulator -d​
  3. To re-initiate the test at a later date, just repeat the steps in the "Run Stress Test" section above, or re-configure the test by reviewing the steps in the "Set-Up Test Scenario" section


That concludes the tutorial on how to use the C SDK in a stress or load test of a ThingWorx application. Be sure to modify the created Thing Template (created in step 6 of the "Set-Up Test Scenario" section) with any business logic required, for instance events and alerts, to ensure a proper test of the application. 


Re: Load Testing through C SDK Remote Device Simulation in ThingWorx

Great post! Can anyone help me locate the SteamSensor app C source code file used for this load testing activity?

Re: Load Testing through C SDK Remote Device Simulation in ThingWorx

You guys make the cool stuff. I have also seen the PDF with the maximum ingestion rate and there was a high correlation to the "decompile" of the infotabel into different values. Therefore CPU was the bottleneck. Is there also information available what happens and if it is possible to stream up to 150k/sec from 10-15 Kepware servers into InfluxDB servers. My assumption is, that is might be possible with a machine ~2x more CPU like the one used in the benchmark


Re: Load Testing through C SDK Remote Device Simulation in ThingWorx

Thanks for the feedback, Stefan!


One of the next simulations we are designing will be similar to the scenario you are describing - multiple factories connected via KepServer(s) to a cloud-deployed ThingWorx instance.  I'd be happy to connect with you to talk more about the approach we are taking there.


The real balancing act in these high-speed data ingestion scenarios is two things:

  • How quickly you need to ingest data 
  • How much business logic needs to be run against data as it is ingested

User requests will put load on the platform as well, but if you can minimize the "inline data processing" that needs to run against each data point, your data ingestion rates can be much higher.


The Remote Monitoring of Assets Reference Benckmark we conducted included a decent number of expression rules being processed against each data point as they came in.  If you reduced that logic - or switched to a more analytics model where you look for trends and patterns as opposed to individual data points - you would likely be able to support higher data rates on the same infrastructure.

Re: Load Testing through C SDK Remote Device Simulation in ThingWorx



While running the websocket traffic using the Python & C simulator (csim), the server reached its CPU limit when approximately 450 connections were established. The system load averages were vey high. On investigating it further, I saw that the processor run length queue and the system context switches per second were very high.


[root@BLTXLL-IDMA0228 ~]# vmstat -w -S M 10 100
procs -----------------------memory---------------------- ---swap-- -----io---- -system-- --------cpu--------
r b swpd free buff cache si so bi bo in cs us sy id wa st
805 0 0 5039 3 1477 0 0 0 2 3 4 9 11 80 0 0
429 0 0 5038 3 1477 0 0 0 0 70491 629581 42 58 0 0 0
430 0 0 5039 3 1477 0 0 0 0 70458 628943 42 58 0 0 0
427 0 0 5039 3 1477 0 0 0 0 70445 627990 42 58 0 0 0
403 0 0 5039 3 1477 0 0 0 1 70593 630295 43 57 0 0 0
422 0 0 5039 3 1477 0 0 0 1 70501 629272 43 57 0 0 0
429 0 0 5039 3 1477 0 0 0 0 70469 629137 42 58 0 0 0
446 0 0 5041 3 1477 0 0 0 0 70490 628577 42 58 0 0 0
500 0 0 5042 3 1477 0 0 0 0 70544 628094 43 57 0 0 0
453 0 0 5042 3 1477 0 0 0 0 70548 626440 44 56 0 0 0
451 0 0 5040 3 1477 0 0 0 0 70443 629529 43 57 0 0 0
433 0 0 5040 3 1477 0 0 0 0 70535 628144 41 59 0 0 0
443 0 0 5040 3 1477 0 0 0 0 70534 627692 41 59 0 0 0

[root@BLTXLL-IDMA0228 ~]# cat /proc/loadavg
431.23 425.81 422.87 557/5116 105907


It seems like by spawning a large number of processes, the system is unable to cope up with these many running and runnable processes and most of the CPU time is being utilised in context switching.


Server details:

CPU - 4 core 

RAM - 8 GB

OS - RHEL 7.7