cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Community Tip - Did you know you can set a signature that will be added to all your posts? Set it here! X

C SDK Character Encoding

No ratings

When I tried to set String property values with Chinese letters by using C SDK, I could see only broken characters in Thingworx. The cause of the problem is simple and it's encoding. In order to solve this problem, you need to convert encoding to UTF-8 in your C code. I used 'libiconv' library to do it. This guide is for Win32 version. If you want to make a library for other platforms such as Linux, you can create or get a library for your own by googling.

How to Get the Source Code of libiconv

At the moment, the most recent version of libiconv is 1.15. You can download the source code of libiconv from here.


How to Build

I used MS Visual Studio 2012, but the explanation can be applied to the earlier versions of MS Visual Studio and express editions.

Step 1. Download the most recent version of libiconv and unzip the file.

Step 2. Make a new Win32 Project.

Let's say "libiconv" as the project name. Check to create directory for solution.

Choose DLL as the application type and check Empty Project for additional options. Click the button "Finish" to generate the new project.

Step 3. Copy files from the folders of libiconv to project folders.

To build "libiconv", you need to compile three files "localcharset.c", "relocatable.c" and "iconv.c". That's the key!

  1. Copy three files "relocatable.h", "relocatable.c" and "iconv.c" in the folder "...\libiconv-1.15\lib\" to the project folder "...\libiconv\libiconv\".
  2. Copy "...\libiconv-1.15\libcharset\lib\localcharset.c" to the project folder "...\libiconv\libiconv\".
  3. Copy "...\libiconv-1.15\libcharset\include\localcharset.h.build.in" to the project folder "...\libiconv\libiconv\" and then, rename the copied "localcharset.h.build.in" to "localcharset.h".
  4. Copy "...\libiconv-1.15\windows\libiconv.rc" to the project folder "...\libiconv\libiconv\".
  5. Make folder "include" under the project folder "...\libiconv\".
  6. Copy "...\libiconv-1.15\include\iconv.h.build.in" to the project include folder "...\libiconv\include" and then, rename the copied "iconv.h.build.in" to "iconv.h".
  7. Copy "...\libiconv-1.15\config.h.in" to the project include folder "...\libiconv\include" and then, rename the copied "config.h.in" to "config.h".
  8. Copy all the header files (*.h) and definition files (*.def) in the folder "...\libiconv-1.15\lib" to the project include folder "...\libiconv\include".

Step 4. Add existing items.

Execute "project > Add Existing items..." at the main menu to add existing items to the project.

Step 5. Project Settings.

You can make 64-bit platform through configuration manager in order to generate libiconv.dll for 64-bit system. You can also make two other configurations "ReleaseStatic" and "DebugStatic" in order to generate libiconvStatic.lib as a static link library.

At the project properties, change Output Directory as "$(SolutionDir)$(Configuration)_$(Platform)\" and Intermediate Directory as "$(SolutionDir)obj\$(ProjectName)\$(Configuration)_$(Platform)\".

Change Include Directories as "..\include;$(IncludePath)":

You have to add "BUILDING_LIBICONV" and "BUILDING_LIBCHARSET" to Peprocessor Definitions of all Platforms and of all configurations.

You'd better set Runtime Library to "Multi-threaded" when building dynamic link library libiconv.dll. Then, the dependency on VC Runtime library can be controlled by the applications that will be built and dynamically linked with libiconv.dll because libiconv.dll does not need VC Runtime library but only the application that uses libiconv.dll may or may not need VC Runtime library. However, when building the static link library libiconvStatic.lib, you can choose Runtime Library option for libiconvStatic.lib depending on the application that uses libiconvStatic.lib.

You have to change Precompiled Header option to "Not Using Precompiled Headers".

Step 6. Tweak the source code.


libiconv.rc

Open libiconv.rc with text editor or the source code editor of Visual Studio IDE by double-clicking libiconv.rc in the Solution explorer and insert some code at line 4 as follows:

/////////////    ADD    /////////////

#define PACKAGE_VERSION_MAJOR 1

#define PACKAGE_VERSION_MINOR 14

#define PACKAGE_VERSION_SUBMINOR 0

#define PACKAGE_VERSION_STRING "1.14"

/////////////////////////////////////

You may be asked to change Line endings to "Windows (CR LF)". Then, let it do so. It will be more convenient for you if you mainly use Windows.

localcharset.c

Open localcharset.c and delete or comment the lines 80 - 83 as follows:

//////////////////  DELETE ////////////////

///* Get LIBDIR.  */

//#ifndef LIBDIR

//# include "configmake.h"

//#endif

///////////////////////////////////////////

iconv.c

Open iconv.c and delete or comment the lines 250 - 252 and add three lines there as follows:

///////////////////////// DELETE ///////////////////////

//size_t iconv (iconv_t icd,

//              ICONV_CONST char* * inbuf, size_t *inbytesleft,

//              char* * outbuf, size_t *outbytesleft)

/////////////////////////   ADD   //////////////////////

size_t iconv (iconv_t icd,

              const char* * inbuf, size_t *inbytesleft,

              char* * outbuf, size_t *outbytesleft)

////////////////////////////////////////////////////////


localcharset.h

Open localcharset.h and delete or comment the lines 21 - 25 and add 7 lines there as follows:

/////////////////////////   DELETE  ////////////////////////

//#if @HAVE_VISIBILITY@ && BUILDING_LIBCHARSET

//#define LIBCHARSET_DLL_EXPORTED __attribute__((__visibility__("default")))

//#else

//#define LIBCHARSET_DLL_EXPORTED

//#endif

/////////////////////////    ADD    //////////////////////

#ifdef BUILDING_LIBCHARSET

#define LIBCHARSET_DLL_EXPORTED __declspec(dllexport)

#elif USING_STATIC_LIBICONV

#define LIBCHARSET_DLL_EXPORTED

#else

#define LIBCHARSET_DLL_EXPORTED __declspec(dllimport)

#endif

////////////////////////////////////////////////////////////////////

config.h

Open config.h in the project include folder "...\libiconv\include" and delete or comment the lines 29 - 30 as follows:

///////////////////////// DELETE ///////////////////////

///* Define as good substitute value for EILSEQ. */

//#undef EILSEQ

////////////////////////////////////////////////////////

Otherwise you can redefine EILSEQ as good substitute value.

iconv.h

Open iconv.h in the project include folder "...\libiconv\include" and delete or comment the line 175 and add 1 line as follows:

/////////////////////////  DELETE  ///////////////////////

//#if @HAVE_WCHAR_T@

/////////////////////////    ADD   //////////////////////

#if HAVE_WCHAR_T

////////////////////////////////////////////////////////////////////////////////

Delete or comment the line 128 and add 1 line as follows:

/////////////////////////  DELETE  ///////////////////////

//#if @USE_MBSTATE_T@

/////////////////////////   ADD   //////////////////////

#if USE_MBSTATE_T

////////////////////////////////////////////////////////////////////////////////

Delete or comment the lines 107-108 and add 2 lines as follows:

/////////////////////////  DELETE  ///////////////////////

//#if @USE_MBSTATE_T@

//#if @BROKEN_WCHAR_H@

/////////////////////////  ADD  //////////////////////

#if USE_MBSTATE_T

#if BROKEN_WCHAR_H

////////////////////////////////////////////////////////////////////////////////

Delete or comment the line 89 and add 2 lines as follows:

/////////////////////////  DELETE ///////////////////////

//extern LIBICONV_DLL_EXPORTED size_t iconv (iconv_t cd, @ICONV_CONST@ char* * inbuf,

//size_t *inbytesleft, char* * outbuf, size_t *outbytesleft);

/////////////////////////    ADD   //////////////////////

extern LIBICONV_DLL_EXPORTED size_t iconv (iconv_t cd, const char* * inbuf,

  size_t *inbytesleft, char* * outbuf, size_t *outbytesleft);

////////////////////////////////////////////////////////////////////////////////

Delete or comment the lines 25 - 30 and add 8 lines as follows:

/////////////////////////  DELETE ///////////////////////

//#if @HAVE_VISIBILITY@ && BUILDING_LIBICONV

//#define LIBICONV_DLL_EXPORTED __attribute__((__visibility__("default")))

//#else

//#define LIBICONV_DLL_EXPORTED

//#endif

//extern LIBICONV_DLL_EXPORTED @DLL_VARIABLE@ int _libiconv_version; /* Likewise */

/////////////////////////    ADD   //////////////////////

#if BUILDING_LIBICONV

#define LIBICONV_DLL_EXPORTED __declspec(dllexport)

#elif USING_STATIC_LIBICONV

#define LIBICONV_DLL_EXPORTED

#else

#define LIBICONV_DLL_EXPORTED __declspec(dllimport)

#endif

extern LIBICONV_DLL_EXPORTED int _libiconv_version; /* Likewise */

////////////////////////////////////////////////////////////////////////////////

How to Use

When you use newly built libiconv, the only header file that you need is iconv.h. You will need to link either the import library libiconv.lib or the static library libiconvStatic.lib in your project property or write the code in one of your source file as follows:

#pragma comment (lib, "libiconv.lib")

or

#pragma comment (lib, "libiconvStatic.lib")

In the source of the application that uses this library either libiconv.dll or libiconvStatic.lib, if you don't define anything but only include iconv.h, your application will use libiconv.dll while it will use libiconvStatic.lib if you define USING_STATIC_LIBICONV before you include iconv.h in your application as follows:

//#define USING_STATIC_LIBICONV

#include <iconv.h>

And in C SDK code, I used a "SteamSensorWithFileTransferAndTunneling" sample code and added codes in the "dataCollectionTask" function as below.

void dataCollectionTask(DATETIME now, void * params) {

  iconv_t ic;

  char* in_buf = "Hi, 文健英";

  char *to_chrset = "UTF-8";

  char *from_chrset = "EUC-KR";

  size_t in_size = strlen(in_buf);

  size_t  out_size = sizeof(wchar_t) * in_size * 4;

  char* out_buf = malloc(out_size);

  // Caution: iconv's inbuf, outbuf are double pointers, so need to define separate pointers and pass addresses.

  char* in_ptr = in_buf;

  char* out_ptr = out_buf;

  size_t out_buf_left;

  size_t result;

  memset(out_buf, 0x00, out_size);

  

  ic = iconv_open(to_chrset, from_chrset);

  if (ic == (iconv_t) -1)

  {

        printf("Not supported code \n");

        exit(1);

  }

  printf("input len = %d, %s\n", in_size, in_buf);

  out_buf_left = out_size;

  iconv(ic, &in_ptr, &in_size, &out_ptr, &out_buf_left);

  //printf("input len = %d, output len=%d %s\n", in_size, out_size - out_buf_left, out_buf);

  iconv_close(ic);

  /* TW_LOG(TW_TRACE,"dataCollectionTask: Executing"); */

  properties.TotalFlow = rand()/(RAND_MAX/10.0);

  properties.Pressure = 18 + rand()/(RAND_MAX/5.0);

  properties.Location.latitude = properties.Location.latitude + ((double)(rand() - RAND_MAX))/RAND_MAX/5;

  properties.Location.longitude = properties.Location.longitude + ((double)(rand() - RAND_MAX))/RAND_MAX/5;

  properties.Temperature  = 400 + rand()/(RAND_MAX/40);

  properties.BigGiantString = out_buf; // Set values for String property

  /* Check for a fault.  Only do something if we haven't already */

  if (properties.Temperature > properties.TemperatureLimit && properties.FaultStatus == FALSE) {

  twInfoTable * faultData = 0;

  char msg[140];

  properties.FaultStatus = TRUE;

  properties.InletValve = TRUE;

  sprintf(msg,"%s Temperature %2f exceeds threshold of %2f",

  thingName, properties.Temperature, properties.TemperatureLimit);

  faultData = twInfoTable_CreateFromString("message", msg, TRUE);

  twApi_FireEvent(TW_THING, thingName, "SteamSensorFault", faultData, -1, TRUE);

  twInfoTable_Delete(faultData);

  }

  /* Update the properties on the server */

  sendPropertyUpdate();

}

Comments

Good sharing stuff. This is helpful for property value transfer for non-English characters

 

i am getting lots of compile errors in libiconv 1.16 and solution does not build.

Version history
Last update:
‎Jun 21, 2017 10:41 AM
Updated by:
Labels (1)