Thingworx Azure IoT Hub Connector messages

[vi1]

Hello,

I am using Azure IoT Hub connector 3.0.1 and Thingworx version version is 8.5.1.

I have been observing messages are missing into Azure IoT Hub connector. I am receiving 20 messages from device to Azure IoT Hub Portal in every minute but all messages are not received into Thingworx Azure IoT Hub, 9 or 10 messages are receiving into Thingworx IoT Hub.

In cxserver log file, sequence number was captured but there is no data. Refer the below logs. Logs are captured from cxserver log file.

"sequence=1370211,sequence=1370212,sequence=1370213,sequence=1370214"---> No data for these sequence number.

..............................................................................................

021-02-15T03:10:09.553 [[encoder-dev-xxx|$Default|AzureIot-dXXXXXXXXXXXXXXXXXXXXXXXXXX]-1-7] DEBUG c.t.p.azure.iot.EventProcessor - Partition events; partitionId=0
2021-02-15T03:10:09.553 [[encoder-dev-xxx|$Default|AzureIot-dXXXXXXXXXXXXXXXXXXXXXXXXXX]-1-7] DEBUG c.t.p.azure.iot.EventProcessor - Received: partitionId=0 offset=1047993549288, sequence=1370211, enqueuedTime=2021-02-15T10:10:09.528Z
2021-02-15T03:10:09.553 [[encoder-dev-xxx|$Default|AzureIot-dXXXXXXXXXXXXXXXXXXXXXXXXXX]-1-7] DEBUG c.t.p.azure.iot.EventProcessor - Received: partitionId=0 offset=1047993560400, sequence=1370212, enqueuedTime=2021-02-15T10:10:09.528Z
2021-02-15T03:10:09.553 [[encoder-dev-xxx|$Default|AzureIot-dXXXXXXXXXXXXXXXXXXXXXXXXXX]-1-7] DEBUG c.t.p.azure.iot.EventProcessor - Received: partitionId=0 offset=1047993563496, sequence=1370213, enqueuedTime=2021-02-15T10:10:09.528Z
2021-02-15T03:10:09.553 [[encoder-dev-xxx|$Default|AzureIot-dXXXXXXXXXXXXXXXXXXXXXXXXXX]-1-7] DEBUG c.t.p.azure.iot.EventProcessor - Received: partitionId=0 offset=1047993575480, sequence=1370214, enqueuedTime=2021-02-15T10:10:09.528Z
2021-02-15T03:10:09.553 [com.thingworx.protocol.azure.iot.EventProcessor-PlatformRetry-0] DEBUG c.t.p.a.i.PropertiesMessageProcessor - Sending property write to thing: thing=edgethingxxx moduleId=null
2021-02-15T03:10:09.553 [com.thingworx.protocol.azure.iot.EventProcessor-PlatformRetry-0] DEBUG c.t.p.a.i.PropertiesMessageProcessor - Sending property write to thing: thing=edgethingxxx moduleId=null
2021-02-15T03:10:09.553 [com.thingworx.protocol.azure.iot.EventProcessor-PlatformRetry-0] DEBUG c.t.p.a.i.PropertiesMessageProcessor - Sending property write to thing: thing=edgethingxxx moduleId=null
2021-02-15T03:10:09.553 [com.thingworx.protocol.azure.iot.EventProcessor-PlatformRetry-0] DEBUG c.t.p.a.i.PropertiesMessageProcessor - Sending property write to thing: thing=edgethingxxx moduleId=null
2021-02-15T03:10:09.555 [NettyClient-NIO-2] DEBUG c.t.s.i.c.DefaultCorrelationChannelImpl - RESPONSE received: V1ResponseMessage [header=V1MessageHeader [messageType=64, requestId=36532, endpointId=4, sessionId=-1, flags=0], resultCode=STATUS_SUCCESS, resultMessage=null, result=null]
2021-02-15T03:10:09.556 [NettyClient-NIO-4] DEBUG c.t.s.i.c.DefaultCorrelationChannelImpl - RESPONSE received: V1ResponseMessage [header=V1MessageHeader [messageType=64, requestId=36535, endpointId=4, sessionId=-1, flags=0], resultCode=STATUS_SUCCESS, resultMessage=null, result=null]
2021-02-15T03:10:09.556 [NettyClient-NIO-5] DEBUG c.t.s.i.c.DefaultCorrelationChannelImpl - RESPONSE received: V1ResponseMessage [header=V1MessageHeader [messageType=64, requestId=36534, endpointId=4, sessionId=-1, flags=0], resultCode=STATUS_SUCCESS, resultMessage=null, result=null]
2021-02-15T03:10:09.556 [NettyClient-NIO-4] DEBUG c.t.s.i.c.DefaultCorrelationChannelImpl - RESPONSE received: V1ResponseMessage [header=V1MessageHeader [messageType=64, requestId=36533, endpointId=4, sessionId=-1, flags=0], resultCode=STATUS_SUCCESS, resultMessage=null, result=null]
2021-02-15T03:10:09.556 [[encoder-dev-xxx|$Default|AzureIot-dXXXXXXXXXXXXXXXXXXXXXXXXXX]-1-7] DEBUG c.t.p.azure.iot.EventProcessor - Checkpointing; partitionId=0
2021-02-15T03:10:09.556 [[encoder-dev-xxx|$Default|AzureIot-dXXXXXXXXXXXXXXXXXXXXXXXXXX]-1-7] DEBUG c.m.a.e.PartitionContext - host
AzureIot-dXXXXXXXXXXXXXXXXXXXXXXXXXX: 0: Saving checkpoint: 1047993575480//1370214
2021-02-15T03:10:09.556 [[encoder-dev-xxx|$Default|AzureIot-dXXXXXXXXXXXXXXXXXXXXXXXXXX]-1-7] DEBUG c.m.a.e.AzureStorageCheckpointLeaseManager - host AzureIot-dXXXXXXXXXXXXXXXXXXXXXXXXXX: 0: Checkpointing at 1047993575480 // 1370214
2021-02-15T03:10:09.556 [[encoder-dev-xxx|$Default|AzureIot-dXXXXXXXXXXXXXXXXXXXXXXXXXX]-1-7] DEBUG c.m.a.e.AzureStorageCheckpointLeaseManager - host AzureIot-dXXXXXXXXXXXXXXXXXXXXXXXXXX: 0: Updating lease
2021-02-15T03:10:09.556 [[encoder-dev-xxx|$Default|AzureIot-dXXXXXXXXXXXXXXXXXXXXXXXXXX]-1-7] DEBUG c.m.a.e.AzureStorageCheckpointLeaseManager - host AzureIot-dXXXXXXXXXXXXXXXXXXXXXXXXXX: 0: Renewing lease

Regards,

Latha

[vi1]

Hello,

Adding observation points:

In Azure IoT Hub Portal: Received messages at below time stamps.

2021-02-16T08:24:33.419Z, 2021-02-16T08:24:33.411Z, 2021-02-16T08:24:33.405Z, 2021-02-16T08:24:33.401Z, 2021-02-16T08:24:33.387Z

In Thingworx Azure IoT Hub Connector: Received messages at 2021-02-16T08:24:33.419Z, there is no remaining timestamps messages. Looks like if time stamp difference in milliseconds to same time(8:24:33), its not receiving all the messages into Thingworx IoT Connector.

Do I need to change any configuration on Thingworx?

Regards,

Latha

[c_lowy]

It seems that this issue requires some in depth investigation and is being addressed in the context of a support case.

The solution will be posted once the case is resolved.

[vi1]

Hello,

Thank you for response. I have raised support case already.

Regards,

Latha

[c_lowy]

@vi1 Indeed I saw that you opened a support case, thanks !

[mnarang · Accepted solution]

This issue has been resolved on the support case. The event processing threads are doing their task and no data is getting missed from subscription. While there are few miss in main EDGE thing in value stream. This is as per design because they are operating on same Thing/Same Value Stream and same property so milliseconds updates are getting missed as thread is overriding it with the last update.