I have an issue with thingworx and the current chrome version. I get a lot or Errors in Application Log:
Error occurred while validating HTTP header: sec-ch-ua. HTTP header value: sec-ch-ua: Invalid input. Please conform to regex ^[0-9a-zA-Z()\-=\*\.\?;,+\/:&_ %$ ¡-'"]*$ with a maximum length of 2000
So i did following steps:
Anyone has an idea?
Which version of ThingWorx are you running? Have you recently upgraded the environment? If so did you remove validation.properties so it gets recreated under the new version?
If you've made any modifications to validation.properties, you'll need to save the original in order to merge your changes back in after the file is recreated.
You also may need to double escape the backslash: \\\\\
Please let us know if you continue to have issues.
I have this behaviour on TWX 8.4. We also have a 9.0 instance for testing. But for some reason Chrome does not send the sec-ch-ua header to this instance (Maybe because it's not https).
So i tried it now with several approaches to add the '\' support to the Regex:
- Original RexEx String:
^[a-zA-Z0-9()\\-=\\*\\.\\?;,+\\/:&_ %\$ \u00A1-\uFFFF'"]*$: Get the error " Please conform to regex ... "
- Modified with '\' Support. I tried it with 3-6 of the '\' in the string. Always with the same result:
^[a-zA-Z0-9()\\-=\\*\\.\\?;,+\\/:&_ %\$ \u00A1-\uFFFF'"\\\]*$ Get the error " Input validation failure"
Is it possible to add an special Regex to handle the sec-ch-ua Header?
So e.g.: Validator.HTTPHeaderValue_sec-ch-ua=^.*$
I tried it with this string, but seems to be ignored by system as nothing changed.
We found that we do have an internal case for this issue. It is currently a work in progress but you can review this article for updates. Currently, the article indicates there is no timeline for a fix but it is actively being worked.
I will also be providing updates here for others in the community.
We received an update from R&D on the internal case with the following suggestions for solving the regex problem. You can start by adding/modifying the following options in ThingworxStorage/esapi/validation.properties:
With these options, multiple encoder errors turn into warning messages in the logs. After turning an error into a warning message you can use logback.xml file (attached) so that the file can either shift the log or disable the class level log.
You will also need to add the following to ThingworxStorage/esapi/validation.properties:
Validator.HTTPHeaderValue=<Modify the regex according to the input value from the error log>
Example: Validator.HTTPHeaderValue=^[a-zA-Z0-9()\\-=\\*\\.\\?;,+\\/:&_ %\$\\\\ \u00A1-\uFFFF'"]*$
After making any needed updates to the attached logback.xml file, copy it to the ThingworxPlatform folder to disable the Encoder level class so the multiple encoding warning will not appear in the Thingworx and Tomcat logs. Restart Tomcat.
After adding the above configuration any warning or error message will then show that it's clearly a regex validation issue, and you can make further adjustments as needed.
Please let us know if this helps to solve your issue.
thanks for your answer, but I am a little confused with it:
So I checked the other parameters you send me ... and they are all already not defined in "validation.properties" but in "ESAPI.properties":
This is the current situation without any “logback.xml”. And to be honest I do not understand this file at all:
So only with modification to "validation.properties" it's working, but I do not understand how it's working.
What did I really changed with AllowMultipleEncoding & AllowMixedEncoding and why is it strongly discouraged?
Is it a fix or is it just and ugly workaround?
By the way ... I attached my working validation.properties file, but added the extension "txt".
This forum did not allowed me to upload a textfile with "properties" extension.
Regex validation ...
Hi @atondorf ,
Glad that the errors are gone after you made the changes. The logback.xml file in the Thingworx.platform folder disables the Encoder level class loggers to avoid the multiple encoding warning in the logs. We will publish an article soon with further details and appropriate steps that shall answer most of your other questions.
Hi @atondorf ,
We are still working on a permanent fix for the issue but just wanted check if the logs are still stable with the changes.
After re-reading my response, I can see that it is confusing. Apologies!
I have reached out to a resource that was involved in the issue originally that should be able to provide a better explanation. If he doesn't respond over the next couple of days, I'll track someone down for a better understanding of what needs to be done.