The community will undergo maintenance on October 16th at 10:00 PM PDT and will be unavailable for up to one hour.
Hi Everybody,
I have an issue with thingworx and the current chrome version. I get a lot or Errors in Application Log:
Error occurred while validating HTTP header: sec-ch-ua. HTTP header value: sec-ch-ua: Invalid input. Please conform to regex ^[0-9a-zA-Z()\-=\*\.\?;,+\/:&_ %$ ¡-'"]*$ with a maximum length of 2000
So i did following steps:
Anyone has an idea?
Hi @atondorf.
Which version of ThingWorx are you running? Have you recently upgraded the environment? If so did you remove validation.properties so it gets recreated under the new version?
If you've made any modifications to validation.properties, you'll need to save the original in order to merge your changes back in after the file is recreated.
You also may need to double escape the backslash: \\\\\
Please let us know if you continue to have issues.
Regards.
--Sharon
Hi,
I have this behaviour on TWX 8.4. We also have a 9.0 instance for testing. But for some reason Chrome does not send the sec-ch-ua header to this instance (Maybe because it's not https).
So i tried it now with several approaches to add the '\' support to the Regex:
- Original RexEx String:
^[a-zA-Z0-9()\\-=\\*\\.\\?;,+\\/:&_ %\$ \u00A1-\uFFFF'"]*$: Get the error " Please conform to regex ... "
- Modified with '\' Support. I tried it with 3-6 of the '\' in the string. Always with the same result:
^[a-zA-Z0-9()\\-=\\*\\.\\?;,+\\/:&_ %\$ \u00A1-\uFFFF'"\\\]*$ Get the error " Input validation failure"
Is it possible to add an special Regex to handle the sec-ch-ua Header?
So e.g.: Validator.HTTPHeaderValue_sec-ch-ua=^.*$
I tried it with this string, but seems to be ignored by system as nothing changed.
Greetings
Andreas
Hi @atondorf.
We found that we do have an internal case for this issue. It is currently a work in progress but you can review this article for updates. Currently, the article indicates there is no timeline for a fix but it is actively being worked.
I will also be providing updates here for others in the community.
Regards.
--Sharon
Thanks for the information!
But I am really curious, why I am not able to just set the regex to accept the backslash ...
Hi @atondorf.
We received an update from R&D on the internal case with the following suggestions for solving the regex problem. You can start by adding/modifying the following options in ThingworxStorage/esapi/validation.properties:
Encoder.AllowMultipleEncoding=true
Encoder.AllowMixedEncoding=true
With these options, multiple encoder errors turn into warning messages in the logs. After turning an error into a warning message you can use logback.xml file (attached) so that the file can either shift the log or disable the class level log.
You will also need to add the following to ThingworxStorage/esapi/validation.properties:
ESAPI.Logger=org.owasp.esapi.reference.Log4JLogFactory
Validator.HTTPHeaderValue=<Modify the regex according to the input value from the error log>
Example: Validator.HTTPHeaderValue=^[a-zA-Z0-9()\\-=\\*\\.\\?;,+\\/:&_ %\$\\\\ \u00A1-\uFFFF'"]*$
After making any needed updates to the attached logback.xml file, copy it to the ThingworxPlatform folder to disable the Encoder level class so the multiple encoding warning will not appear in the Thingworx and Tomcat logs. Restart Tomcat.
After adding the above configuration any warning or error message will then show that it's clearly a regex validation issue, and you can make further adjustments as needed.
Please let us know if this helps to solve your issue.
Regards.
--Sharon
Hi @slangley,
thanks for your answer, but I am a little confused with it:
So I checked the other parameters you send me ... and they are all already not defined in "validation.properties" but in "ESAPI.properties":
This is the current situation without any “logback.xml”. And to be honest I do not understand this file at all:
So only with modification to "validation.properties" it's working, but I do not understand how it's working.
What did I really changed with AllowMultipleEncoding & AllowMixedEncoding and why is it strongly discouraged?
Is it a fix or is it just and ugly workaround?
By the way ... I attached my working validation.properties file, but added the extension "txt".
This forum did not allowed me to upload a textfile with "properties" extension.
Regex validation ...
Greetings
Andreas
Hi @atondorf ,
Glad that the errors are gone after you made the changes. The logback.xml file in the Thingworx.platform folder disables the Encoder level class loggers to avoid the multiple encoding warning in the logs. We will publish an article soon with further details and appropriate steps that shall answer most of your other questions.
Regards,
Bhushan
Hi @atondorf ,
We are still working on a permanent fix for the issue but just wanted check if the logs are still stable with the changes.
Regards,
Bhushan
Hi @atondorf ,
I have a quick update from R&D, the issue is no longer reproducible in the latest ThingWorx releases. I have updated the information in Article https://www.ptc.com/en/support/article/cs327573. Let me know if you still have any further questions
Regards,
Bhushan
Hi @atondorf.
After re-reading my response, I can see that it is confusing. Apologies!
I have reached out to a resource that was involved in the issue originally that should be able to provide a better explanation. If he doesn't respond over the next couple of days, I'll track someone down for a better understanding of what needs to be done.
Happy Holidays!
--Sharon