Creating Profile aborted - Spark maxResultSize
Jul 31, 2018
04:57 AM
- Mark as New
- Bookmark
- Subscribe
- Mute
- Subscribe to RSS Feed
- Permalink
- Notify Moderator
Jul 31, 2018
04:57 AM
Creating Profile aborted - Spark maxResultSize
A 'CreateProfile' job is aborted, throwing this exception:
SparkException: Job aborted due to stage failure: Total size of serialized results of 69 tasks (1026.2 MB) is bigger than spark.driver.maxResultSize (1024.0 MB)
|
Obviously, thingworx analytics is using apache spark for analyzing datasets. I need the maxResultSize to be configurable in order to be able to get the profiles of a large dataset.
Is are any possibility to change the maxResultSize in the micro service?
Labels:
- Labels:
-
Analytics
0 REPLIES 0
