Creating Profile aborted - Spark maxResultSize

Start a topic

cancel

Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Search instead for

Did you mean:

cancel

Turn on suggestions

Auto-suggest helps you quickly narrow down your search results by suggesting possible matches as you type.

Showing results for

Search instead for

Did you mean:

Close X

Community Tip - Want the oppurtunity to discuss enhancements to PTC products? Join a working group! X

Community
ThingWorx
ThingWorx Developers
Creating Profile aborted - Spark maxResultSize

Options

Subscribe to RSS Feed
Mark Topic as New
Mark Topic as Read
Float this Topic for Current User
Bookmark
Subscribe
Mute
Printer Friendly Page

Creating Profile aborted - Spark maxResultSize

skef

12-Amethyst

‎Jul 31, 2018 04:57 AM

Mark as New
Bookmark
Subscribe
Mute
Subscribe to RSS Feed
Permalink
Print
Notify Moderator

‎Jul 31, 2018 04:57 AM

Creating Profile aborted - Spark maxResultSize

A 'CreateProfile' job is aborted, throwing this exception:

SparkException: Job aborted due to stage failure: Total size of serialized results of 69 tasks (1026.2 MB) is bigger than spark.driver.maxResultSize (1024.0 MB)

Obviously, thingworx analytics is using apache spark for analyzing datasets. I need the maxResultSize to be configurable in order to be able to get the profiles of a large dataset.

Is are any possibility to change the maxResultSize in the micro service?

Labels: