Is there a color type that can be used in a custom widget that allows a user to pick a color similar to the colorpicker in a styledefinition?
Unfortunately there is not. As of right now, using the STYLEDEFINITION and allowing developers to pick a custom style, as you mentioned, is the best way to add styles to custom widgets.
That said, this functionality does exist within Thingworx Studio when building mobile applications and AR experiences (which also have the ability for custom widgets, with a type color for a color picker).
Will this functionality be available in a future version of the composer?
My goal is that there is a property called background color so the developer can pick just a background color.
Using a style definition is a bit overkill in this case, and might get confusing as there are multiple background colors in a style definition (primary and secondary).
Yes, it will be because the new mashup builder is being built based on Thingworx Studio. That said, I do not know how long away this will be from a production release from a road map perspective. An alternative, for the time being, would be to set the property as a STRING type and have them enter in the hex color code value -- this will be easy to modify, later, to use the new approach since the new Thingworx Studio mashup builder will be CSS based.