No experience with that function. Are you querying a Documentum repository? I have DQL queries of various types saved.
Here's one: select object_name, r_version_label, owner_name, r_object_type, a_content_type from dm_sysobject WHERE (((upper(object_name) = upper(' myobjectname')))) AND folder('/foldername', descend)
AND folder() searches a top level folder (or cabinet). descend recurses all folders in the named folder.
The query above probably has more () than necessary. I tended to build queries, save them. Copy paste and adapt them. Save them. And repeat as I had different needs. If it worked, I was happy ... I got no style points ... so ...
DQL can be persnickety about quote types. If you're already doing something close to this, I'll guess proper quoting might be your problem.
It might help to know that the DQL is translated prior to execution to the query language of the hosting database. In our case, it was SQL Server or t-sql. You might be in Oracle. *Maybe* this is ... relevant in edge cases?