Generic Web Services AddContent Error

[DavidLuxford]

I'm attempting to use the Generic Web Services SOAP API to change a file in WindChill and am running into a "Object "wt.doc.WTDocument>39760" is not persistent" error.

As per the [tiny] docs, I get the UFID of the content holder I want to update, call GetUploadHandles to get a URL to upload the content item to, upload the new revision of the file, and then call AddContent with the content holder's UFID and the handle returned by GetUploadHandles. The API always fails with the SOAP fault:

wt.util.WTException: wt.util.WTRuntimeException: Object "wt.doc.WTDocument>39760" is not persistent.

What's funny about this is the error appears to include an almost-but-not-quite-valid OBID. None of our Generic Web Services tasks have been modified, so I am wondering if this is a bug in the shipped tasks.

The only lead I have is from the Tomcat log, where I see what appears to be an error deserializing a SOAP call while parsing an integer, but it doesn't make sense because there are no integer fields in this method call:

java.lang.NumberFormatException: For input string: ""
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Integer.parseInt(Integer.java:470)
...
at com.infoengine.soap.util.sax.BodyDeserializer.startElement(BodyDeserializer.java:84)
...
at com.infoengine.soap.SoapRPCRouter.doPost(SoapRPCRouter.java:340)
...
at java.lang.Thread.run(Thread.java:619)

And since it says 'for input string ""', that makes me think some internal call is missing a parameter. The SOAP call is generated via soapUI, a Java client, and is valid. So...I dunno...

Other useful information:

1. Login is an admin user

2. Content holder is a WTDocument located in \cybernet_product called UAC with a single content item whose filename is UAC.pptx.

3. No errors uploading the file or in any of thepreceeding API calls.

4. File is not checked out.

5. I've tried calling AddContent with a contenthandle where the URL is the google logo, but that makes no difference.

6. I've tried changing the method in the contenthandle to GET, no difference.

7. Also tried setting the contenthandle url to just "uac.pptx", no difference.

If anyone has any insight or ideas, I'd appreciate your thoughts.

[JørnAHansen]

We have not experienced the actual error you receive.

We have AddContent working with 9.1 M020. Below is a snippet of Python code using cURL and Suds to do the actual heavylifting.

Getting something similar to work from .NET though has taken a lot of work including sniffing data on the wire with Wireshark. Let's just say that different implementations of SOAP are not compatible out of the box 😞

Note: PDFsnaplib in the code below is a thin wrapper around suds doing a bit of password handling etc.

import PDFsnaplib
import os
import logging
logging.basicConfig(level=logging.INFO, filename='wncSoap.log', filemode='w')

## 1 --->
def getHandles(filenames):
handles = s.client.service.GetUploadHandles(filenames)
#print handle[0].url
return handles

## 2 --->
def upload(handles, filenames):
user = PDFsnaplib.username + ":" + PDFsnaplib.password_string # ugly hack - but handy
i = 0
for filename in filenames:
cmd = 'curl -k --user ' + user + ' --form "filename=@' + filename + '" "' + handles[i].url + '"'
os.system(cmd)
i += 1

## 0 ---> prepare
s = PDFsnaplib.Snapper("tstpdm", '.')
#print s.client
logging.getLogger('suds.client').setLevel(logging.DEBUG)

filenames = ["8001234_W30.txt", "Upload.py"]
handles = getHandles(filenames)
upload(handles, filenames)

## 3 ---> do the magic
doc = s.client.service.Query("wt.doc.WTDocument", "number = '0000000201'")

print s.client.service.AddContent(doc[0].ufid, handles)

Here is the init method from PDFsnaplib.Snapper

class Snapper():

def __init__(self, host="yourwindchillservername.com", outpath="."):
logging.basicConfig(level=logging.INFO)
#logging.getLogger('suds.client').setLevel(logging.DEBUG) # will show all SOAP messages 🙂

# doctor needed due to Axis common ommision - see https://fedorahosted.org/suds/wiki/Documentation#FIXINGBROKENSCHEMAs
imp = Import('http://schemas.xmlsoap.org/soap/encoding/')
imp.filter.add('http://www.ptc.com/infoengine/soap/rpc/message/')
doctor = ImportDoctor(imp)

## service documented at http://yourwindchillservername.com/Windchill/infoengine/jsp/tools/doc/index.jsp
service="com.ptc.windchill.ws"
#url = 'http://' + host + '/Windchill/servlet/SimpleTaskDispatcher?CLASS=' + service
url = 'http://' + host + '/Windchill/servlet/RPC?CLASS=' + service
self.client = Client(url, doctor=doctor, username=get_user(host), password=get_passwd(host))
self.hostname = host
self.outpath = abspath(outpath) + '\\'

HTH,

Jørn

[DavidLuxford]

Thankyou. I analyzed your code and didn't find anything different from what I am doing, but you have pointed me along the path of SOAP communication issues and I'd like to explore that. That could well be it here. Would you mind capturing a request/response log for when you do a GetUploadHandles/upload/AddContent call sequence and posting it? I think that is what I need to figure this out. I've done so and have attached my log if you're interested. WireShark is fine, or you can try Fiddler, which is lighter weight.

Once again, thankyou for your time!

[JørnAHansen]

I have sent you a private message with a logfile.

Hope you can narrow it further down with this!

Otherwise just give me a poke again

/Jørn

[DavidLuxford]

That was quite helpful, thankyou for your help! Your log helped me fix a couple problems and I am stuck with only one real issue. Here's my key findings:

1. When WindChill marshalls a contenthandle object, it does not properly encoded it in XML. For example, the end of the upload handle URL has an & in it, but it's returned unescaped, *and expects it to be sent back unescaped* in the AddContent call. Doesn't seem to like CDATA in soap requests.

2. The case of the content item's fileName must match exactly with that in WindChill.

3. There is some sensitivity to the fileName used for upload, too, but I haven't figured that out yet, because...

4. This API is unstable. I call it once and it works, change the file data, call it again 15 seconds later, and it fails with the "Object is not persisted" error. Sometimes if I wait longer it works, but it seems if I call too soon, it stays stuck, and I'm not sure how soon too soon is. In all cases, identical call sequences and arguments, nobody else using this file. This is odd.

If you could send me just the headers from the curl upload, that would help me with #3. Hey, at least I have it working sometimes now. Your response has been quite helpful, thankyou again.

David L-

[DavidLuxford]

After a lot of work, I have gotten the AddContent call sequence to work somewhat reliably. This part of WindChill appears very brittle - not merely difficult, but unstable. It works once, then breaks with the same data and call sequence called 30 seconds later. I think PTC has some work to do here.

Here are the problems I encountered and solutions. My client is WCF, but that is hardly relevant since I have found I have to hand craft nearly all SOAP interaction due to InfoEngine's poor interoperability (this is item 5).
1. GetUploadHandles violates XML. The url member is returned unencoded. Since the URL always includes an &, which must be escaped in XML, this is problematic for all clients. This can result in AddContent failing with "File not found".
2. GetUploadHandles must be called with the exact case of the content item's fileName or AddContent will fail with "File not found".
3. The name of the file uploaded to the GetUploadHandles URL needs to be the same as that file - not just the node name, the entire filename must equal the content item fileName. Consequently, you may need to copy your file to update to a temp file with the same name, change the working directory to the temp directory, upload specifying just that as the filename, then restore the directory. Alternately, you could override the filename in the upload header.
4. AddContent violates XML. AddContent expects to receive contenthandles where the url is unencoded. This can result in AddContent returning "File not found" or "Object is not persisted".
5. AddContent does not recognize standard SOAP requests. In particular, it cannot parse SOAP where array element data is specified via an href to another part of the XML where it is fully defined. This applies to all other InfoEngine SOAP operations.
6. If calls to AddContent are not spaced by 15 minutes, it will fail with "Object is not persisted" errors. Success percentages decrease with less time delays, from 0% at 15 seconds, 35% at 2 min, to 100% at 15 min. The following workaround increases success to 75% with no delay:
if (fault exception message contains "not persistent")
// This workaround will not work without this delay!
Wait 10 seconds
Checkout the content holder
Undo checkout of content holder
Retry AddContent
7. Adding a 10 second sleep after the entire AddContent call sequence increases success rates to 95%.

Thanks again to Jorn, whose quick response with code samples and logs got me unstuck.

[DavidLuxford]

See summary in my final reply.

[DavidLuxford]

One more note for anyone else who comes across this. I've found that the AddContent API is poorly named. AddContent creates a new iteration and attaches to it only the files you have specified in the call. Consequently, if you call AddContent with one file on a content holder that has 3 in the latest iteration, you will get a new iteration with only one file. This is not obvious from the name or limited commentary. Also, if the content holder is already checked out to you, AddContent will fail with "content holder is already checked out"...even though you are the one that has it checked out. Finally, although AddContent wants it's url element to look exactly as provided by GetUploadHandles - which means not properly XML encoded, since the & to specify the filename argument is not encoded - it will not accept URLs from a content item's downloadURL property. In other words, you can't try to save time by finding the latest rev of the content items and just getting their URLs and providing them in the contenthandle array for AddContent. Those URLs have more &s and that is where AddContent breaks in parsing the request, encoded or not. Thus I hypotheisze a workaround would be to manually download each content item from the latest iteration, then call GetUploadHandles for them all, upload all of them + your one changed file, and then call AddContent.

If somebody knows a better way, I'm all ears.