REST API, Upload file, Stage2 'Failure'

[jlpatte2]

I am using the ODATA REST API to try and upload new file content to a document per REST API Documentation: WCCG_RESTAPI_173713.pdf, Page ~51.

I have successfully executed Stage 1

POST https://server_base_addresshere/Windchill/servlet/odata/DocMgmt/Documents('OR%3Awt.doc.WTDocument%3A553531117')/PTC.DocMgmt.uploadStage1Action HTTP/1.1
Accept: application/xml
Authorization: Basic basicAuthHash
CSRF_NONCE: FFYxrn635KI1zhh0VyJmx0bN1OttoGAbYzweyD35ipN2mVwGfG5r1wSAi89hoC5HUGZFxgj9sZ8P/y1ALW8ImkiO1JsD/CI5Pz90gRfPtvJwgHEMc2IIlzyGhpt08yU=
Content-Type: application/json; charset=utf-8
Host: windchill.plexus.com
Cookie: JSESSIONID=92811C6234652861D37C0EEA3252D1CE.tomcat2
Content-Length: 15
Expect: 100-continue

{"noOfFiles":1}

I get back a successful set of replicaURL, MasterURL, streamIDs and FileNames.

I then create a stage 2 request following the guidelines in the rest document

POST https://replicaurlBaseAddressHere/Windchill/servlet/WindchillGW/wt.fv.uploadtocache.DoUploadToCache_Server/doUploadToChache_Replica?mk=wt.fv.uploadtocache.DoUploadToCache_Server&VaultId=1027696&FolderId=495134041&CheckSum=55879253&sT=1549994693&sign=q8pdvh4fy1dCM7srwl3%2BOwH2W0Wti7AEpgS2xpfdgHZMpHsTze1WPl2OiB%2FG3FyLPqJCU%2BBkjdbxXuX5IiE4Bw%3D%3D&site=https%3A%2F%2Fbasewindchilladdres.com%2FWindchill%2Fservlet%2FWindchillGW&AUTH_CODE=RSA&isProxy=true&delegate=wt.fv.uploadtocache.DefaultRestFormGeneratorDelegate HTTP/1.1
Accept: application/xml
Authorization: Basic basicauthhash
CSRF_NONCE: FFYxrn635KI1zhh0VyJmx0bN1OttoGAbYzweyD35ipN2mVwGfG5r1wSAi89hoC5HUGZFxgj9sZ8P/y1ALW8ImkiO1JsD/CI5Pz90gRfPtvJwgHEMc2IIlzyGhpt08yU=
Content-Type: multipart/form-data; boundary="-----------------------------boundary"
Host: replicaURLServerHostNameHere
Content-Length: 3027
Expect: 100-continue

-------------------------------boundary
Content-Type: text/plain; charset=utf-8
Content-Disposition: form-data; name=Master_URL

https://BaseWindchillurl.com/Windchill/servlet/WindchillGW
-------------------------------boundary
Content-Type: text/plain; charset=utf-8
Content-Disposition: form-data; name=CacheDescriptor_array

26646757:29232496:26646757
-------------------------------boundary
Content-Type: text/plain
Content-Disposition: form-data; name=26646757; filename=config.da0; filename*=utf-8''config.da0

[CFG]
config last_saved "16:06:16, 08 Feb 2019"
...file contents here ...
[ENDCFG]

-------------------------------boundary--

This then is sent, and it "works" as in there is no error, but it returns an empty contentInfos structure:

HTTP/1.1 200 200
Date: Tue, 12 Feb 2019 18:04:56 GMT
Server: Apache/2.4.25 (Win64) mod_jk/1.2.42 OpenSSL/1.0.2k
X-Ptc-Connected: 1
X-Frame-Options: ALLOW-FROM https://windchillbaseurl.com
X-do-not-compress-this: 1
Vary: Accept-Encoding,User-Agent
access-control-allow-origin: https://windchillbaseurl.com
access-control-allow-credentials: true
expires: Thu, 01 Jan 1970 00:00:00 GMT
Content-Length: 19
Content-Type: text/json;charset=UTF-8

{"contentInfos":[]}

Does anyone have any ideas why? (note the listed data has been sanitized to remove company and password data. So far I have tried the following:

• A variety of different content types for each of the 3 multi part form parts.
• Fooled with the cache-descriptor lines a bit
• Have tried with the source document checked in, checked out, with an already existing file, and w/o.

Any help would be greatly appreciated because I feel like I'm super close to getting this working, but so far w/o any kind of error message to point me in the right direction.

Thanks, Josh Patterson

[jlpatte2]

So since I wasn't actually getting an error from stage 2, I looked at the encoded info that stage 3 needs, and there doesn't seem to be any new information in it besides the size of the file created on the server side. So "guessing" that it is the same as the client and transfer file size, I generated a stage 3 request.

Interestingly that returned a object must be checked out error. So I implemented object checkout (however even with checkout I still get back and empty result from stage 2), now I get the following XML error response to stage 3:

<?xml version='1.0' encoding='UTF-8'?><error xmlns="http://docs.oasis-open.org/odata/ns/metadata"><code>null</code><message>3</message></error>

a very helpful null coded error "3".

[psychodracon]

Any news on this? I`m also trying to use upload rest api and i`m getting same error.

[jlpatte2]

Unfortunately I have not found a solution. It has shifted to my back burner due to other issues beyond my control, though I would love to get an answer here.

Josh

[psychodracon]

I managed to get little bit furder with this. It appears that "boundary" must be always two "-" longer so if you put 

boundary to 123456798 you must start your request body with --123456789.

Problem is that now i`m not getting empty "contentInfos" but server is returning error:

2019-02-26 16:28:47,104 ERROR [ajp-nio-127.0.0.1-8010-exec-1] wt.fv.uploadtocache -
java.lang.Exception: java.lang.NullPointerException
at wt.fv.uploadtocache.DoUploadToCache_Server.doMultiPartUploadToCache_Server(DoUploadToCache_Server.java:1848)... 
2019-02-26 16:28:47,120 ERROR [ajp-nio-127.0.0.1-8010-exec-1] wt.fv.uploadtocache.DefaultRestFormGeneratorDelegate - Upload content failed with message [java.lang.NullPointerException]

[jlpatte2]

Did you ever make forward progress on this?

[amarsingh]

@jlpatte2 It looked like your stage 2 Header has below. Please remove it and that should work

Accept: application/xml

Please refer to this document Windchill REST Services 1.5 User's Guide (page 173)

NOTE: If you are using Postman, make sure you are using the latest version or the chrome extension (deprecated)

The rest looks something like

POST <Upload URL from stage 1>
Authorization: Basic basicauthhash
CSRF_NONCE: <CSRF_NONCE>

Content-Type: multipart/form-data; boundary=----WebKitFormBoundary7MA4YWxkTrZu0gW

------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="Master_URL"

http://amarsingh2v2k16.ptcnet.ptc.com/Windchill/servlet/WindchillGW
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="CacheDescriptor_array"

12001:12001:12001;14011:14011:14011;14012:14012:14012;
------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="12001"; filename="file1.txt"
Content-Type: text/plain


------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="14011"; filename="file2.txt"
Content-Type: text/plain


------WebKitFormBoundary7MA4YWxkTrZu0gW
Content-Disposition: form-data; name="14012"; filename="file3.txt"
Content-Type: text/plain


------WebKitFormBoundary7MA4YWxkTrZu0gW--

[GavinGuo]

Hi,

Did you test the three stages of uploading document contents successfully? I did try to test it and I'm sure my contents are correct, but I still meet the NullPointerException on stage2. 

The complete stack trace as below:

2020-04-10 14:31:21,338 DEBUG [ajp-nio-127.0.0.1-8010-exec-2] wt.fv.uploadtocache - DoUploadToCache_Server: doUploadToChache_Master FIRST STAGE - READ FROM REQUEST...
2020-04-10 14:31:21,338 DEBUG [ajp-nio-127.0.0.1-8010-exec-2] wt.fv.uploadtocache - Content Type = multipart/form-data; boundary=--boundary
2020-04-10 14:31:21,339 DEBUG [ajp-nio-127.0.0.1-8010-exec-2] wt.fv.uploadtocache - Getting information for uploading to cache server
2020-04-10 14:31:21,339 DEBUG [ajp-nio-127.0.0.1-8010-exec-2] wt.fv.uploadtocache - Vault Id = 163101
2020-04-10 14:31:21,339 DEBUG [ajp-nio-127.0.0.1-8010-exec-2] wt.fv.uploadtocache - Folder Id = 163108
2020-04-10 14:31:21,341 DEBUG [ajp-nio-127.0.0.1-8010-exec-2] wt.fv.uploadtocache - Caught exception! Message=<doMultiPartUploadToChache_Server : Caught throwable while storing content in vault.>
2020-04-10 14:31:21,341 DEBUG [ajp-nio-127.0.0.1-8010-exec-2] wt.fv.uploadtocache - Exception=java.lang.Exception: java.lang.NullPointerException, msg=java.lang.NullPointerException, lmsg=java.lang.NullPointerException
2020-04-10 14:31:21,342 ERROR [ajp-nio-127.0.0.1-8010-exec-2] wt.fv.uploadtocache -
java.lang.Exception: java.lang.NullPointerException
at wt.fv.uploadtocache.DoUploadToCache_Server.doMultiPartUploadToCache_Server(DoUploadToCache_Server.java:1848)
at wt.fv.uploadtocache.DoUploadToCache_Server.doUploadToChache_Master(DoUploadToCache_Server.java:131)
at sun.reflect.GeneratedMethodAccessor416.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at wt.httpgw.HTTPServer.processRequest(HTTPServer.java:346)
at wt.httpgw.MethodRequestHandler.handleRequest(MethodRequestHandler.java:58)
at wt.httpgw.HTTPGatewayServlet.serviceWithoutFilters(HTTPGatewayServlet.java:336)
at wt.httpgw.FilterChainImpl.doFilter(FilterChainImpl.java:94)
at wt.session.SessionContextDestroyer.doFilterInternal(SessionContextDestroyer.java:136)
at wt.session.SessionContextDestroyer.doFilter(SessionContextDestroyer.java:184)
at wt.httpgw.FilterChainImpl.doFilter(FilterChainImpl.java:81)
at wt.httpgw.HTTPGatewayServlet.service(HTTPGatewayServlet.java:235)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:741)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:231)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at wt.licenseusage.licensing.LicenseFilter.doFilter(LicenseFilter.java:47)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.ptc.jws.servlet.filter.WsdlServletFilter.doFilter(WsdlServletFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at com.ptc.core.ui.validation.URLValidationFilter.doFilter(URLValidationFilter.java:85)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at wt.httpgw.filter.WTContextBeanFilter.doWithWtContextBeanHandler(WTContextBeanFilter.java:104)
at wt.httpgw.filter.WTContextBeanFilter.doFilter(WTContextBeanFilter.java:58)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at wt.servlet.CompressionFilter.doFilter(CompressionFilter.java:301)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at wt.servlet.RequestInterrupter.doFilter(RequestInterrupter.java:335)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at wt.servlet.ServletRequestMonitor.doFilter(ServletRequestMonitor.java:1660)
at wt.servlet.ServletRequestMonitorFilter.doFilter(ServletRequestMonitorFilter.java:56)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:193)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:166)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:199)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:96)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:493)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:137)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:81)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:87)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:343)
at org.apache.coyote.ajp.AjpProcessor.service(AjpProcessor.java:476)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:66)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:808)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1508)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException
at wt.util.MPInputStream.genBodyHeaders(MPInputStream.java:307)
at wt.util.MPInputStream.hasMoreObjectBodies(MPInputStream.java:234)
at wt.fv.uploadtocache.DoUploadToCache_Server.doMultiPartUploadToCache_Server(DoUploadToCache_Server.java:1669)
... 54 more
2020-04-10 14:31:21,343 DEBUG [ajp-nio-127.0.0.1-8010-exec-2] wt.fv.uploadtocache - DoUploadToCache_Server: doUploadToChache_Master SECOND STAGE - WRITE INTO RESPONSE...
2020-04-10 14:31:21,344 DEBUG [ajp-nio-127.0.0.1-8010-exec-2] wt.fv.uploadtocache - DoUploadToCache_Server: doMultiPartUploadToCache_Server : DelegateClasswt.fv.uploadtocache.DefaultRestFormGeneratorDelegate
2020-04-10 14:31:21,344 ERROR [ajp-nio-127.0.0.1-8010-exec-2] wt.fv.uploadtocache.DefaultRestFormGeneratorDelegate - Upload content failed with message [java.lang.NullPointerException]
2020-04-10 14:31:21,345 ERROR [ajp-nio-127.0.0.1-8010-exec-2] wt.servlet.ServletRequestMonitor.request - 2020-04-10 14:31:21.336 +0000, q3pthcz;k8u8i5uk;9268;47o69t;5862, -, 127.0.0.1, /Windchill/servlet/WindchillGW/wt.fv.uploadtocache.DoUploadToCache_Server/doUploadToChache_Master, mk=wt.fv.uploadtocache.DoUploadToCache_Server&VaultId=163101&FolderId=163108&CheckSum=193224&sT=1586528488&sign=1BDYHFDlKskt0cmPNg62Yg%3D%3D&site=http%3A%2F%2Fgavin.ptc.com%2FWindchill%2Fservlet%2FWindchillGW&AUTH_CODE=HmacMD5&isProxy=true&delegate=wt.fv.uploadtocache.DefaultRestFormGeneratorDelegate, POST, 500, 0.0, 0.0086866

Could some give more suggestions? Very appreciate that. Thanks.

[jlpatte2]

I never got this working, and the project I was using it for got canceled. Apologies.

Josh

[schauvelin]

If you are still interested in this topic, please check article : https://www.ptc.com/en/support/article/CS330694 

Kind regards. 

[ldutoit · Accepted solution]

Hi Everyone,

I'm somehow very glad I'm not the only one struggling with the Stage2 load but also sad that I've struggled for 2 weeks already trying to get this to work ...
I'm able to complete the whole upload process(stage1,2 &3) with the appropriate 200 or 201 responses but when I try open an .xltm file from Windchill it is corrupt. I did have a look at :https://www.ptc.com/en/support/article/CS330694  but still no luck.

I can Load a simple .txt file into Windchill and open it afterwards fine but that is obviously a very simple characters format.

I get stuck with .xltm, .xls, .pdf files. What is the destinguishing parameters in the loads based on file type?

If I open up my local .xltm file(the one that does open in Excel fine and which I use for the Windchill upload) and compare it in Notepad++ with the file I uploaded into Windchill I see the local(not broken file) open in ANSI format whereas the Windchill file is UTF-8. There is definately some differences in the file when I compare in in Notepad++. I've tried a few things in Posman without any luck. Still getting a corrupt UTF-8 encode file out of Windchill. 

Can somebody please share the tricks of the trade here cause I'm desperate on this final hurdles.

Much appreciated.

[jukka]

Hi,

Yeah, FileReader does not work. It will mess up charset or something. Use FormData() and append method.

function fUploadStage2(ReplicaUrl,StreamIds,FileNames,FileSize,InputFile) {
	let formData = new FormData();
	formData.append('Master_URL','https://pdmtest.xxxxx.com/Windchill/servlet/WindchillGW');
	formData.append('CacheDescriptor_array', StreamIds+':'+FileNames+':'+StreamIds+':'+FileSize);
	formData.append(FileNames,InputFile);
	xhrstg2.open("POST", ReplicaUrl);
	xhrstg2.setRequestHeader("CSRF_NONCE", token);
	xhrstg2.send(formData);
}

[ldutoit]

Hi jukka,

thanks for the reply.

I'm not really a developer. Your response above looks like C#/Java. I would like to understand the Postman setup for the 3 stage Uploads process for each file type(e.g. Excel, PDF, Word).

Is it too much to ask for a Postman OpenAPI .json file attached here for Excel file type(e.g. xltm file type)? I would then be able open the .json in Postman to review where my mistakes are with my current Stage2 Load. Like i say my files currently loaded to windchill is currupt via Postman call. Would very much appreciate help with this please.

much aprreciated thanks

[jukka]

Hi,

The code is javascript. I took two screenshots from postman "stage 2". I hope these images helps.

[ldutoit]

Below I have a comparison between 2 files:
The LHS is the ANSI format pdf file that opens up correctly. This file was the file I attached to me front end. 

This file then gets streamed via binary into Windchill.

The RHS is the Windchill file that I've downloaded again. When i try open this file though it comes back with file corrupt message.

If looking closely at the picture below it is evident that each line has resembling characters but also we have characters that does not resemble. For example RHS has squares where LHS has some other characters. The LHS is ANSI where the RHS(corrupt file) is UTF-8. I think my question should be how does my Front End UI change ANSI in UTF-8?

[neisenhuth]

There is a typo within the request URL. Search for "Chache". You have to remove the letter "h" from that word. Unfortunately this typo is within the PTC help example. I´ve reported the typo to be fixed. Hope this helps.