generating index on large document

[Darquise]

I have a 2500 page document that is taking 6 hours to generate. It's a custom bookmap with maprefs to topicrefs.

I have 2 issues:

1. The length of time it takes to generate the PDF:
   Is there a way to break up this document and still get a full index with correct page numbers? 
2. The styling of the index is messed up halfway through the index.
   We have been generating individual indexes for each document and the index looks good. Once we compiled the full book/PDF, the styling was all messed up. Our deadline to finalize content is next week. Note we have approx 142 topicrefs, broken down into approx 20 maprefs. Also, note, we used PE 7.0 to generate the PDF. I am currently testing with Styler to see if there's any improvement.

Can anyone help or offer advice on how to handle this large document or why the styling of the index is all messed up. I am attaching a screenshot of the index.

I can live with the 6 hour performance, however, I really need to fix the style.

[GarethOakes]

Which formatting engine are you using? APP or FOSI?

[Darquise]

APP Engine.

[GarethOakes]

OK that's good, that's the currently supported engine. Have you tried updating to the latest version of Arbortext? It is now up to 8.1.1.0 as of last month. The 7.x series is now out of support.

[Darquise]

I haven't tried to generate this book PDF with v8.0. We've had major performance issues with 7.1 and 8. Therefore, we've been using 7.0 for large book documents. For individual topics, we use AE 7.1 and PE 8.0.

There's a planned fixed for our PE performance issue: 

Is 8.1 released already? Support site of down right now. I'll have to check tomorrow.

[GarethOakes]

Support site is working here and yes the 8-1-1-0 release is available for PE, Editor, Styler.

The reason I ask is that fixing those index errors will be a real pain, much easier to update the software and take advantage of any fixes PTC has in place.

BTW, here at GPSL (www.gpsl.co) we specialize in support and development services for Arbortext, including APP Engine work. If you get stuck we may be able to help.

[Darquise]

Yes, it's working now. I am in the process of downloading and installing 8.1.

Thank you for the info and offer to help.

[Darquise]

I upgraded PE 8.1 and allowed the stylesheets to be updated. I adjusted PE performance, e.g. javavmmemory, bigjobthreshold etc. It took over 12 hours to run, and failed to create the PDF! That happened twice that the PDF failed with 8.1.

I spent the weekend increasing chunks of content until I could produce the error. Unfortunately, it happened when I hit approx 2400 page using PE Interactive 7.1.

Basically I have 20 chapter elements with maprefs - these all passed the test.

Then I added the Appendices, which have 4 appendix - this one failed.

I then tried 2 chapter elements with appendices and it passed.

It seems to be when I get to approx 2400 pages - very strange, very frustrating.

Any advice or suggestions in the meantime?

We have a 64bit Windows Server 2012 R2 Standard server, 32GB RAM

2 processors

Tomcat:

2048MB Initial memory pool

4096 MB Max memory pool

[GarethOakes]

You've done a lot of work - that sounds very frustrating. Has PTC support investigated this for you yet? They may be able to get a support engineer to look more deeply. We also have technical specialists here at GPSL that can investigate, but obviously at a cost. We basically would have to turn on all the debugging options, slice and dice the data as you have done, then dig into the formatting code to see what is taking up all the formatting time.

I remember some other cases where due to extreme size of the data we had to rewrite the stylesheets in native APP code (not using Styler dialog box) as this can give a huge speed boost.

I'm not sure any of this helps but perhaps the insights are useful?

[GarethOakes]

Sorry to hear of your issues. You can try PTC support, otherwise we do have staff who can assist with debugging this issue but that would be on commercial terms.

[Darquise]

Thank you for the offer. We've been working with Oberon in the past and they've done a lot of our ACLs and stylesheets.

We ended up creating an RDS document and publishing the PDF from the RDS. We had to create a minimal custom folder, use rds.dtd and dcf etc. We merged our styler modules and named it same as the RDS document. This allowed us to get a clean PDF to send to the printer. Note that it took 30 hrs to create the RDS, 8 hours to open the RDS in AE and then another 3 hours to generate the PDF :o!

We will be sending our findings to PTC to see if there's a bug. However, I suspect it could be in our ACL.

[GarethOakes]

Oberon are good guys!

Nothing clever comes to mind, however if you are going to be publishing a lot of large documents like this you might want to switch away from Styler stylesheets. We have not found them to have good performance characteristics for large documents.