cancel
Showing results for 
Search instead for 
Did you mean: 
cancel
Showing results for 
Search instead for 
Did you mean: 

Community Tip - Did you get an answer that solved your problem? Please mark it as an Accepted Solution so others with the same problem can find the answer easily. X

Restore system

BenLoosli
23-Emerald II

Restore system

When doing an upgrade on different virtual hardware, UpgradeManager modified my production database and now Windchill will not start. My first comment is that this should NEVER happen. I do upgrades on new computers so my existing systems remains viable.

Now that I have a corrupt system due to PTC, how do I restore it? I have backups taken about a 6 weeks before the attempted upgrade. That is my fault for not taking them just before the upgrade. My backup data consists of the Oracle DB, the file vaults and an LDIF export. What had me confused was that my source Windchill system was still running after the failed upgrade attempt. Only when IT rebooted the computer did the issue come up with the method servers failing to start due to incompatible data.

So, what steps do I need to take to restore the system to my April backup? I am thinking about wiping both systems and rebuilding them from scratch and then importing the Oracle and LDIF files and restoring the vaults. Is there a restore method that I could do without wiping the 2 servers?

 

This issue has also made me gun shy on doing upgrades to my other 2 Windchill systems as they are both current active production systems. This third Windchill was taken offline in July due to a contract termination, but I was using it for the practice of going from Windchill 11.0 M030 to Windchill 12.0.20.6, as that is the configuration of one of my production systems, the other is 11.1 M020 CPS20. I will do full backups of the Oracle and LDAP before doing upgrades in the future, even for testing. Both of these systems have daily Oracle dumps done nightly and the 11.1 systems uses robocopy for a daily backup vault copy.

16 REPLIES 16
TomU
23-Emerald IV
(To:BenLoosli)

Right off the bat I would get an 'enterprise down' case open with PTC.

 

Second, are you sure it modified the database itself, or do you think it may have changed the codebase and/or Windchill DS?  If it really was the database (and only the database), then the only good solution is probably going to be restoring the database from a previous backup.  Anything created since that backup was taken will need to be removed from the file vault to prevent conflicts going forward.  (Pretty easy to do by timestamp or use the 'remove unreferenced files' command on the vault.)

 

Preventing something like this is why I modify the hosts files on my Windchill test systems and block access to the production database server.  Only the production Windchill system is allowed to talk to the production database server.

TomU
23-Emerald IV
(To:TomU)

I'm sure it wouldn't be supported by PTC, but you could restore the last backup of the database to a separate instance and then run a comparison tool on the two of them to see what changed.  Maybe you can identify some key differences that can be manually undone.  Obviously take a fresh backup of the current database before you attempt to change anything.

BenLoosli
23-Emerald II
(To:TomU)

The current Oracle database is already corrupt and is useless.

BenLoosli
23-Emerald II
(To:TomU)

Since this Windchill system is no longer used in production, the Enterprise Down call is not a viable option.

I discovered the database mods when I tried to use the backup for another upgrade manager run and it came back with errors stating that 12.0.2.4 cannot be upgraded to 12.0.2.6. I worked with Ken at PTC on a call to get to where I am now. He agreed that the database was modified by the attempted upgrade manager run in July when I was trying 12.0.2.4. 

 

Going back to the April backup and deleting added content will be next action. Do I do it on a clean install of 11.0 M030 CPS06 or try to replace the LDAP and Oracle databases on the existing install?

TomU
23-Emerald IV
(To:BenLoosli)

I can understand the upgrade manager potentially talking to the wrong database and even the wrong LDAP, but I don't think it would have modified the codebase (Windchill installation) on a completely different server.  Do you have any evidence that the installation (codebase) was modified?  LDAP changes should be very obvious.  Just take a look at the LDAP entries and see which system they're referencing.  I'd probably just start by restoring the database to the last known good backup and possibly restoring the LDAP if changes are obvious there.  I really doubt you will need to touch the codebase.

BenLoosli
23-Emerald II
(To:TomU)

It is only the Oracle database that is corrupt on the source server. The LDAP is fine. When Windchill goes to start it reports this in the log file: 

Some process has updated the database with a record of versioned products that is newer than the set of version products presently installed for this process. You will need to update this process's installed products before it can use the database.

TomU
23-Emerald IV
(To:BenLoosli)

There's probably one entry in some table that you could change and make it 'happy'.  Not supported of course....

@BenLoosli 

Is the database actually corrupted?

What are you seeing that makes you characterize the dB as “corrupted”?

Can you connect to the dB using sqlplus? If no, what error are you receiving?

 

Are you sure it’s not just a few dB table column values that need to be edited to allow your Windchill system to connect to it?

 

Have you looked at the few dB table column values that should be pointed toward the Windchill server to see if they are still pointing to the correct server?

 

hi @BenLoosli 

 

As @d_graham said check the DB table where is the version of windchill saved. All update and upgrade are stored there and Windchill will not start if there is a wrong last stored row with wrong version.

You can try to remove wrong row or update columns to a correct values and try to run the MethodServer.

 

I would check all WtUpgInst tables

example in MSSQL

select * from WindchillDB.WtUpgInst_Installation
select * from WindchillDB.WtUpgInst_VersionedAssembly
select * from WindchillDB.WtUpgInst_UpgradeTask

 

I had trouble to start a MS and the cs helped me > cs244368

 

PetrH. 

BenLoosli
23-Emerald II
(To:HelesicPetr)

Petr,

From that CS article, I used SQL developer and found the following information:

BenLoosli_0-1661798966626.png

The article seems to be lacking in detailed information as to what exactly to update.

What is the entry for where the update command says: <enter the current id of Windchill version>?

Is it just 11.0 or something else?

ISYPGRADEMANAGERENTRY is already set to 1 for the 2 12 versions listed and 1 11 version.

If the ID is the value in the ID column, which doer I set since 2,3 & 4 are already set to 1?

 

BenLoosli
23-Emerald II
(To:d_graham)

UpgradeManager is flagging the error when doing the initial verify. The source system when I do a Windchill Version shows 11.0, which it should. The UpgradeManager with 12.0.2.4 was run back in June or July, but the system kept running fine until IT rebooted it in August. My backup data was taken on July 28th and that is the data that I am using for the upgrade import.
Source Version: ODATA.2.3.10.00.04,PJL.12.0.20.04.38,WSP.12.0.20.04.38,PDML.12.0.20.04.38,IE.12.0.20.04.38,WNC.12.0.20.04.38,COMMONPDM.12.0.20.04.38
Target Version: ODATA.2.3.20.00.02,PJL.12.0.20.06.31,WSP.12.0.20.06.31,PDML.12.0.20.06.31,IE.12.0.20.06.31,WNC.12.0.20.06.31,COMMONPDM.12.0.20.06.31

After the IT reboot, I see this in my log files and the system starts, crashes and then restarts, over and over.
Some process has updated the database with a record of versioned products that is newer than the set of version products presently installed for this process. You will need to update this process's installed products before it can use the database.

If there is some table entry that I can try, that would be great.

@BenLoosli 

 

The data you show is in table WTUPGINST_VERSIONEDASSEMBLY column RELEASEID.

If you take a look at the data in that table (there are only three columns so pretty simple) sort it by ID and you'll essentially see what is displayed in a Windchill shell when you run windchill version.  What's displayed is the RELEASEID where INSTALLATIONID has the largest number.  Meaning, the initial Windchill install is INSTALLATIONID = 1. Next CSP upgrade is INSTSALLATIONID = 2.  You get the idea.

 

Take a look at that table and see if anything rings a bell.

 

David

mmeadows-3
13-Aquamarine
(To:d_graham)

@BenLoosli 

Just another way to do the same thing...

 

The XML export of this command will also show you a structured view where each set of updates are collected by each run of the PSI/Upgrade Manager, including date.  The dates can help you get to the modules that might have changed.

UpgradeManager -exportdbhistory dbHistory.xml

 

You can modify the dbHistory.xml file (copy for backup first) and import it to update these tables.  Since it is a command from PTC, it can be trusted to make the appropriate changes.  The copy ensures you can roll back if it doesn't work.  It is also database agnostic.

UpgradeManager -importdbhistory dbHistory_mod.xml

 

 

avillanueva
22-Sapphire I
(To:BenLoosli)

@BenLoosli oof, I know that feeling of dread when I am in these situations. I would love to see a post mortem summary when you get this sorted out as to how this happened.  I cannot assist with as this is where I call for help too.  I suspect the the db.properties or perhaps the upgrade manager entries were pointing to the wrong DB host. Is that what happened? I hope you are doing well. BTW, how many Windchill instances do you have running?

BenLoosli
23-Emerald II
(To:avillanueva)

Thinks are going well all things considered. I have 3 Windchill systems running different builds that we are trying  to get to Windchill 12.0.2.x.

 

mmeadows-3
13-Aquamarine
(To:BenLoosli)

The Upgrade Manager's DB history includes what was changed and on what dates back to the initial installation of Windchill. You can use it to track down the installed component numbers for all modules..
UpgradeManager -exportdbhistory dbHistory.xml

Note: We can also edit the file and re-load it. It may be easier than hacking database entries. However, I would make a backup of your currently broken database before doing this.

The following comes from my upgrade notes. It is not your current issue but the technique may prove useful.

Uninstall Modules During Rehosting
When we request the database history, the command exports an XML file with the full update/upgrade history of the system. This content is stored in database tables that start with ‘WtUpgInst_*’.
select owner, table_name from dba_tables where table_name like '%WTUPGINST_%';
We can query the database to get a list of all installed modules. (SQL Server)
select * from INFORMATION_SCHEMA.TABLES where TABLE_NAME LIKE '%VersionedAssembly%';
select * from WtUpgInst_VersionedAssembly;
And if we know the offending module(s), we can query for them directly.
select * from WtUpgInst_VersionedAssembly where releaseId like '%infomodeler%';
PTC does not recommend changing any of this information through database queries. Uninstallation of any module is not supported, so don’t bother logging a PTC Tech Supt call for this. However, sometimes there are installed modules that we don’t have or won’t seriously impact Windchill: PSM, PTC’s development environment, infomodeler, etc. When we encounter these modules, first check the knowledge base to see if there are explicit uninstall instructions for the module. Search for ‘uninstall’ and ‘module’ and the name of the module. Several modules have explicit instructions: MPMLink, SOLR, COGNOS, etc. If you can’t find instructions, follow these steps.
1. Ensure WindchillDS is running and run the UpgradeManager command from a Windchill shell:
UpgradeManager -exportdbhistory dbHistory.xml
This will export the DB history to the location where the command was executed (usually %wt_home%).
2. Copy the file to a new name.
copy /y %wt_home%\dbHistory.xml %wt_home%\dbHistory_mod.xml
3. Edit the file and delete all rows of the undesired entries.
%wt_home%\dbHistory_mod.xml
4. Import the modified history file.
UpgradeManager -importdbhistory dbHistory_mod.xml
5. Using a database query (as the database user), confirm module no longer exists.
select * from WtUpgInst_VersionedAssembly where releaseId like '%infomodeler%';
6. Start Apache and run the update_tool to clean up any issues.
bin\update_tool.bat -username wcadmin -password wcadmin -noui
windchill version
The installed version information should no longer include the offending module.

7. Now clean cache and restart Windchill.


Top Tags