Looking for something? Try here..

Friday, September 1, 2023

Oracle Enterprise Manager OMS OPatch upgrade mess up

 I was trying to patch my lab environment Oracle Enterprise Manager (OEM) OMS to the latest version. While doing so, we have to update the OMSPatcher and OPatch utility both to the latest version available. 

After the update to both OMSPatcher and OPatch has been done, I tried running the analyze command where I got a weird error thrown at me as below. 

[oracle@linux-8 35174911]$ omspatcher apply -analyze
java.lang.NoClassDefFoundError: oracle/oplan/sdk/intg/OPlanIntgFactory
        at oracle.opatchauto.oms.OMSPatchingSession.<clinit>(OMSPatchingSession.java:75)
        at oracle.opatchauto.oms.OPatchAuto.<clinit>(OPatchAuto.java:94)
Caused by: java.lang.ClassNotFoundException: oracle.oplan.sdk.intg.OPlanIntgFactory
        at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:355)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
        ... 2 more

OMSPatcher failed with error code 255
[oracle@linux-8 35174911]$ 
Now, the issue might have been caused either by omspatcher update or opatch update. The key is the first line of the error (java.lang.NoClassDefFoundError) for which there are multiple hits on support note all mentioning to restore the entire middleware home to fix this mess. 

I have followed the Oracle support note Doc ID 2646080.1 and Doc ID 2728285.1 to patch OMSPatcher and OPatch utility as a pre step and how did it mess up the software? 

So, here comes the twist. Support note EM 13.4: OMSPatcher Commands Failing With Error "java.lang.NoClassDefFoundError: oracle/oplan/sdk/intg/OPlanIntgFactory" (Doc ID 2749759.1) mentions the cause could be due to renaming the OPatch directory prior to OPatch upgrade. 

So scrolling back the screen, I found that I have moved the OPatch directory to /home/oracle since this is my practice of backing up the OPatch directory for all the database patching that we do on regular basis. So in that same thought, I used mv command to backup the OPatch directory. 

The solution to the issue is to rsync the files from old OPatch directory to new OPatch directory 
[oracle@linux-8 35174911]$ which rsync
/usr/bin/rsync
[oracle@linux-8 35174911]$ rsync -a -v --ignore-existing /home/oracle/OPatch/ /oem/oracle/middleware/OPatch/
sending incremental file list
./
opatchauto
auto/
auto/core/
auto/core/bin/
auto/core/bin/opatchauto.sh
auto/core/bin/opatchautoCopy.sh
auto/core/bin/opatchautoEnv.sh
auto/core/bin/oplan.sh
auto/core/bin/patchingWallet.sh
auto/core/modules/
auto/core/modules/com.oracle.glcm.oplancore.core_13.9.4.2.jar
auto/core/modules/com.oracle.glcm.osyscore.automation_13.9.1.0.jar
...
...
...
plugins/maven/com/oracle/oracle.glcm.oplan.core/13.9.4/oracle.glcm.oplan.core-13.9.4.pom
plugins/opatchauto/
plugins/opatchauto/modules/
plugins/opatchauto/modules/oracle.glcm.opatchauto.fmw.actions.classpath.jar
scripts/

sent 1,848,369 bytes  received 1,158 bytes  3,699,054.00 bytes/sec
total size is 5,708,468  speedup is 3.09
[oracle@linux-8 35174911]$
Once this step is done, the analyze command worked as expected. 

[oracle@linux-8 35174911]$ omspatcher apply -analyze
OMSPatcher Automation Tool
Copyright (c) 2017, Oracle Corporation.  All rights reserved.


OMSPatcher version : 13.9.4.8.0
OUI version        : 13.9.4.0.0
Running from       : /oem/oracle/middleware
Log file location  : /oem/oracle/middleware/cfgtoollogs/omspatcher/opatch2023-09-01_01-02-52AM_1.log

OMSPatcher log file: /oem/oracle/middleware/cfgtoollogs/omspatcher/35174911/omspatcher_2023-09-01_01-02-59AM_analyze.log

Please enter OMS weblogic admin server URL(t3s://linux-8.selvapc.com:7102):>
 
Take away: 

In the patching document, the below entry is clearly presented as a note 

So it was clearly my mistake to overlook the instructions provided causing the issue. 
*** Always prepare a step by step document with clear instructions before attempting the deployment ***

Since this is my test lab, I'm ok to play with the software. 
*** Never ever attempt any deployments straight on to production database/environment even when you have an emergency situation. Have a test system ready, deploy and mimic the steps which worked perfect on the production ***
Since I have done OPatch upgrade multiple times just by replacing the OPatch directory and unzipping the patch, I have been overconfident during this step causing the mess. 
*** Never blindly do anything without looking at the instructions even though you have done it a 1000 times. There is always a chance of change or improvement ***

References: 

EM 13c: How To Upgrade Enterprise Manager 13.4 Cloud Control OMSPatcher Utility to the Latest Version (Doc ID 2646080.1)EM 13.4: OMSPatcher Commands Failing With Error "java.lang.NoClassDefFoundError: oracle/oplan/sdk/intg/OPlanIntgFactory" (Doc ID 2749759.1)


Happy OPatch upgrading...!!!

1 comment: