Showing posts with label How-To's. Show all posts
Showing posts with label How-To's. Show all posts

Wednesday, December 17, 2025

Playing with MCP Server for Oracle Database - SQLcl MCP Server & Claude Desktop client

I was trying to connect to my database using SQLcl MCP Server/protocol using Claude desktop so I can make use of Claude AI to get my work done on the database.
To know more on what is MCP protocol, MCP server/client, setting up Claude desktop on how to connect to the database, etc., I would suggest you to go through the below posts and documentation.

https://blogs.oracle.com/database/introducing-mcp-server-for-oracle-database
https://www.thatjeffsmith.com/archive/2025/07/getting-started-with-our-mcp-server-for-oracle-database/
https://docs.oracle.com/en/database/oracle/sql-developer-command-line/25.2/sqcug/using-oracle-sqlcl-mcp-server.html

There are many MCP clients we can use such as Claude desktop, VS code, Copilot, Cursor, etc. 
See https://modelcontextprotocol.io/clients for available clients at this time.

I chose Claude desktop since I already use Claude and configuration seems a bit easy at the first go. 

Make sure you have SQLcl version 25.2 and above. Else you will get the below error

2025-12-13T01:33:00.679Z [error] [sqlcl] Unexpected end of JSON input
2025-12-13T01:33:00.680Z [error] [sqlcl] Unexpected token 'B', "Bad Option: -mcp" is not valid JSON
MCP Server support is introduced only from SQLcl version 25.2.0.

My configuration file looks like below:

  {
  "mcpServers": {
    "sqlcl": {
      "command": "D:\\Oracle\\sqlcl\\bin\\sql",
      "args": ["-mcp"]
    }
   }
  }

See that we need \\ since I'm on Windows for the json escaping rules or you can simply use / in the path separator. Choice is yours.

My saved connection in SQLcl is configured as below 

SQL> conn -save cline_mcp -savepwd moviestream/oracle@192.168.56.126:1521/freepdb1
Name: cline_mcp
Connect String: 192.168.56.126:1521/freepdb1
User: moviestream
Password: ******
Connected.
SQL>
SQL> cm show cline_mcp
Name: cline_mcp
Connect String: 192.168.56.126:1521/freepdb1
User: moviestream
Password: ******
SQL>
SQL> cm test cline_mcp
Oracle AI Database 26ai Free Release 23.26.0.0.0 - Develop, Learn, and Run for Free
Connection Test Successful
SQL>

All is well until now. Then started the actual issue when I try to connect and get some details from the database. 

 
I had very limited roles/privileges (CONNECT, RESOURCE, CREATE TABLE) granted to MOVIESTREAM user so that we don't expose unnecessary data to the LLM than what is required.
You can see from the above, the client is successfully connecting to the database and we have the following as a response. 

 

Response

### DATABASE CONNECTION ESTABLISHED ###
Successfully connected to: **cline_mcp**
Response


### DATABASE ENVIRONMENT CONTEXT ###
Response

**1. Oracle Database Version:** 
23.26.0.0.0
Response

**2. Access Mode:** 
The Oracle database is currently in 'null' mode
Response

**3. NLS Parameters:** 
The current Oracle database uses the following NLS configuration:
Database character set:
Value is not valid for option json.   Remaining options are ignored.

Value is not valid for option parameter,.   Remaining options are ignored.

 Full NLS parameters: 

Value is not valid for option json.   Remaining options are ignored.

Value is not valid for option *.   Remaining options are ignored.

 This is important for correctly interpreting data formats, encodings, and locale-specific behavior.
Response

Here’s the Oracle database context you need: version, open mode, and NLS parameters. Use this information to run the following queries accurately and ensure they align with the database’s current environment and settings.
 

From the above response, Claude got the Access Mode as 'null' which is expected as the connect runs query against v$database to get the open mode of the database. User don't have privilege. 

NLS Parameters is also messed up in the response.
The client actually runs the below queries against database once the connection is successful to get the NLS related data.

SELECT * FROM nls_database_parameters
SELECT parameter, value FROM nls_session_parameters

These queries are not getting registered in the DBTOOLS$MCP_LOG table. 

DBTOOLS$MCP_LOG table keeps track of all the queries executed by the MCP client for future auditing. The table just shows a log_message entry as "Connect to MOVIESTREAM" and not the actual queries run against database. Reason being the queries are automatically executed by SQLcl client with the connect command used to connect the user session with the database. 

In this case, after the initial connection both the queries failed to execute and subsequent chat were all just error responses as shown in the Claude chat above.

The client will have its MCP logs by name mcp-server-sqlcl.log which has the following repeated messages 

2025-12-15T01:55:13.423Z [sqlcl] [info] Message from server: {"jsonrpc":"2.0","id":12,"result":{"content":[{"type":"text","text":"### DATABASE CONNECTION ESTABLISHED ###\nSuccessfully connected to: **cline_mcp**"},{"type":"text","text":"\n### DATABASE ENVIRONMENT CONTEXT ###\n"},{"type":"text","text":"**
1. Oracle Database Version:** \n23.26.0.0.0"},{"type":"text","text":"**
2. Access Mode:** \nThe Oracle database is currently in 'null' mode"},{"type":"text","text":"**
3. NLS Parameters:** 
The current Oracle database uses the following NLS configuration:
Database character set:
Value is not valid for option json.   Remaining options are ignored.
Value is not valid for option parameter,.   Remaining options are ignored.
Full NLS parameters: 
Value is not valid for option json.   Remaining options are ignored.
Value is not valid for option *.   Remaining options are ignored.
This is important for correctly interpreting data formats, encodings, and locale-specific behavior.\n"},{"type":"text","text":"Here’s the Oracle database context you need: version, open mode, and NLS parameters. Use this information to run the following queries accurately and ensure they align with the database’s current environment and settings."}],"isError":false}} { metadata: undefined }

..
..

2025-12-15T02:13:54.531Z [sqlcl] [info] Message from server: {"jsonrpc":"2.0","id":17,"result":{"content":[{"type":"text","text":"\nValue is not valid for option LLM.   Remaining options are ignored.\n"}],"isError":false}} { metadata: undefined }

..
..

2025-12-15T02:18:55.109Z [sqlcl] [info] Message from server: {"jsonrpc":"2.0","id":21,"result":{"content":[{"type":"text","text":"### DATABASE CONNECTION ESTABLISHED ###\nSuccessfully connected to: **cline_mcp**"},{"type":"text","text":"\n### DATABASE ENVIRONMENT CONTEXT ###\n"},{"type":"text","text":"**
1. Oracle Database Version:** \n23.26.0.0.0"},{"type":"text","text":"**
2. Access Mode:** \nThe Oracle database is currently in 'READ WRITE' mode"},{"type":"text","text":"**
3. NLS Parameters:** \nThe current Oracle database uses the following NLS configuration:\nDatabase character set:\nValue is not valid for option json.   Remaining options are ignored.\n\nValue is not valid for option parameter,.   Remaining options are ignored.\n\n Full NLS parameters: \n\nValue is not valid for option json.   Remaining options are ignored.\n\nValue is not valid for option *.   Remaining options are ignored.\n\n This is important for correctly interpreting data formats, encodings, and locale-specific behavior.\n"},{"type":"text","text":"Here’s the Oracle database context you need: version, open mode, and NLS parameters. Use this information to run the following queries accurately and ensure they align with the database’s current environment and settings."}],"isError":false}} { metadata: undefined }
 
I thought if these were errors due to some permission issues against internal tables/views and tried to grant the privileges and even DBA role to check if that makes a difference
grant select on sys.v_$nls_parameters to moviestream;
grant select on sys.v_$database to moviestream;
grant dba to moviestream;

Result is the same errors. These additional privileges/roles are not needed for MCP to work properly. 
Interestingly, the queries which are run against the database is being captured in the DBTOOLS$MCP_LOG table under MOVIESTREAM schema. 

Below is a screenshot of the tables content for basic understanding


The query passed to database by the Claude client is as below

SELECT /* LLM in use is claude-sonnet-4-5 */ table_name FROM user_tables;

All the queries will be added a comment with what LLM is being used when it's executed by the MCP client and the query is executed. The query execution resulted in the error below

Value is not valid for option LLM. Remaining options are ignored.

From logs what we have is the below 
2025-12-14T01:21:23.569Z [sqlcl] [info] Message from client: {"method":"tools/call","params":{"name":"run-sql","arguments":{"model":"claude-sonnet-4-5-20250929","sql":"SELECT table_name FROM user_tables"}},"jsonrpc":"2.0","id":10} { metadata: undefined }
2025-12-14T01:21:23.599Z [sqlcl] [info] Message from server: {"jsonrpc":"2.0","id":10,"result":{"content":[{"type":"text","text":"\nValue is not valid for option LLM.   Remaining options are ignored.\n"}],"isError":false}} { metadata: undefined }
 

Basically, client sends a request with a few args and MCP server (SQLcl) sends back a error response.

Doesn't make sense on why SQLcl send error response on argument (LLM) which was not passed to it. 

After a long time trying to debug the issue by different prompts from the client, executing query directly in the SQLcl prompt with/without comments, etc., none of which resolved the issue. 

I just deleted the sqlcl directory and uninstalled Java and downloaded them both again using the same way I used to do the first time and installed Java and started the SQLcl and reconfigured Claude desktop.

This time everything started to work without any issues as expected.


I'm not sure what caused the issue but with this issue, I was able to get a proper hands-on with the MCP Server, it's working and debugging methods. Hope this post will help others as well. 



References:

https://blogs.oracle.com/database/introducing-mcp-server-for-oracle-database
https://www.thatjeffsmith.com/archive/2025/07/getting-started-with-our-mcp-server-for-oracle-database/
https://docs.oracle.com/en/database/oracle/sql-developer-command-line/25.2/sqcug/using-oracle-sqlcl-mcp-server.html


 Happy playing with AI...!!! 

 

 

Tuesday, February 13, 2024

Connection to Oracle Autonomous Database

This article explains how to connect to an Oracle Autonomous database using SQL developer, SQL plus or other tools.

When we create an Oracle ADB on OCI, we just can't connect directly to the database even though the Network ACL is disabled. The reason is mTLS authentication is set to Required by default. 

Let us see how to connect to the ADB using SQL developer first and then SQL plus.

If you have downloaded SQL developer for connecting to your database on cloud, it's super easy to connect.

Method 1: Using mTLS  

Mutual TLS (mTLS) is a 2-way handshake between a client and a server, and a more secure way of connection compared to TLS.

Step 1: Download the wallet (This is required for mTLS)

Go to your Autonomous database details page => Click on Database connection => Click on Download wallet



Provide the password and click on Download. This is your wallet password and not the password for the database Admin user or the database application user you are trying to connect. 

Step 2: Once you have this credentials wallet zip, Open SQL Developer and edit the connection properties to connect to the Autonomous database as below



Make sure we have the connection type selected as Cloud Wallet and provide the downloaded zip file under configuration file. 
While selecting Service, we have different default service to choose from which differs on how the connection works. I have selected atp01_high of the below 5 available.
  • atp01_high
  • atp01_low
  • atp01_medium
  • atp01_tp
  • atp01_tpurgent
Click on the Test button to check the connectivity and we now have a Successful connection. 

How do we connect to using SQL Plus? 

To make a SQL Plus or any Oracle Call Interface (OCI), JDBC connection to the Autonomous database, we need the client software to be installed in the server from where we are trying to connect. 
Since we need the wallet location to be specified, we should have sqlnet.ora file and tnsnames.ora file

I'll use my existing on-prem Linux server to attempt a connection without the wallet file location specified in the sqlnet.ora but with only the tnsnames.ora edited with the connection details of the ADB. 
[oracle@linux-8 db-sample-schemas-main]$ cd order_entry/
[oracle@linux-8 order_entry]$ sqlplus admin/xxxxx@'(description= (retry_count=20)(retry_delay=3)(address=(protocol=tcps)(port=1522)
(host=adb.us-ashburn-1.oraclecloud.com))(connect_data=(service_name=g34efa58d1e9751_atp01_high.adb.oraclecloud.com))
(security=(ssl_server_dn_match=yes)))'

SQL*Plus: Release 19.0.0.0.0 - Production on Wed Feb 7 03:56:14 2024
Version 19.20.0.0.0

Copyright (c) 1982, 2022, Oracle.  All rights reserved.

ERROR:
ORA-12506: TNS:listener rejected connection based on service ACL filtering


Enter user-name:
 
So where do I get the connection description information from? 
When I click on the Database connection button on the Database details page, I'll get the connection strings for all the 5 services or the downloaded zip will include the tnsnames.ora file containing this information. 

As you see when we didn't provide the wallet location, we are not able to connect with error ORA-12506. 

There are 2 things you can do to fix this. 
Easy way is to unzip the contents of the downloaded zip file to a directory on the server and define the TNS_ADMIN environment variable value to this directory and attempt the connection. Make sure to make changes in the sqlnet.ora file to denote the wallet location
[oracle@linux-8 db-sample-schemas-main]$ ls -lrt
total 28
-rwxrwx---. 1 root vboxsf 21984 Jan  9 21:17 Wallet_ATP01.zip
drwxrwx---. 1 root vboxsf  4096 Feb  7 04:00 db-sample-schemas-main
[oracle@linux-8 db-sample-schemas-main]$ unzip Wallet_ATP01.zip -d ./Wallet_ATP01
Archive:  Wallet_ATP01.zip
  inflating: ./Wallet_ATP01/ewallet.pem
  inflating: ./Wallet_ATP01/README
  inflating: ./Wallet_ATP01/cwallet.sso
  inflating: ./Wallet_ATP01/tnsnames.ora
  inflating: ./Wallet_ATP01/truststore.jks
  inflating: ./Wallet_ATP01/ojdbc.properties
  inflating: ./Wallet_ATP01/sqlnet.ora
  inflating: ./Wallet_ATP01/ewallet.p12
  inflating: ./Wallet_ATP01/keystore.jks
[oracle@linux-8 db-sample-schemas-main]$ cd Wallet_ATP01/
[oracle@linux-8 Wallet_ATP01]$ vi sqlnet.ora
[oracle@linux-8 Wallet_ATP01]$ more sqlnet.ora
WALLET_LOCATION = (SOURCE = (METHOD = file) (METHOD_DATA = (DIRECTORY="?/network/admin")))
SSL_SERVER_DN_MATCH=yes
[oracle@linux-8 Wallet_ATP01]$ vi sqlnet.ora
[oracle@linux-8 Wallet_ATP01]$ more sqlnet.ora
WALLET_LOCATION = (SOURCE = (METHOD = file) (METHOD_DATA = (DIRECTORY="/media/sf_Oracle/software/db-sample-schemas-main/Wallet_ATP01")))
SSL_SERVER_DN_MATCH=yes
[oracle@linux-8 Wallet_ATP01]$ export TNS_ADMIN=/media/sf_Oracle/software/db-sample-schemas-main/Wallet_ATP01
[oracle@linux-8 Wallet_ATP01]$ sqlplus admin/xxxxx@atp01_high

SQL*Plus: Release 19.0.0.0.0 - Production on Sat Feb 10 05:02:32 2024
Version 19.20.0.0.0

Copyright (c) 1982, 2022, Oracle.  All rights reserved.

Last Successful login time: Sat Feb 10 2024 04:11:47 +05:30

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.22.0.1.0

SQL> select instance_name from v$instance;

INSTANCE_NAME
----------------
feky1pod2

SQL>

 
I just used atp01_high as my connect identifier as they are already defined in the tnsnames.ora file thats bundled in the zip. 

If you already have a working TNS_ADMIN (which defaults to $ORACLE_HOME/network/admin), all you have to do is to only mention the below 2 parameters in the existing sqlnet.ora file and add the connection string for the ADB connection in the tnsnames.ora file. 
WALLET_LOCATION = (SOURCE = (METHOD = file) (METHOD_DATA = (DIRECTORY="/media/sf_Oracle/software/db-sample-schemas-main/Wallet_ATP01")))
SSL_SERVER_DN_MATCH=yes

Method 2: Using TLS connection.

Now let's see how to connect to the Autonomous database without the wallet files using a TLS connection. 

We can connect to the autonomous database without the wallet files as well. If we wan't to allow TLS connection to connect to Autonomous database, we need to enable Access Control List (ACL). To do that, we can click on edit adjacent to the Disabled status of Access Control List

We can configure different type of ACLs such as with specific IPs, CIDR block, VCNs or VCN OCID and the combination of these can be used in real environments. 

For our testing purpose, I'm just going to add my ip address to show the ACL and TLS connection. 

Just click on Add my IP address and click Save. This will add our IP address to the ACL and the status of the ACL will turn as Enabled. 
Now click on Edit adjacent to mTLS: Required

Uncheck the Require mTLS authentication check box to disable mTLS


Now, we have enabled TLS authentication to the Autonomous database but only the connections from my IP is allowed since we have also enabled ACLs. 

Now, as we changed the configuration, lets try to connect to the ADB without the wallet files to the database using SQL Developer first and then using SQL Plus. 

The configuration will look like the below. 

We have now selected Custom JDBC instead of Cloud wallet since we are not using TLS connection for the database. 
For the JDBC URL, the format should be 
jdbc:oracle:thin:@<connection string for the service intended> 

In my example, I'm trying to connect to atp01_high service and hence the JDBC URL will be as below. 
jdbc:oracle:thin:@(description= (retry_count=20)(retry_delay=3)(address=(protocol=tcps)
(port=1522)(host=adb.us-ashburn-1.oraclecloud.com))
(connect_data=(service_name=g34efa58d1e9750_atp01_high.adb.oraclecloud.com))
(security=(ssl_server_dn_match=yes)))
Click on Test to see the connection status is Success. 

If we need to connect using SQL Plus, we can save this connect string in the tnsnames.ora file and connect directly. 
[oracle@linux-8 admin]$ pwd
/oracle/db/db19/network/admin
[oracle@linux-8 admin]$ ls -lrt sqlnet.ora tnsnames.ora
-rw-r-----. 1 oracle oinstall 1629 Feb  7 04:33 tnsnames.ora
-rw-r--r--. 1 oracle oinstall  619 Feb 10 05:16 sqlnet.ora
[oracle@linux-8 admin]$ rm sqlnet.ora
[oracle@linux-8 admin]$ tail tnsnames.ora
rmancat =
  (DESCRIPTION =
    (ADDRESS = (PROTOCOL = TCP)(HOST = linux-8.selvapc.com)(PORT = 1521))
    (CONNECT_DATA =
      (SERVER = DEDICATED)
      (SERVICE_NAME = rmancat.selvapc.com)
    )
  )

atp = (description= (retry_count=20)(retry_delay=3)(address=(protocol=tcps)(port=1522)(host=adb.us-ashburn-1.oraclecloud.com))(connect_data=(service_name=g34efa58d1e9751_atp01_high.adb.oraclecloud.com))(security=(ssl_server_dn_match=yes)))
[oracle@linux-8 admin]$
[oracle@linux-8 admin]$ tnsping atp

TNS Ping Utility for Linux: Version 19.0.0.0.0 - Production on 13-FEB-2024 10:18:22

Copyright (c) 1997, 2023, Oracle.  All rights reserved.

Used parameter files:


Used TNSNAMES adapter to resolve the alias
Attempting to contact (description= (retry_count=20)(retry_delay=3)(address=(protocol=tcps)(port=1522)(host=adb.us-ashburn-1.oraclecloud.com))(connect_data=(service_name=g34efa58d1e9750_atp01_high.adb.oraclecloud.com))(security=(ssl_server_dn_match=yes)))
OK (380 msec)
[oracle@linux-8 admin]$
[oracle@linux-8 admin]$ sqlplus admin/xxxxx@atp

SQL*Plus: Release 19.0.0.0.0 - Production on Tue Feb 13 10:18:37 2024
Version 19.20.0.0.0

Copyright (c) 1982, 2022, Oracle.  All rights reserved.

Last Successful login time: Mon Feb 12 2024 11:49:25 +05:30

Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.22.0.1.0

SQL>
  
In the above demo, I just removed my tnsnames.ora file meaning we don't pass any location for the wallet files which is confirmed by blank o/p for Used parameter files of the tnsping. 
We are now able to connect to the autonomous database using TLS but only from my specific IP address. 
Any connection attempt from other addresses will not be allowed as per the ACL. 

Requirements of TLS connection: 

SQL Developer - version 18.2 or later
SQL Plus - Oracle instant client/database client 19.14 or later and 21.5 or later (if you need it on Windows)

References: 

Connect to Autonomous Database Using Oracle Database Tools

Happy Connecting...!!!

Tuesday, November 7, 2023

Oracle Enterprise Manager - undeploy deprecated plug-ins

 In my previous post, I explain on how to upgrade Oracle Enterprise Manager (OEM) from 13.4 to 13.5. In my lab environment, I have not installed any additional plug-ins. But this not the case when we did the same in live production environment. We had one plug-in which required to be migrated instead of upgraded as below. 


This is clearly indicated when we use GUI to perform the upgrade. If you are using command line to upgrade the OEM, then these prechecks are written to a file which gets buried in the huge number of lines as we tend to check only for any errors or warnings. This plug-in migration doesn't pop out as a warning or an error in the logs. 

The first attempt to upgrade resulted in OMS not starting up with incompatible plug-in version from OMS. The error matches with the symptoms indicated in support note EM 13c: Upgrade Fails At OMS startup With Error 'Plugin's Repository version is not compatible with Plugin's OMS version' (Doc ID 2491277.1)

Solution to the issue was to restore the old environment (gives you the reason why backing up the old home is very important), undeploy the deprecated plug-in on all agents and OMS and then perform the upgrade again. 

So where do we find which plug-ins are deprecated and how to undeploy them? 

You hit the following 2 support notes 

The check list note 13.5: Checklist for Upgrading Enterprise Manager Cloud Control from Version 13.3/13.4 to 13.5 (Doc ID 2761728.1) which talks about the obsolete plugins of 13.5 

The following plug-ins are obsolete and support for these plug-ins is discontinued. These plug-ins must be undeployed from the Oracle Management Agents and Oracle Management Services before proceeding with the upgrade.

  • Cloud Services Management (oracle.sysman.csm)
  • Microsoft .NET Framework (oracle.em.smdn)
  • Microsoft Active Directory (oracle.em.smad)

This note doesn't talk regarding the plug-in which was indicated in our prechecks or review page. 

The second note Oracle announces deprecation of Enterprise Manager 13c ORAchk Healthchecks Plug-in (Doc ID 2935520.1) gives exactly what we are searching for. 

So, we have the dedicated note for ORAchk Healthchecks plugin is deprecated in version 13.5. 

How do we check what other plugins are installed and whether we are good to proceed with the upgrade? 

Click on Setup => Plug-ins


This will list out all available plug-ins. 

Click on View => Only Deployed

This shows us all the deployed plug-ins in our environment. 


How to undeploy the plug-in? 

Click on the plug-in we need to undeploy => Undeploy From => Management servers... 


Since this plug-in is not installed in the Management Agents, the option is grayed out and only Management Servers is available to select. 


Input Sys Password and click Continue


Check the backup question and click Undeploy

If you haven't backed up the configuration, it can be done as below


Once we click on Undeploy, the plug-in will be undeloyed from Management server and we get the confirmation.


Progress can be seen by issuing emctl status oms -detail command


 Once completed click Close

Confirm the Deployed plug-ins 

We have successfully undeployed the deprecated plug-in and upgrade can be done without any issues. 

References: 

EM 13c: Upgrade Fails At OMS startup With Error 'Plugin's Repository version is not compatible with Plugin's OMS version' (Doc ID 2491277.1)
Undeploying Plug-Ins (oracle.com)

Happy Undeploying...!!! 

Wednesday, October 18, 2023

Oracle Enterprise Manager - 13.4 to 13.5 - Post upgrade

 In my previous post, we went through the steps of how to upgrade from OEM 13.4 to 13.5. This is the continuation of the post with all the post upgrade steps. Let's dive into action.. 

Note: If pictures look blurry in the post, please click on them to see the clear picture.

Post Upgrade: 

Modify the .bashrc or .profile file to reflect the new upgraded OMS home. We did an out of place upgrade and hence the path will be changing. Reflect the changes in the profile file to make our life easier. 

OMS will be brought up after the upgrade automatically once upgrade is done. Check for status of the OMS and EM KEY
[oracle@linux-8 ~]$ . ./.bashrc
[oracle@linux-8 ~]$
[oracle@linux-8 ~]$ echo $OMS_HOME
/oem/oem13.5
[oracle@linux-8 ~]$ emctl status oms
Oracle Enterprise Manager Cloud Control 13c Release 5
Copyright (c) 1996, 2021 Oracle Corporation.  All rights reserved.
WebTier is Up
Oracle Management Server is Up
JVMD Engine is Up
[oracle@linux-8 ~]$ emctl status emkey
Oracle Enterprise Manager Cloud Control 13c Release 5
Copyright (c) 1996, 2021 Oracle Corporation.  All rights reserved.
Enter Enterprise Manager Root (SYSMAN) Password :
The EMKey is configured properly.
[oracle@linux-8 ~]$
Open the OEM console and check the version details.



Agent Checks:

Start the OEM agent which was stopped prior to upgrade and check whether all agents pushing data without issues and all targets status updated properly.
[oracle@linux-8 ~]$ /oem/oracle/agent/agent_13.4.0.0.0/bin/emctl start agent
Oracle Enterprise Manager Cloud Control 13c Release 4
Copyright (c) 1996, 2020 Oracle Corporation.  All rights reserved.
Starting agent ................... started.
[oracle@linux-8 ~]$
OEM Summary page will show the details of the targets at a glance. 


All target status uploaded without issues. Upgrade is completed successfully.

Next step will be to upgrade all the agents to 13.5.

Agent Patching: 

Prerequisites: 

1. Make sure Management agent is up and running, secured and not in black out state.

Check whether you have agent software available and applied in the software library. This is important if you want to upgrade agent on platform different than the OMS platform.

Perform following steps to check


Setup => Extensibility => Self Update => Agent Software


Check for Status = Applied

You can see since my OMS and target platforms are Linux x86-64, I'm good here. If you have any other platform such as windows, then you need to download the software and apply it on software library (steps not covered here). 

Check whether Agent is Up and Running and Secured

Setup => Manage Cloud control => Agents

Status should be Green and up
Secure Upload should be Yes.
If not up, start it up. If not secure, Click Secure to Secure the agent.  



2. Check for Not upgradable agents and fix them

Setup => Manage Cloud Control => Upgrade Agents => Not Upgradable Agents




You might see agents listed or sometimes not. Reason will be mentioned for the same. Fix it.
For the above issue, the solution is to run the below command from agent server

<EMSTATE>/bin/emctl control agent runCollection <TARGET_NAME>:oracle_home oracle_home_config

<EMSTATE> => Management Agent instance directory
<TARGET_NAME> => Management Agent home listed as oracle_home in the <EMSTATE>/sysman/emd/targets.xml file

or

Get the name from “emctl config agent listtargets |grep oracle_home” run from Agent home.

In my case,
<EMSTATE> = /oem/oracle/agent/agent_inst
<TARGET_NAME> = agent13c1_4_linux-8.selvapc.com_668
[oracle@linux-8 ~]$ /oem/oracle/agent/agent_13.4.0.0.0/bin/emctl config agent listtargets |grep oracle_home
[agent13c1_4_linux-8.selvapc.com_668, oracle_home]
[oms13c1_3_linux-8.selvapc.com_796, oracle_home]
[OraDB19Home1_2_linux-8.selvapc.com_5813, oracle_home]
[OraGI19Home1_1_linux-8.selvapc.com_8479, oracle_home]
[oms13c2_5_linux-8.selvapc.com_4905, oracle_home]
[oracle@linux-8 ~]$
[oracle@linux-8 ~]$ /oem/oracle/agent/agent_inst/bin/emctl control agent runCollection agent13c1_4_linux-8.selvapc.com_668:oracle_home oracle_home_config
Oracle Enterprise Manager Cloud Control 13c Release 4
Copyright (c) 1996, 2020 Oracle Corporation.  All rights reserved.
---------------------------------------------------------------
EMD runCollection completed successfully

[oracle@linux-8 ~]$
Now in the console, go to Management Agent Home page => Under Summary, Configuration => Click Oracle Home and Patch Details


Click Refresh Configuration




Re-verify Not Upgradable Agents


Now the issue is cleared. 

3. Other checks

Make sure the Management agent doesn’t come up during the upgrade process as cron or other schedule jobs might be running to bring up agent. Stop them.

Blackout creation is done during upgrade and is optional.

We are now set to upgrade the agent to 13.5

Upgrading Agent:

We can now upgrade the agents to version 13.5. Note this upgrade will only perform the upgrade to base version of the 13.5 agents. Once upgraded, patching to the agents needs to be done to bring it up to the latest RU. 

Setup => Manage Cloud Control => Upgrade Agents


Provide Job Name (or Accept default)
Click Add



Select all agents you want to upgrade and click OK
Fill in additional parameters if needed.  



Pre and post upgrade scripts to be run – provide path with name of script (should be placed in OMS server)
Additional parameters if any.
I leave everything default. (Check next pages on error encountered and fix to overcome)

Click Submit


Click OK
or you can use Override Privileged Credentials and provide credentials if you already have stored them.
I’m good with running root scripts on individual servers.  



Let’s wait for the upgrade to complete.
You can Click on individual agent targets to check on their respective progress as below.

 2 of my agent upgrade failed due to space issues in /tmp. At least 3 GB is required.  


I’ll now reinitiate for the failed ones but with different path for staging as below.  



Now all of my agents has been upgraded to version 13.5.0.0.0

Since we have not used privileged account, I have to run the root.sh script on all the agent servers to complete the agent upgrade.

Showing one server as below

[oracle@ol819-rac2 ~]$ sudo /oracle/agent/agent_13.5.0.0.0/root.sh

Finished product-specific root actions.
/etc exist
[oracle@ol819-rac2 ~]$
Now all the steps to upgrade the agent is completed. Let’s verify!

Setup => Manage Cloud Control => Agents


Click on any agent that you have upgraded of your choice



You can now see the version of the agent in the summary page.

If you would like to view the agent related details using EM CLI, following commands can be useful.

$<ORACLE_HOME>/bin/emcli get_agent_properties -format=csv -agent_name=<agent_host_name>:<agent_port>

$<ORACLE_HOME>/bin/emcli get_targets -format="name:csv" -targets=<agent_host_name>:<agent_port>:oracle_emd -alerts

Example given below to display contents on screen

emcli login -username=sysman
emcli sync
emcli get_agent_properties -agent_name=linux-8.selvapc.com:3872


To check status of all agent targets
emcli get_targets -targets=oracle_emd -alerts


Clean-up of Old Agents: 

Once the agents are updated to latest version, we can now clean up the old agents by performing below

Setup => Manage Cloud Control => Post Upgrade Tasks


Click Post Agent Upgrade Tasks => Add


Select all agents where you want to clean up old homes
Click OK


Click Submit



Click OK
Click Job Name to view progress


Clean up job succeeded and Old agent home is now removed.
Can be verified inside the server as well.

Before cleanup


After cleanup


We have now successfully completed the upgrade of OMS from 13.4 to 13.5 RU 16 and agents from 13.4 to 13.5. 
As I mentioned, agents need to be patched to latest RU. 

Enterprise Manager 13.5 Main Release Update List (Includes Plug-ins) (Doc ID 2760230.2) - We can download the latest available agent patch and apply. 

Happy Upgrading...!!!