Thursday, February 18, 2021

Converting Oracle Linux 7 to RedHat Enterprise Linux 7

RHEL 8 is available now and Oracle has already started certifying their products on RHEL 8. RHEL 7 (7.7) EUS is going to end by August 2021. Companies are slowly planning to upgrade their on-prem environments to RHEL 8 to comply with the standards. 

I have a system which is running Oracle Linux 7.5. I would like to try upgrading the system to RHEL 8 to see whether is it worth performing an in place upgrade of the OS. 

We will be using subscription-manager and will directly connect to customer portal as I don't have RedHat Satellite installed. Free non-prod/single user subscription can be obtained by registering in https://developers.redhat.com 

Note: There is an in-place upgrade strategy from RHEL 6 to RHEL 7 but is not supported by Redhat but RHEL 7 to RHEL 8 in-place upgrade is officially supported by Redhat via Leapp. So to perform this upgrade, I would like to convert my Oracle Linux to Redhat Linux. In this post, we will look at the conversion process of the same. 

Current environment that I'm using: 

[root@linux75 ~]# uname -a
Linux linux75.selvapc.com 4.1.12-112.16.4.el7uek.x86_64 #2 SMP Mon Mar 12 23:57:12 PDT 2018 x86_64 x86_64 x86_64 GNU/Linux
[root@linux75 ~]# cat /etc/os-release
NAME="Oracle Linux Server"
VERSION="7.5"
ID="ol"
VERSION_ID="7.5"
PRETTY_NAME="Oracle Linux Server 7.5"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:oracle:linux:7:5:server"
HOME_URL="https://linux.oracle.com/"
BUG_REPORT_URL="https://bugzilla.oracle.com/"

ORACLE_BUGZILLA_PRODUCT="Oracle Linux 7"
ORACLE_BUGZILLA_PRODUCT_VERSION=7.5
ORACLE_SUPPORT_PRODUCT="Oracle Linux"
ORACLE_SUPPORT_PRODUCT_VERSION=7.5
[root@linux75 ~]#

[oracle@linux75 ~]$ sqlplus / as sysdba

SQL*Plus: Release 12.2.0.1.0 Production on Thu Feb 18 00:11:07 2021

Copyright (c) 1982, 2016, Oracle.  All rights reserved.


Connected to:
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production

SQL> select * from v$version;

BANNER                                                                               CON_ID
-------------------------------------------------------------------------------- ----------
Oracle Database 12c Enterprise Edition Release 12.2.0.1.0 - 64bit Production              0
PL/SQL Release 12.2.0.1.0 - Production                                                    0
CORE    12.2.0.1.0      Production                                                                0
TNS for Linux: Version 12.2.0.1.0 - Production                                            0
NLSRTL Version 12.2.0.1.0 - Production                                                    0

SQL>

As you can see I'm running Oracle database on Oracle Linux and the whole idea is to check whether the database is still operational without any issues post conversion. 

Step 1: Install convert2rhel package

convert2rhel package won't be available in default repos. We need to add EPEL repo and then install the package.
Results truncated for easy understanding.. 

[root@linux75 ~]#
[root@linux75 ~]#
[root@linux75 ~]# yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm
Loaded plugins: langpacks, ulninfo
epel-release-latest-7.noarch.rpm                                                                                                                                  |  15 kB  00:00:00
Examining /var/tmp/yum-root-94f61j/epel-release-latest-7.noarch.rpm: epel-release-7-13.noarch
Marking /var/tmp/yum-root-94f61j/epel-release-latest-7.noarch.rpm to be installed
Resolving Dependencies
...
...
...

Installed:
  epel-release.noarch 0:7-13

Complete!
[root@linux75 ~]# yum install convert2rhel
Loaded plugins: langpacks, ulninfo
epel/x86_64/metalink                                                                                                                                              | 4.7 kB  00:00:00
epel                                                                                                                                                              | 4.7 kB  00:00:00
(1/3): epel/x86_64/group_gz                                                                                                                                       |  95 kB  00:00:01
(2/3): epel/x86_64/updateinfo                                                                                                                                     | 1.0 MB  00:00:03
(3/3): epel/x86_64/primary_db                                                                                                                                     | 6.9 MB  00:00:07
Resolving Dependencies
...
...
...
  Verifying  : pexpect-2.3-11.el7.noarch                                                                                                                                             2/3
  Verifying  : convert2rhel-0.13-1.el7.noarch                                                                                                                                        3/3

Installed:
  convert2rhel.noarch 0:0.13-1.el7

Dependency Installed:
  pexpect.noarch 0:2.3-11.el7                                                             python-dateutil.noarch 0:1.5-7.el7

Complete!
[root@linux75 ~]#
Step 2: Install Red Hat GPG signing key

If you have a Redhat 7 installation media, then you can mount the DVD and use the below command 

# mount /dev/sr0 /dvd
# rpm --import /dvd/RPM-GPG-KEY-redhat-release

Or if you don't have an installation media handy, you can download the file from Redhat portal Product signing keys and copy the same under /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release.

[root@linux75 ~]# cp /media/sf_Oracle/software/RPM-GPG-KEY-redhat-release.txt /etc/pki/rpm-gpg/RPM-GPG-KEY-redhat-release
[root@linux75 ~]#
Step 3: Install Redhat Subscription Management related packages. 

convert2rhel package expects the following packages in the /usr/share/convert2rhel/subscription-manager directory. Download the following packages from Redhat Packages which requires subscription or can be obtained from RHEL 7 installation media

subscription-manager-rhsm-1.24.45-1.el7_9.x86_64.rpm
subscription-manager-1.24.45-1.el7_9.x86_64.rpm
subscription-manager-rhsm-certificates-1.24.45-1.el7_9.x86_64.rpm
python-syspurpose-1.24.45-1.el7_9.x86_64.rpm -- needed as a dependency

We will create the directory, copy the packages and also install the packages as sometime the package installation fails via convert2rhel tool due to dependencies. 
[root@linux75 software]# mkdir /usr/share/convert2rhel/subscription-manager
[root@linux75 software]# ls -lrt subscription-manager-*
-rwxrwx---. 1 root vboxsf  340028 Feb 17 16:04 subscription-manager-rhsm-1.24.45-1.el7_9.x86_64.rpm
-rwxrwx---. 1 root vboxsf 1132252 Feb 17 16:27 subscription-manager-1.24.45-1.el7_9.x86_64.rpm
-rwxrwx---. 1 root vboxsf  246792 Feb 17 16:28 subscription-manager-rhsm-certificates-1.24.45-1.el7_9.x86_64.rpm
[root@linux75 software]# ls -lrt python-syspurpose*
-rwxrwx---. 1 root vboxsf 280436 Feb 17 16:36 python-syspurpose-1.24.45-1.el7_9.x86_64.rpm
[root@linux75 software]# cp subscription-manager-* /usr/share/convert2rhel/subscription-manager/
[root@linux75 software]#
[root@linux75 software]# rpm -ivh python-syspurpose-1.24.45-1.el7_9.x86_64.rpm
warning: python-syspurpose-1.24.45-1.el7_9.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:python-syspurpose-1.24.45-1.el7_9################################# [100%]
[root@linux75 software]# rpm -e rhn-client-tools-2.0.2-21.0.3.el7.noarch --nodeps
[root@linux75 software]# rpm -ivh subscription-manager-rhsm-certificates-1.24.45-1.el7_9.x86_64.rpm
warning: subscription-manager-rhsm-certificates-1.24.45-1.el7_9.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:subscription-manager-rhsm-certifi################################# [100%]
[root@linux75 software]# rpm -ivh subscription-manager-rhsm-1.24.45-1.el7_9.x86_64.rpm
warning: subscription-manager-rhsm-1.24.45-1.el7_9.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:subscription-manager-rhsm-1.24.45################################# [100%]
[root@linux75 software]# rpm -ivh subscription-manager-1.24.45-1.el7_9.x86_64.rpm
warning: subscription-manager-1.24.45-1.el7_9.x86_64.rpm: Header V3 RSA/SHA256 Signature, key ID fd431d51: NOKEY
Preparing...                          ################################# [100%]
Updating / installing...
   1:subscription-manager-1.24.45-1.el################################# [100%]
[root@linux75 software]#
Step 4: Install redhat-release-server package

convert2rhel package requires redhat-release-server package to register the system to either Satellite or subscription account. So we create /usr/share/convert2rhel/redhat-release/Server/ directory and place the redhat-release-server-7.9-5.el7_9.x86_64.rpm which can be downloaded from Redhat portal. 

[root@linux75 software]# mkdir -p /usr/share/convert2rhel/redhat-release/Server/
[root@linux75 software]# cp redhat-release-server-7.9-5.el7_9.x86_64.rpm /usr/share/convert2rhel/redhat-release/Server/
[root@linux75 software]#
Note: All the packages and release server installed are of version 7.9 and hence when converting Oracle Linux to Redhat Enterprise Linux, the version will be upgraded to latest version 7.9. 

Step 5: Start the convert2rhel utility

We are now prepared for converting our system to RHEL. 
If the database is running, stop the database and other applications. 
Run yum clean all to start yum from clean state and start the convert2rhel utility. 
This step will take more time depending on the system resources and internet bandwidth that you have. My system took around 30 minutes to convert. 

I'm using my Developer subscription account to register my system to RHSM and you may use yours. If you are using Redhat satellite, the utility command might differ to register to Satellite. 

  1. Start the utility with RHSM username and password
  2. Accept EULA
  3. Accept system conversion
  4. Accept system conversion
  5. Enter number of chosen subscription
  6. Accept final system conversion dialog post which automatic rollback is not possible. 
  7. Finally reboot

[root@linux75 software]# convert2rhel --username nagulan.selvakumar --password ******
...
...
WARNING - By continuing you accept this EULA.

Continue with the system conversion? [y/n]:y


[02/18/2021 01:57:58] TASK - [Prepare: Gather system information] *******************************
Name:                Oracle Linux Server
OS major version:    7
Architecture:        x86_64
Config filename:     oracle-7-x86_64.cfg
Running the 'rpm -Va' command which can take several minutes. It can be disabled by using the --no-rpm-va option.
...
...
...
rhn-setup-gnome-2.0.2-21.0.3.el7.noarch     N/A       anaconda
rhnsd-5.0.13-10.0.1.el7.x86_64              N/A       anaconda
yum-rhn-plugin-2.0.1-10.0.1.el7.noarch      N/A       anaconda


Continue with the system conversion? [y/n]:y
...
...
...
[02/18/2021 02:00:44] TASK - [Convert: List third-party packages] *******************************
WARNING - Only packages signed by Oracle Linux Server are to be reinstalled. Red Hat support won't be provided for the following third party packages:

Package                         Packager        Repository
-------                         --------        ----------
epel-release-7-13.noarch        Fedora Project  N/A
convert2rhel-0.13-1.el7.noarch  Fedora Project  epel


Continue with the system conversion? [y/n]:y
...
...
...
[02/18/2021 02:01:57] TASK - [Convert: Subscription Manager - Subscribe system] *****************
Building subscription-manager command ...
    ... activation key not found, username and password required
    ... username detected
    ... password detected
Registering system by running subscription-manager command ...

...
...
...
Service Level:       Self-Support
Usage:
Add-ons:
Subscription Type:   Standard
Starts:              01/31/21
Ends:                01/31/22
Entitlement Type:    Physical


Enter number of the chosen subscription:1

Attaching subscription with pool ID 8a85f99a774ac5420177570d75625d4e to the system ...

...
...
...
[02/18/2021 02:06:55] TASK - [Convert: Subscription Manager - Rename repositories] **************
Renamed: /etc/yum.repos.d/epel-testing.repo -> /etc/yum.repos.d/epel-testing.repo.convert2rhel_renamed
Renamed: /etc/yum.repos.d/epel.repo -> /etc/yum.repos.d/epel.repo.convert2rhel_renamed
WARNING - The tool allows rollback of any action until this point.
WARNING - By continuing all further changes on the system will need to be reverted manually by the user, if necessary.

Continue with the system conversion? [y/n]: y



[02/18/2021 02:08:38] TASK - [Convert: Import Red Hat GPG keys] *********************************

[02/18/2021 02:08:38] TASK - [Convert: Prepare kernel] ******************************************
Installing RHEL kernel ...
Loaded plugins: langpacks, product-id, search-disabled-repos, subscription-
              : manager
...
...
...

[02/18/2021 02:35:32] TASK - [Final: Non-interactive mode] **************************************
For the non-interactive use of the tool, run the following command:
convert2rhel -u nagulan.selvakumar -p ***** --pool 8a85f99a774ac5420177570d75625d4e -v Server -y


WARNING - In order to boot the RHEL kernel, restart of the system is needed.
[root@linux75 software]# reboot

Step 6: Verification

Now we can verify whether system is converted and Oracle database is working without any issues. 
[root@linux75 ~]# cat /etc/os-release
NAME="Red Hat Enterprise Linux Server"
VERSION="7.9 (Maipo)"
ID="rhel"
ID_LIKE="fedora"
VARIANT="Server"
VARIANT_ID="server"
VERSION_ID="7.9"
PRETTY_NAME="Red Hat Enterprise Linux"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:redhat:enterprise_linux:7.9:GA:server"
HOME_URL="https://www.redhat.com/"
BUG_REPORT_URL="https://bugzilla.redhat.com/"

REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7"
REDHAT_BUGZILLA_PRODUCT_VERSION=7.9
REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux"
REDHAT_SUPPORT_PRODUCT_VERSION="7.9"
[root@linux75 ~]#
[root@linux75 ~]# subscription-manager list
+-------------------------------------------+
    Installed Product Status
+-------------------------------------------+
Product Name:   Red Hat Enterprise Linux Server
Product ID:     69
Version:        7.9
Arch:           x86_64
Status:         Subscribed
Status Details:
Starts:         01/31/2021
Ends:           01/31/2022

[root@linux75 ~]#
[oracle@linux75 ~]$ sqlplus / as sysdba

SQL*Plus: Release 12.2.0.1.0 Production on Thu Feb 18 02:45:10 2021

Copyright (c) 1982, 2016, Oracle.  All rights reserved.

Connected to an idle instance.

SQL> startup
ORACLE instance started.

Total System Global Area 1073741824 bytes
Fixed Size                  8628936 bytes
Variable Size             448791864 bytes
Database Buffers          608174080 bytes
Redo Buffers                8146944 bytes
Database mounted.
Database opened.
SQL> select INSTANCE_NAME, HOST_NAME, VERSION from v$instance;

INSTANCE_NAME    HOST_NAME                      VERSION
---------------- ------------------------------ -----------------
orcl             linux75.selvapc.com            12.2.0.1.0

SQL>
All is good....!!! 

References: 

How to convert from CentOS or Oracle Linux to RHEL

Happy Converting !!! :) 

Thursday, February 11, 2021

Oracle Grid/RDBMS 19c on RHEL 8 - Tips, issues and work arounds

I did an install of Oracle GI and Database version 19c on RHEL 8 in my home lab (we can download a copy of RHEL8 for personal/non-production use if registered in https://developers.redhat.com) where I encountered a few issues. 

Let's see them with their work arounds in this post. 

1.    Oracle software installation prerequisite steps can be automatically done by installing Database pre-install rpms from the below repository links

http://public-yum.oracle.com/repo/OracleLinux/OL8/baseos/latest/x86_64/getPackage/oracle-database-preinstall-19c-1.0-1.el8.x86_64.rpm

http://yum.oracle.com/repo/OracleLinux/OL8/baseos/latest/x86_64/getPackage/oracle-database-preinstall-19c-1.0-1.el8.x86_64.rpm

2.    Mandatory patch requirement

RHEL was released later than Oracle 19c. So when attempting to install Oracle 19c software, we need to apply patch along with the installation of the software. The minimum required patch level to apply is GIRU19.6 or DBRU19.6 with few oneOff patches or GIRU19.7 or DBRU19.7 patch. As I'm installing now, post GIRU19.10 is released, I'm patching to the latest available patch. So I'm using the command as below to invoke the Grid setup. 

[oracle@linux-8 19]$ ./gridSetup.sh -applyRU /media/sf_Oracle/software/32226239

oracle@linux-8 db19]$ ./runInstaller -applyRU /media/sf_Oracle/software/32226239/32218454

Patch 32226239: GI RELEASE UPDATE 19.10.0.0.0 can be downloaded from Oracle support. This patch includes DB patch (32218454) as well.

3.    INS-08101 Unexpected error 'supportedOSCheck'

As the software is released prior to RHEL/OL 8, the software doesn't recognizes the OS version and throws the below error. 

Work Around: 
Set the environment variable CV_ASSUME_DISTID to OL7 and then start the Grid setup/installer. 

[oracle@linux-8 19]$ export CV_ASSUME_DISTID=OL7

4.    compat-libcap1-1.10 package missing error

Prerequisites check page shows that compat-libcap1-1.10 package is missing.

Work Around: 
This error can safely be ignored. Fix all other errors apart from this package error and click on Ignore all check box and proceed with the installation. 

5.    OPatch requirement

As we need to apply patch as well, we have to use the latest version of OPatch while installing software along with patching. Check for Patch 6880880 in Oracle support and download the latest patch available with correct version of the OS selected. At the time of writing this post, the latest version is OPatch Version: 12.2.0.1.23

6. ASM FD is not supported

During "Create ASM Disk Group" page, when Configure ASM Filter Driver is selected, the following error is thrown. 
AFD-9213: Configuration file 'symvers-4.18.0-240.10.1.el8_3.x86_64.gz' in the /boot directory does not exist or cannot be read.

Note: If you don't apply RU along when starting the gridSetup, then the error will be as below
AFD-620: AFD is not supported on this operating system version: 'unknown'

Work Around: 
Referring to RedHat support document 5174441, the location of /boot/symvers-$(uname -r).gz has been changed to /usr/lib/modules/$(uname -r)/symvers.gz in RHEL 8.
Also, we are running the below kernel level which is not supported by Oracle for ACFS and ASM FD at the time of installation (up to 193 is supported). Refer Support note 1369107.1 for compatibility check

[root@linux-8 modules]# uname -r        
4.18.0-240.10.1.el8_3.x86_64                


As the patch level is still not officially supported, I don't want to copy the symvers file to /boot and try which might mess up other things. So deselecting the ASM FD for now and continuing installation

7.    Automatic root script execution

If you would like the installer to run the root scripts automatically by providing the information such as root password or sudo privileges, edit the /etc/hosts file with the server details. Else the installer will fail stating unknown host. 



References: 
Requirements for Installing Oracle Database/Client 19c on OL8 or RHEL8 64-bit (x86-64) (Doc ID 2668780.1)
ACFS and AFD 19c Supported Platforms
How To Download And Install The Latest OPatch(6880880) Version (Doc ID 274526.1)
Known Issues and Bugs for Oracle Linux 8 and Red Hat Enterprise Linux 8
Patch 32226239: GI RELEASE UPDATE 19.10.0.0.0

Happy Installing...!!!