{ "date": "2021-05-14T11:20:43", "slug": "el-carro-the-oracle-operator-for-kubernetes", "link": "https://www.dbi-services.com/blog/el-carro-the-oracle-operator-for-kubernetes/", "title": { "rendered": "El Carro: The Oracle Operator for Kubernetes" }, "content": { "rendered": "

By Franck Pachot

\n

.
\nGoogle Cloud, Open Source and Oracle Databases… what seems to be a paradox is possible, thanks to cloud providers who contribute to open infrastructure. The idea is to use Operators (custom resource controllers on Kubernetes) to automate the Oracle Database operations in a standard, open and portable way. If you ever attempted to run Oracle Database on containers, trying to keep up with the DevOps approach, you know that it requires a bit of complexity and careful orchestration.

\n

The public announce was on the Google Open Source Blog: Modernizing Oracle operations with Kubernetes and El Carro. This is an Open Source project where we can contribute: https://github.com/GoogleCloudPlatform/elcarro-oracle-operator. I’ve tried the simplest thing: install Oracle XE – the free edition of Oracle, because it is the only one that you can deploy without cross-checking, with your lawyers, the license contracts and the “educational purpose only” documents about Oracle audit policies. But running Oracle on Kubernetes applies the same rules as virtualization: count the vCPU or the physical processors (depending on the hypervisor isolation accepted by Oracle). Basically the “installed or running” terms apply where the image is pulled.

\n

Download El Carro software and install Oracle 18c XE

\n

I’ll run all this from the Cloud Shell but of course you can do it from any configured gcloud CLI.

\n
\nfranck@cloudshell:~ (google-cloud.424242)$ gcloud alpha iam service-accounts list\n\nDISPLAY NAME                            EMAIL                                               DISABLED\nCompute Engine default service account  424242424242-compute@developer.gserviceaccount.com  False\n
\n

I take note of my service account from there.

\n

Installing Oracle is 3 lines only:

\n
mkdir -p $HOME/elcarro-oracle-operator\ngsutil -m cp -r gs://elcarro/latest $HOME/elcarro-oracle-operator\nbash $HOME/elcarro-oracle-operator/latest/deploy/install-18c-xe.sh --service_account 424242424242-compute@developer.gserviceaccount.com
\n

I’m following the instructions from https://github.com/GoogleCloudPlatform/elcarro-oracle-operator/blob/main/docs/content/quickstart-18c-xe.md here.

\n

\"\"

\n

This takes a while the first time (45 minutes) because it has to create the image, built from oracle-database-xe-18c-1.0-1.x86_64.rpm RPM. The image is nearly 6GB and is stored in your Container Registry. Then it creates the cluster. The default cluster name is “gkecluster”, the CDB name is GCLOUD and the defaut zone is us-central1-a but you can pass the -c -k -z option on the install-18c-xe.sh to change those defaults. It creates a 2 nodes, total 4 vCPUs, 15 GB RAM, 200GB persistent storage. The namespace is “db”.

\n
\n...\nkubeconfig entry generated for gkecluster.\nNAME        LOCATION       MASTER_VERSION   MASTER_IP     MACHINE_TYPE   NODE_VERSION     NUM_NODES  STATUS\ngkecluster  us-central1-a  1.19.9-gke.1900  34.67.217.61  n1-standard-2  1.19.9-gke.1900  2          RUNNING\nstorageclass.storage.k8s.io/csi-gce-pd created\nvolumesnapshotclass.snapshot.storage.k8s.io/csi-gce-pd-snapshot-class created\nnamespace/operator-system created\n...\nWaiting for startup, statuses: InstanceReady=, InstanceDatabaseReady=, DatabaseReady=\nWaiting for startup, statuses: InstanceReady=, InstanceDatabaseReady=, DatabaseReady=CreatePending\n...\nWaiting for startup, statuses: InstanceReady=CreateInProgress, InstanceDatabaseReady=, DatabaseReady=CreatePending\n...\nWaiting for startup, statuses: InstanceReady=CreateComplete, InstanceDatabaseReady=CreateInProgress, DatabaseReady=CreatePending\n...\nWaiting for startup, statuses: InstanceReady=CreateComplete, InstanceDatabaseReady=CreateComplete, DatabaseReady=CreatePending\nWaiting for startup, statuses: InstanceReady=CreateComplete, InstanceDatabaseReady=CreateComplete, DatabaseReady=CreateComplete\nOracle Operator is installed. Database connection command:\n> sqlplus scott/tiger@35.224.235.49:6021/pdb1.gke\nfranck@cloudshell:~ (google-cloud.424242)$\n
\n

Be patient… it is Oracle, it has a pre-DevOps installation timing… And this is why it is really good to have a standardized way for automation. Building your own is a lot of effort because each iteration takes time to validate.

\n

So, all is installed, with a service endpoint exposed to the public internet on port 6021:

\n
\n[opc@a ~]$ ~/sqlcl/bin/sql scott/tiger@35.224.235.49:6021/pdb1.gke\n\nSQLcl: Release 21.1 Production on Fri May 14 10:40:03 2021\n\nCopyright (c) 1982, 2021, Oracle.  All rights reserved.\n\nConnected to:\nOracle Database 18c Express Edition Release 18.0.0.0.0 - Production\nVersion 18.4.0.0.0\n\nSQL> select * from v$session_connect_info;\n\n   SID    SERIAL#    AUTHENTICATION_TYPE    OSUSER                                                    NETWORK_SERVICE_BANNER    CLIENT_CHARSET    CLIENT_CONNECTION    CLIENT_OCI_LIBRARY    CLIENT_VERSION            CLIENT_DRIVER            CLIENT_LOBATTR    CLIENT_REGID    CON_ID\n______ __________ ______________________ _________ _________________________________________________________________________ _________________ ____________________ _____________________ _________________ ________________________ _________________________ _______________ _________\n   283      20073 DATABASE               opc       TCP/IP NT Protocol Adapter for Linux: Version 18.0.0.0.0 - Production     Unknown           Heterogeneous        Unknown               21.16.0.0.0       jdbcthin : 21.1.0.0.0    Client Temp Lob Rfc On                  0         3\n   283      20073 DATABASE               opc       Encryption service for Linux: Version 18.0.0.0.0 - Production             Unknown           Heterogeneous        Unknown               21.16.0.0.0       jdbcthin : 21.1.0.0.0    Client Temp Lob Rfc On                  0         3\n   283      20073 DATABASE               opc       Crypto-checksumming service for Linux: Version 18.0.0.0.0 - Production    Unknown           Heterogeneous        Unknown               21.16.0.0.0       jdbcthin : 21.1.0.0.0    Client Temp Lob Rfc On                  0         3\n\nSQL> select initcap(regexp_replace(reverse('El Carro'),'(.)\\1+| ','\\1')) \"K8s Operator for\" from dual;\n\n  K8s Operator for\n__________________\nOracle\n\nSQL> \n
\n

Now you see where the “El Carro” name comes from, right? 🤣

\n

I can check the pods, from the Web Console, or CLI, remember the namespace is ‘db’:

\n
franck@cloudshell:~ (google-cloud.424242)$ kubectl get pods -n db\n\nNAME                                     READY   STATUS    RESTARTS   AGE\nmydb-agent-deployment-6c8b7647fb-d4lkf   2/2     Running   0          77m\nmydb-sts-0                               4/4     Running   0          77m\n\nfranck@cloudshell:~ (google-cloud.424242)$ kubectl get services -n db\n\nNAME                TYPE           CLUSTER-IP      EXTERNAL-IP     PORT(S)                         AGE\nmydb-agent-svc      ClusterIP      10.59.242.135             3202/TCP,9161/TCP               107m\nmydb-dbdaemon-svc   ClusterIP      10.59.244.15              3203/TCP                        107m\nmydb-svc            LoadBalancer   10.59.245.142   35.224.235.49   6021:30156/TCP,3307:32007/TCP   107m\nmydb-svc-node       NodePort       10.59.250.249             6021:32512/TCP,3307:31243/TCP   107m\nfranck@cloudshell:~ (google-cloud.424242)$\n
\n

The service is exposed externally by the LoadBalancer on the Secure Listener port

\n

I can connect to the container to look at what is running there.

\n
\nfranck@cloudshell:~ (google-cloud.424242)$ kubectl  exec -it -n db mydb-sts-0 -c oracledb -- bash -i\n\nbash-4.2$ grep \":[YN]\" /etc/oratab\n\nGCLOUD:/opt/oracle/product/18c/dbhomeXE:N\n \nbash-4.2$ . oraenv <<<GCLOUD\n\nORACLE_SID = [] ? The Oracle base remains unchanged with value /opt/oracle\n \nbash-4.2$ ps -fp $(pgrep tnslsnr)\n\nUID          PID    PPID  C STIME TTY          TIME CMD\noracle       488       1  0 09:32 ?        00:00:00 /opt/oracle/product/18c/dbhomeXE/bin/tnslsnr SECURE -inherit\n\nbash-4.2$ lsnrctl status SECURE          \n\nLSNRCTL for Linux: Version 18.0.0.0.0 - Production on 14-MAY-2021 10:18:11\n\nCopyright (c) 1991, 2018, Oracle.  All rights reserved.\n\nTNS-01101: Could not find listener name or service name SECURE\n\nbash-4.2$ lsnrctl status //localhost:6021\n\nLSNRCTL for Linux: Version 18.0.0.0.0 - Production on 14-MAY-2021 10:17:42\n\nCopyright (c) 1991, 2018, Oracle.  All rights reserved.\n\nConnecting to (DESCRIPTION=(CONNECT_DATA=(SERVICE_NAME=))(ADDRESS=(PROTOCOL=TCP)(HOST=127.0.0.1)(PORT=6021)))\nSTATUS of the LISTENER\n------------------------\nAlias                     SECURE\nVersion                   TNSLSNR for Linux: Version 18.0.0.0.0 - Production\nStart Date                14-MAY-2021 09:32:52\nUptime                    0 days 0 hr. 44 min. 49 sec\nTrace Level               off\nSecurity                  ON: Local OS Authentication\nSNMP                      OFF\nListener Parameter File   /u02/app/oracle/oraconfig/network/SECURE/listener.ora\nListener Log File         /u02/app/oracle/diag/tnslsnr/mydb-sts-0/secure/alert/log.xml\nListening Endpoints Summary...\n  (DESCRIPTION=(ADDRESS=(PROTOCOL=ipc)(KEY=REGLSNR_6021)))\n  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcp)(HOST=mydb-sts-0)(PORT=6021)))\n  (DESCRIPTION=(ADDRESS=(PROTOCOL=tcps)(HOST=127.0.0.1)(PORT=5500))(Security=(my_wallet_directory=/opt/oracle/admin/GCLOUD_uscentral1a/xdb_wallet))(Presentation=HTTP)(Session=RAW))\nServices Summary...\nService \"GCLOUD.gke\" has 1 instance(s).\n  Instance \"GCLOUD\", status UNKNOWN, has 1 handler(s) for this service...\nService \"GCLOUDXDB.gke\" has 1 instance(s).\n  Instance \"GCLOUD\", status READY, has 1 handler(s) for this service...\nService \"GCLOUD_uscentral1a.gke\" has 1 instance(s).\n  Instance \"GCLOUD\", status READY, has 1 handler(s) for this service...\nService \"PDB1.gke\" has 2 instance(s).\n  Instance \"GCLOUD\", status UNKNOWN, has 1 handler(s) for this service...\n  Instance \"GCLOUD\", status READY, has 1 handler(s) for this service...\nService \"c246feca2ab003e8e0530901380a0e21.gke\" has 1 instance(s).\n  Instance \"GCLOUD\", status READY, has 1 handler(s) for this service...\nThe command completed successfully\n\nbash-4.2$\n
\n

Here is the Oracle XE listener, with TNS_ADMIN in /u02/app/oracle/oraconfig/network/SECURE

\n

You can have a look at the available operations from the samples (that you can customize and tun with `kubectl apply -n db -f`):

\n
\nfranck@cloudshell:~ (google-cloud.424242)$ ls ./elcarro-oracle-operator/latest/samples\n\nv1alpha1_backup_rman1.yaml            v1alpha1_database_pdb1.yaml\nv1alpha1_backup_rman2.yaml            v1alpha1_database_pdb2.yaml\nv1alpha1_backup_rman3.yaml            v1alpha1_database_pdb3.yaml\nv1alpha1_backup_rman4.yaml            v1alpha1_database_pdb4.yaml\nv1alpha1_backupschedule.yaml          v1alpha1_export_dmp1.yaml\nv1alpha1_backup_snap1.yaml            v1alpha1_export_dmp2.yaml\nv1alpha1_backup_snap2.yaml            v1alpha1_import_pdb1.yaml\nv1alpha1_backup_snap_minikube.yaml    v1alpha1_instance_18c_XE_express.yaml\nv1alpha1_config_bm1.yaml              v1alpha1_instance_18c_XE.yaml\nv1alpha1_config_bm2.yaml              v1alpha1_instance_custom_seeded.yaml\nv1alpha1_config_gcp1.yaml             v1alpha1_instance_express.yaml\nv1alpha1_config_gcp2.yaml             v1alpha1_instance_gcp_ilb.yaml\nv1alpha1_config_gcp3.yaml             v1alpha1_instance_minikube.yaml\nv1alpha1_config_minikube.yaml         v1alpha1_instance_standby.yaml\nv1alpha1_cronanything.yaml            v1alpha1_instance_unseeded.yaml\nv1alpha1_database_pdb1_express.yaml   v1alpha1_instance_with_backup_disk.yaml\nv1alpha1_database_pdb1_gsm.yaml       v1alpha1_instance.yaml\nv1alpha1_database_pdb1_unseeded.yaml\nfranck@cloudshell:~ (google-cloud.424242)$\n
\n

Storage snapshots (v1alpha1_backup_snap2.yaml), backups (v1alpha1_backup_rman3.yaml), exports (v1alpha1_export_dmp1.yaml)

\n

All is documented: https://github.com/GoogleCloudPlatform/elcarro-oracle-operator and will probably evolve.

\n", "protected": false } }