deon/spdocker
Archived
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

Complete upgrade and spell check

Browse Source
This commit is contained in:
Deon George 2018-03-14 17:34:06 +11:00
parent 068d02a01d
commit 244fbfa243
No known key found for this signature in database
GPG Key ID: 7670E8DC27415254
7 changed files with 143 additions and 71 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

BIN
source/img/header_img/oc.jpg Normal file
View File

Binary file not shown.
Side by Side

After

Width:  |  Height:  |  Size: 182 KiB

12
source/index.md
View File

@ -30,27 +30,27 @@ Docker is tool that will make the lives of system admins easier. It really helps
3. It separates files relating to any application into 3 specific categories: 3. It separates files relating to any application into 3 specific categories:
1. Application related files - in Docker, they are the put inside the Docker image, when the image is built. 1. Application related files - in Docker, they are the put inside the Docker image, when the image is built.
2. Application data files - this is the data that personalises the application, and often is linked to the specific application version that created the data. In general terms, this has business value, it could be your Email database data, your RDBMS database data, etc 2. Application data files - this is the data that personalises the application, and often is linked to the specific application version that created the data. In general terms, this has business value, it could be your Email database data, your RDBMS database data, etc
3. Temporary data files - these are the files that your application may create as a result of running, (eg: temp files, etc) - they have no value, but are needed by the application to run. If they are lost, there is no concern. 3. Temporary data files - these are the files that your application may create as a result of running, (EG: temp files, etc) - they have no value, but are needed by the application to run. If they are lost, there is no concern.
So, for any application, you just need to keep the Application related files (1) and the Application data files (2). So, for any application, you just need to keep the Application related files (1) and the Application data files (2).
By design containers should be considered ephemeral, ie: they can be destroyed and created many times. As long as the Application data files are connected to the same version of an Application related files when a Container is started, the application will live on until it is retired. By design containers should be considered ephemeral, IE: they can be destroyed and created many times. As long as the Application data files are connected to the same version of an Application related files when a Container is started, the application will live on until it is retired.
In Docker, your Application related files are in a Docker Image, your Application data files are on "Persistent Storage" and the Temporary data files are often created inside the container, and discarded when the container is destroyed. In Docker, your Application related files are in a Docker Image, your Application data files are on "Persistent Storage" and the Temporary data files are often created inside the container, and discarded when the container is destroyed.
# Docker and Spectrum Protect # Docker and Spectrum Protect
## Spectrum Protect Server ## Spectrum Protect Server
So from a Spectrum Protect point of view, if we are to run a Spectrum Protect server, containerizing it enables you to leverage points all three points above: So from a Spectrum Protect point of view, if we are to run a Spectrum Protect server, containerising it enables you to leverage points all three points above:
1. Regardless of whether you'll have 1 SP server or 100, you install it "once" and store it in your Docker Image library (aka a Docker Registry) and you can deploy it many times almost instantly. 1. Regardless of whether you'll have 1 SP server or 100, you install it "once" and store it in your Docker Image library (aka a Docker Registry) and you can deploy it many times almost instantly.
2. When you upgrade your Spectrum Protect server, you have easy upgrade and back-out plan, and in fact you can even "test" the upgrade before you do it live. 2. When you upgrade your Spectrum Protect server, you have easy upgrade and back-out plan, and in fact you can even "test" the upgrade before you do it live.
3. When you deploy it, your "Application data files" will be your Spectrum Protect configuration (ie: the database), and the Spectrum Protect configuration data (ie: your disk storage pools). 3. When you deploy it, your "Application data files" will be your Spectrum Protect configuration (IE: the database), and the Spectrum Protect configuration data (IE: your disk storage pools).
You can place those anywhere you like following your organisations data deployment policies, but when referenced inside the container, they are always referenced at the same point that SP has been prepared for when you have "installed it". You can place those anywhere you like following your organisations data deployment policies, but when referenced inside the container, they are always referenced at the same point that SP has been prepared for when you have "installed it".
If you are interested in Dockerising Spectrum Protect, this might help you get [started](server). If you are interested in Dockerising Spectrum Protect, this might help you get [started](server).
## Spectrum Protect Client ## Spectrum Protect Client
From a client point of view, ie: protecting your business application and enabling you to recover it, by Dockerising your application, you have many your Spectrum Protect management even easier. IE: From the same three points above: From a client point of view, IE: protecting your business application and enabling you to recover it, by Dockerising your application, you have many your Spectrum Protect management even easier. IE: From the same three points above:
1. Application related files - ie: these are Docker Images and dont really need to be protected (daily at least). You'll have a copy of your Docker Image on your host(s) where your application is deployed and a copy in your Docker registry. 1. Application related files - IE: these are Docker Images and don't really need to be protected (daily at least). You'll have a copy of your Docker Image on your host(s) where your application is deployed and a copy in your Docker registry.
If your host ever dies, you'll setup a new host, ```docker pull``` the application, recover your data and start your application. (If anything, you could protect your "built" application by protecting the Docker Registry that it is stored in.) If your host ever dies, you'll setup a new host, ```docker pull``` the application, recover your data and start your application. (If anything, you could protect your "built" application by protecting the Docker Registry that it is stored in.)
2. Application data files, which represent your applications configuration and data, you do want to protect this, and in many cases you can protect this from "outside" the container. 2. Application data files, which represent your applications configuration and data, you do want to protect this, and in many cases you can protect this from "outside" the container.
There is no silver bullet on how to protect this data, so it will depend on the application and whether it provides an API or a "dump" capability on your approach to protecting it. In very, very simple terms, you could "pause" a Docker container, snapshot the devices (disk or file system) where the persistent storage is held and you have the equivalent of a hardware based snapshot backup. There is no silver bullet on how to protect this data, so it will depend on the application and whether it provides an API or a "dump" capability on your approach to protecting it. In very, very simple terms, you could "pause" a Docker container, snapshot the devices (disk or file system) where the persistent storage is held and you have the equivalent of a hardware based snapshot backup.

12
source/server/firstrun.md
View File

@ -7,20 +7,20 @@ related:
- { page: "index", title: "Containerising SP" } - { page: "index", title: "Containerising SP" }
- { page: "normalrun", title: "Normal Run of Container" } - { page: "normalrun", title: "Normal Run of Container" }
- { page: "upgrade", title: "Upgrading SP" } - { page: "upgrade", title: "Upgrading SP" }
- { page: "recover", title: "Recoverying the SP Database" } - { page: "recover", title: "Recovering the SP Database" }
--- ---
# First deployment of a Spectrum Protect Server (in a container) # First deployment of a Spectrum Protect Server (in a container)
> **NOTES**: > **NOTES**:
> * The following instructions are for a Spectrum Protect 8.1.4 instance. I used the same procedure described [here](/server) to build the Docker Image. The only difference is that I performed a new install using the installation media that would be available from Passport Advantage. (Thus it was just an install, not an 8.1.0 install and followed by a 8.1.4 upgrade). > * The following instructions are for a Spectrum Protect 8.1.4 instance. I used the same procedure described [here](/server) to build the Docker Image. The only difference is that I performed a new install using the installation media that would be available from Passport Advantage. (Thus it was just an install, not an 8.1.0 install and followed by a 8.1.4 upgrade).
> * Make sure you follow the guidelines for space required as presented in the [Knowledge Center](https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.4/srv.install/t_srv_plan_capacity-linux.html). > * Make sure you follow the guidelines for space required as presented in the [Knowledge Center](https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.4/srv.install/t_srv_plan_capacity-linux.html).
Our Docker Image will have the SP database in `/database`, the instance user in `/tsm` and storage pools in `/data`. Ultimately these will be volumes provisioned from the host and using host storage, or your prefered Docker persistent volume storage technology. Our Docker Image will have the SP database in `/database`, the instance user in `/tsm` and storage pools in `/data`. Ultimately these will be volumes provisioned from the host and using host storage, or your preferred Docker persistent volume storage technology.
> * The storage used in the example below is from the host file systems, passed to the container with Docker's `-v` option. > * The storage used in the example below is from the host file systems, passed to the container with Docker's `-v` option.
> * This implementation is using DISK or CLOUD storage pool storage. While accessing tape should be possible, I havent explored what is required for that to work. > * This implementation is using DISK or CLOUD storage pool storage. While accessing tape should be possible, I haven't explored what is required for that to work.
## Configuration ## Configuration
Our configuration will use these defaults Our configuration will use these defaults
```bash Container Variables ```bash Helpful Variables
TSM_DB=/srv/sp/database TSM_DB=/srv/sp/database
TSM_USER=/srv/sp/user TSM_USER=/srv/sp/user
TSM_DATA=/srv/sp/data TSM_DATA=/srv/sp/data
@ -33,7 +33,7 @@ TSM_DATA=/srv/sp/data
## First Run Init #1 ## First Run Init #1
The first init run will be to create the database layout, and to copy the initialise database (as performed when containering SP) onto our persistent storage. The first init run will be to create the database layout, and to copy the initialise database (as performed when containerising SP) onto our persistent storage.
`docker run --detach=false --hostname=tsm --interactive=false -v $TSM_USER:/tsm.init -v $TSM_DB:/database.init --privileged=true --rm=true --restart=no --tty=false --name=spectrumprotect ibm/spectrumprotect:8.1.4 init1` `docker run --detach=false --hostname=tsm --interactive=false -v $TSM_USER:/tsm.init -v $TSM_DB:/database.init --privileged=true --rm=true --restart=no --tty=false --name=spectrumprotect ibm/spectrumprotect:8.1.4 init1`
@ -75,6 +75,6 @@ ANR1912I Stopping the activity log because of a server shutdown.
ANR0369I Stopping the database manager because of a server shutdown. ANR0369I Stopping the database manager because of a server shutdown.
``` ```
<small>**NOTE**: With 8.1.4 I noticed a lot of `DBI1306N` messages floating up the screen. That doesnt appear to be an actual problem.</small> <small>**NOTE**: With 8.1.4 I noticed a lot of `DBI1306N` messages floating up the screen. That doesn't appear to be an actual problem.</small>
The "Init" steps will only need to be run the once for any "new" instance of SP. Thereafter, you'll start SP using the Normal steps documented [here](normalrun.html). The "Init" steps will only need to be run the once for any "new" instance of SP. Thereafter, you'll start SP using the Normal steps documented [here](normalrun.html).

22
source/server/index.md
View File

@ -7,7 +7,7 @@ related:
- { page: "firstrun", title: "Initial Container Setup" } - { page: "firstrun", title: "Initial Container Setup" }
- { page: "normalrun", title: "Normal Run of Container" } - { page: "normalrun", title: "Normal Run of Container" }
- { page: "upgrade", title: "Upgrading SP" } - { page: "upgrade", title: "Upgrading SP" }
- { page: "recover", title: "Recoverying the SP Database" } - { page: "recover", title: "Recovering the SP Database" }
- { page: "runoc", title: "Running OC in a container" } - { page: "runoc", title: "Running OC in a container" }
--- ---
# Putting Spectrum Protect into a Container # Putting Spectrum Protect into a Container
@ -18,19 +18,19 @@ related:
For example if your storage driver is `devicemapper` then the default volume size `Base Device Size` may be 10GB, which wont be enough for a SP installation. You will need to increase it to at least 15GB - details on how to do that are outside of the scope of this document. For example if your storage driver is `devicemapper` then the default volume size `Base Device Size` may be 10GB, which wont be enough for a SP installation. You will need to increase it to at least 15GB - details on how to do that are outside of the scope of this document.
> * Also make sure your host has sufficient memory to perform the build - SP requires at least 12GB of RAM to pass the installation logic. > * Also make sure your host has sufficient memory to perform the build - SP requires at least 12GB of RAM to pass the installation logic.
I actually perform the build on an iMac with 16GB of memory allocated to Docker. It is also using the `overlay` storage driver, which means the size of the volume the build container gets is the available free capacity of my /Users mount point. I actually perform the build on an iMac with 16GB of memory allocated to Docker. It is also using the `overlay` storage driver, which means the size of the volume the build container gets is the available free capacity of my /Users mount point.
> * The accompaning files for this build are available in my [gitlab](http://dev.leenooks.net/deon/spdocker/tree/master) > * The accompanying files for this build are available in my [gitlab](http://dev.leenooks.net/deon/spdocker/tree/master)
> * And lastly, most importantly, running SP in a container is **not officially supported by IBM** - so you cannot make a helpdesk call about having problems. > * And lastly, most importantly, running SP in a container is **not officially supported by IBM** - so you cannot make a help-desk call about having problems.
Technically it should work without problems, but if you are experiencing a problem, be prepared to recreate your environment on a physical host and validate that your problem still exists (and thus not a result of Docker or the implementation on Docker). Technically it should work without problems, but if you are experiencing a problem, be prepared to recreate your environment on a physical host and validate that your problem still exists (and thus not a result of Docker or the implementation on Docker).
## Create your Dockerfile ## Create your Dockerfile
As at the time of writing these instructions, Red Hat linux is not available in the Docker hub - but CentOS is a good clone. We'll use it As at the time of writing these instructions, Red Hat Linux is not available in the Docker hub - but CentOS is a good clone. We'll use it
### Image preparation ### Image preparation
```Dockerfile ```Dockerfile
FROM centos:7 FROM centos:7
``` ```
Next we need to install some dependancies - these are documented in the [Knowledge Center](https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.4/srv.install/r_srv_lnx_sysreq_inst_x-linux.html) as a requirement for running SP on Linux. You might like to check to make sure these are still all that is required for newer releases of SP. Next we need to install some dependencies - these are documented in the [Knowledge Center](https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.4/srv.install/r_srv_lnx_sysreq_inst_x-linux.html) as a requirement for running SP on Linux. You might like to check to make sure these are still all that is required for newer releases of SP.
```Dockerfile ```Dockerfile
RUN yum -y install http://yum.leenooks.net/CentOS/7/base/redhat-release-7-6.el7.centos.12.2.x86_64.rpm \ RUN yum -y install http://yum.leenooks.net/CentOS/7/base/redhat-release-7-6.el7.centos.12.2.x86_64.rpm \
@ -41,11 +41,11 @@ RUN yum -y install http://yum.leenooks.net/CentOS/7/base/redhat-release-7-6.el7.
yum clean all && rm -rf /var/tmp/* yum clean all && rm -rf /var/tmp/*
``` ```
<small>**NOTE**: During the installation of SP, it tests to make sure it is running on RedHat linux by testing the existence of an RPM package `redhat-release`. The redhat-release installed on line 1 is a fake RPM package - it installs no files, but just emulates the existance of that required RPM package.</small> <small>**NOTE**: During the installation of SP, it tests to make sure it is running on RedHat Linux by testing the existence of an RPM package `redhat-release`. The redhat-release installed on line 1 is a fake RPM package - it installs no files, but just emulates the existence of that required RPM package.</small>
### Spectrum Protect Installation ### Spectrum Protect Installation
Now copy in our silent install configuration files. These files are based on the silent install xml files that come on the installation media. Some details on these files are also in the [Knowledge Center](https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.4/srv.install/t_srv_inst_silently-linux.html) Now copy in our silent install configuration files. These files are based on the silent install XML files that come on the installation media. Some details on these files are also in the [Knowledge Center](https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.4/srv.install/t_srv_inst_silently-linux.html)
```Dockerfile ```Dockerfile
COPY install.xml install-fp.xml /tmp/ COPY install.xml install-fp.xml /tmp/
``` ```
@ -67,7 +67,7 @@ RUN SOURCE_URL=http://YOUR_SITE_URL_HERE && \
rm -rf /tmp/build /tmp/install*xml rm -rf /tmp/build /tmp/install*xml
``` ```
**NOTE**: Some information on what is occuring here. **NOTE**: Some information on what is occurring here.
* This is an "new install" (lines 1-6) then followed by an "upgrade" (lines 7-11). The reason for the new install first, is that the Spectrum Protect license files and license libraries are not available in upgrades (from Fix Central). So you'll need to install from your downloaded media from Passport Advantage, and then upgrade it to the latest release from Fix Central. * This is an "new install" (lines 1-6) then followed by an "upgrade" (lines 7-11). The reason for the new install first, is that the Spectrum Protect license files and license libraries are not available in upgrades (from Fix Central). So you'll need to install from your downloaded media from Passport Advantage, and then upgrade it to the latest release from Fix Central.
If you sourced your installation media from Passport Advantage, then the license files and libraries are probably in that installation media, so you only need to run lines 1-6. If you sourced your installation media from Passport Advantage, then the license files and libraries are probably in that installation media, so you only need to run lines 1-6.
* To avoid the resulting Docker Image being too large, we download the installation media from a web server (Line 3) and then delete it after it has been expanded (line 4). We also do the same for up upgrade package (lines 8 & 9). * To avoid the resulting Docker Image being too large, we download the installation media from a web server (Line 3) and then delete it after it has been expanded (line 4). We also do the same for up upgrade package (lines 8 & 9).
@ -76,11 +76,11 @@ RUN SOURCE_URL=http://YOUR_SITE_URL_HERE && \
### Spectrum Protect Post-Installation ### Spectrum Protect Post-Installation
Now we set our user environment that will utlimately run SP Now we set our user environment that will ultimately run SP
```Dockerfile ```Dockerfile
ENV USER=tsm USERDIR=/tsm USERID=201 GROUPID=201 ENV USER=tsm USERDIR=/tsm USERID=201 GROUPID=201
``` ```
<small>**NOTE**: You can set these according to your enterprise policy. `USERDIR` will ultimately need to be persistant storage using your favourite storage provisioning methodology. I generally host mount when starting the container using `-v /host/path/user:/tsm` and it works well. I have used some 3rd party persistent storage offerings, however, they often have not worked. While I havent explored in detail why not, I suspect it is because of Direct IO calls being made by DB2 and those 3rd party offerings not supporting those sytem calls.</small> <small>**NOTE**: You can set these according to your enterprise policy. `USERDIR` will ultimately need to be persistent storage using your favourite storage provisioning methodology. I generally host mount when starting the container using `-v /host/path/user:/tsm` and it works well. I have used some 3rd party persistent storage offerings, however, they often have not worked. While I haven't explored in detail why not, I suspect it is because of Direct IO calls being made by DB2 and those 3rd party offerings not supporting those system calls.</small>
Now we create our user Now we create our user
@ -141,6 +141,6 @@ REPOSITORY TAG IMAGE ID CREATED SIZE
ibm/spectrumprotect 8.1.1 14aba816e231 2 minutes ago 2.96GB ibm/spectrumprotect 8.1.1 14aba816e231 2 minutes ago 2.96GB
``` ```
**NOTES** **NOTES**
* Make sure you have enough space in your `/var/lib/docker` directory - the build will use about 9 GBs. * Make sure you have enough space in your `/var/lib/docker` directory - the build will use about 9 GB's.
Now that your container is built, you'll want to start it a specific way, depending on whether you are starting a clean new instance of SP, or an existing instance. Those details are [here](firstrun.html). Now that your container is built, you'll want to start it a specific way, depending on whether you are starting a clean new instance of SP, or an existing instance. Those details are [here](firstrun.html).

12
source/server/normalrun.md
View File

@ -7,17 +7,17 @@ related:
- { page: "index", title: "Containerising SP" } - { page: "index", title: "Containerising SP" }
- { page: "firstrun", title: "Initial Container Setup" } - { page: "firstrun", title: "Initial Container Setup" }
- { page: "upgrade", title: "Upgrading SP" } - { page: "upgrade", title: "Upgrading SP" }
- { page: "recover", title: "Recoverying the SP Database" } - { page: "recover", title: "Recovering the SP Database" }
--- ---
# Normal Run of a Spectrum Protect Server (in a container) # Normal Run of a Spectrum Protect Server (in a container)
> **NOTES**: > **NOTES**:
> * The storage used in the example below is from the host file systems, passed to the container with Docker's `-v` option. > * The storage used in the example below is from the host file systems, passed to the container with Docker's `-v` option.
> * This implementation is using DISK or CLOUD storage pool storage. While accessing tape should be possible, I havent explored what is required for that to work. > * This implementation is using DISK or CLOUD storage pool storage. While accessing tape should be possible, I haven't explored what is required for that to work.
## Configuration ## Configuration
Our configuration will use these defaults Our configuration will use these defaults
```bash Container Variables ```bash Helpful Variables
TSM_DB=/srv/sp/database TSM_DB=/srv/sp/database
TSM_USER=/srv/sp/user TSM_USER=/srv/sp/user
TSM_DATA=/srv/sp/data TSM_DATA=/srv/sp/data
@ -56,9 +56,9 @@ You should now be able to connect with the Admin Client.
**NOTES**: **NOTES**:
* For Spectrum Protect you need to use `--privileged=true`. From what I have worked out, DB2 needs this. I did find some notes on DB2 in docker [here](https://www.ibm.com/developerworks/data/library/techarticle/dm-1602-db2-docker-trs/index.html), and tried these options without using `--privileged=true` but DB2 still reported some errors. * For Spectrum Protect you need to use `--privileged=true`. From what I have worked out, DB2 needs this. I did find some notes on DB2 in docker [here](https://www.ibm.com/developerworks/data/library/techarticle/dm-1602-db2-docker-trs/index.html), and tried these options without using `--privileged=true` but DB2 still reported some errors.
I dont think it is an issue running SP in privileged mode, but if you do find a solution to have it run without full privileges, I would be interested ;) I don't think it is an issue running SP in privileged mode, but if you do find a solution to have it run without full privileges, I would be interested ;)
The link above has some important tips regarding DB2 memory and the `--memory=` setting. The link above has some important tips regarding DB2 memory and the `--memory=` setting.
* When defining your DISK based storage pools (or STGPOOLDIR for your cloud pools), remember the path to your "data" is `/data` inside the container. If you use another path, you will loose that data when the container is destroyed. * When defining your DISK based storage pools (or STGPOOLDIR for your cloud pools), remember the path to your "data" is `/data` inside the container. If you use another path, you will loose that data when the container is destroyed.
* You should be able to run many SP instances on the same host (or swarm) even at different version levels. In container, they can all respond to port 1500, and outside the container, they'll need to have unique ports eg: `-p 1501:1500`, `-p 1502:1500`, etc. (Just remember to think about resource requirements and potential resource contention.) * You should be able to run many SP instances on the same host (or swarm) even at different version levels. In container, they can all respond to port 1500, and outside the container, they'll need to have unique ports EG: `-p 1501:1500`, `-p 1502:1500`, etc. (Just remember to think about resource requirements and potential resource contention.)
* Use the same `--hostname=` setting that you used when running the *init* steps above. DB2 is sensitive to hostname changes. * Use the same `--hostname=` setting that you used when running the *init* steps above. DB2 is sensitive to hostname changes.
* Spectrum Scale makes a great persistent storage filesytem for Docker containers, and gives you great optoins for Specturm Protect. For example, you could move SP to different hosts very easily (stop on hosta, start on hostb - if both hosts see the same Spectrum Scale file system and your database/data are on it). * Spectrum Scale makes a great persistent storage file system for Docker containers, and gives you great options for Spectrum Protect. For example, you could move SP to different hosts very easily (stop on hosta, start on hostb - if both hosts see the same Spectrum Scale file system and your database/data are on it).

24
source/server/runoc.md
View File

@ -1,30 +1,30 @@
--- ---
title: Operations Center title: Operations Center
header-img: img/header_img/server.jpg header-img: img/header_img/oc.jpg
date: 2018-03-07 09:50:12 date: 2018-03-07 09:50:12
description: Operations Center in a Docker Container description: Operations Centre in a Docker Container
related: related:
- { page: "index", title: "Containerising SP" } - { page: "index", title: "Containerising SP" }
- { page: "runoc", title: "Running OC in a container" } - { page: "runoc", title: "Running OC in a container" }
--- ---
> **NOTES** > **NOTES**
> * Operations Center is a good fit for a container - especially in a cluster (eg: Swarm), it requires no persistent storage (other than the initial config). > * Operations Centre is a good fit for a container - especially in a cluster (EG: Swarm), it requires no persistent storage (other than the initial config).
> * Make sure your host has sufficient memory to perform the build - OC requires at least 4GB of RAM to pass the installation logic. > * Make sure your host has sufficient memory to perform the build - OC requires at least 4GB of RAM to pass the installation logic.
> * The accompaning files for this build are available in my [gitlab](http://dev.leenooks.net/deon/spdocker/tree/oc) > * The accompanying files for this build are available in my [gitlab](http://dev.leenooks.net/deon/spdocker/tree/oc)
> * And lastly, most importantly, running SP in a container is **not officially supported by IBM** - so you cannot make a helpdesk call about having problems. > * And lastly, most importantly, running SP in a container is **not officially supported by IBM** - so you cannot make a help-desk call about having problems.
Technically it should work without problems, but if you are experiencing a problem, be prepared to recreate your environment on a physical host and validate that your problem still exists (and thus not a result of Docker or the implementation on Docker). Technically it should work without problems, but if you are experiencing a problem, be prepared to recreate your environment on a physical host and validate that your problem still exists (and thus not a result of Docker or the implementation on Docker).
The process for Containerising OC is very similiar to [Containerising Spectrum Portect](index.html). The process for Containerising OC is very similar to [Containerising Spectrum Protect](index.html).
## Create your Dockerfile ## Create your Dockerfile
As at the time of writing these instructions, Red Hat linux is not available in the Docker hub - but CentOS is a good clone. We'll use it As at the time of writing these instructions, Red Hat Linux is not available in the Docker hub - but CentOS is a good clone. We'll use it
### Image preparation ### Image preparation
```Dockerfile ```Dockerfile
FROM centos:7 FROM centos:7
``` ```
Operations Center doesnt really need any dependancies - you can check the [Knowledge Center](https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.4/srv.install/c_oc_inst_reqs_os.html) for any updates. However, to pass the installation logic, we do need to have a redhat-release RPM installed. For CentOS it doesnt exist (CentOS calls it centos-release), so I created one. Operations Centre doesn't really need any dependencies - you can check the [Knowledge Center](https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.4/srv.install/c_oc_inst_reqs_os.html) for any updates. However, to pass the installation logic, we do need to have a redhat-release RPM installed. For CentOS it doesn't exist (CentOS calls it centos-release), so I created one.
```Dockerfile ```Dockerfile
RUN yum -y install http://yum.leenooks.net/CentOS/7/base/redhat-release-7-6.el7.centos.12.2.x86_64.rpm \ RUN yum -y install http://yum.leenooks.net/CentOS/7/base/redhat-release-7-6.el7.centos.12.2.x86_64.rpm \
@ -35,11 +35,11 @@ RUN yum -y install http://yum.leenooks.net/CentOS/7/base/redhat-release-7-6.el7.
yum clean all && rm -rf /var/tmp/* yum clean all && rm -rf /var/tmp/*
``` ```
<small>**NOTE**: During the installation of SP, it tests to make sure it is running on RedHat linux by testing the existence of an RPM package `redhat-release`. The redhat-release installed on line 1 is a fake RPM package - it installs no files, but just emulates the existance of that required RPM package.</small> <small>**NOTE**: During the installation of SP, it tests to make sure it is running on RedHat Linux by testing the existence of an RPM package `redhat-release`. The redhat-release installed on line 1 is a fake RPM package - it installs no files, but just emulates the existence of that required RPM package.</small>
### Operations Centre Installation ### Operations Centre Installation
Now copy in our silent install configuration files. These files are based on the silent install xml files that come on the installation media. Some details on these files are also in the [Knowledge Center](https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.4/srv.install/t_srv_inst_silently-linux.html) Now copy in our silent install configuration files. These files are based on the silent install XML files that come on the installation media. Some details on these files are also in the [Knowledge Center](https://www.ibm.com/support/knowledgecenter/SSEQVQ_8.1.4/srv.install/t_srv_inst_silently-linux.html)
```Dockerfile ```Dockerfile
COPY install.xml install-fp.xml /tmp/ COPY install.xml install-fp.xml /tmp/
@ -57,7 +57,7 @@ RUN SOURCE_URL=http://YOUR_SITE_URL_HERE && \
rm -rf /tmp/build /tmp/install*xml rm -rf /tmp/build /tmp/install*xml
``` ```
**NOTE**: Some information on what is occuring here. **NOTE**: Some information on what is occurring here.
* To avoid the resulting Docker Image being too large, we download the installation media from a web server (Line 3) and then delete it after it has been expanded (line 4). * To avoid the resulting Docker Image being too large, we download the installation media from a web server (Line 3) and then delete it after it has been expanded (line 4).
* Line 5 is the silent installation method for installing SP components. * Line 5 is the silent installation method for installing SP components.
* Line 6 deletes any installation files to reduce the size of the resulting Docker Image. * Line 6 deletes any installation files to reduce the size of the resulting Docker Image.
@ -87,5 +87,5 @@ Now that your container is built, starting OC is simple
`docker run --detach=true --interactive=false --link=spectrumprotect -v <HOST_PATH>/serverConnection.properties:/opt/tivoli/tsm/ui/Liberty/usr/servers/guiServer/serverConnection.properties --memory=1g -p 11443:11443 --privileged=false --rm=false --restart=no --tty=false --name=spectrumprotectoc ibm/spectrumprotect-oc:8.1.1` `docker run --detach=true --interactive=false --link=spectrumprotect -v <HOST_PATH>/serverConnection.properties:/opt/tivoli/tsm/ui/Liberty/usr/servers/guiServer/serverConnection.properties --memory=1g -p 11443:11443 --privileged=false --rm=false --restart=no --tty=false --name=spectrumprotectoc ibm/spectrumprotect-oc:8.1.1`
***NOTES***: ***NOTES***:
* The OC container needs to connect to your SP server(s). If your SP server is running in a container, you can use the `--link=` to link to the container name of your server (not necessarily required). If you are running your container in a cluster (eg: Docker Swarm), then you'll need to use the appropriate configuration for your cluster to do container-to-container networking. * The OC container needs to connect to your SP server(s). If your SP server is running in a container, you can use the `--link=` to link to the container name of your server (not necessarily required). If you are running your container in a cluster (EG: Docker Swarm), then you'll need to use the appropriate configuration for your cluster to do container-to-container networking.
* The `-v <HOST_PATH>/serverConnection.properties:../serverConnection.properties` will saving you having to do a "setup" of OC if the container is ever stopped and destroy (this will happen in cluster environments). Make sure the file reference in the `-v` option is available on your cluster persistent storage. * The `-v <HOST_PATH>/serverConnection.properties:../serverConnection.properties` will saving you having to do a "setup" of OC if the container is ever stopped and destroy (this will happen in cluster environments). Make sure the file reference in the `-v` option is available on your cluster persistent storage.

122
source/server/upgrade.md
View File

@ -7,20 +7,21 @@ related:
- { page: "index", title: "Containerising SP" } - { page: "index", title: "Containerising SP" }
- { page: "firstrun", title: "Initial Container Setup" } - { page: "firstrun", title: "Initial Container Setup" }
- { page: "normalrun", title: "Normal Run of Container" } - { page: "normalrun", title: "Normal Run of Container" }
- { page: "recover", title: "Recoverying the SP Database" } - { page: "recover", title: "Recovering the SP Database" }
--- ---
# Upgrading Spectrum Protect Server (in a container) # Upgrading Spectrum Protect Server (in a container)
**NOT YET COMPLETE**
> **NOTES**: > **NOTES**:
> * The storage used in the example below is from the host file systems, passed to the container with Docker's `-v` option. > * The storage used in the example below is from the host file systems, passed to the container with Docker's `-v` option.
> * Upgrading needs special thought. > * Upgrading needs special thought.
I have found that upgrading within a major release, eg: 7.1.0 to 7.1.1, 7.1.2 to 7.1.3, etc works transparently. Upgrading from 6 to 7 (I havent tried) and 7 to 8 (failed for me). Spectrum Protect is smart enough to realise that it has been upgraded and will do a one time schema update. However, DB2 needs thought, especially if there has been a major change in DB2 between SP releases. I have found that upgrading within a major release, EG: 7.1.0 to 7.1.1, 7.1.2 to 7.1.3, etc works transparently. If upgrading that results in a version of DB2 changing (like v7 to v8) additional consideration needs to be made that has not yet been captured in my `upgrade` option below. (Upgrading from v7 to v8 failed for me because of DB2 issues that I wasn't able to resolve - DB2 changed versions between 7 and 8.)
> * As always, make sure you have a backout plan, in case things go pair shaped. I exploit Docker it provides a great backout option, especially if you use it with persistenet Docker Container storage that can exploit snapshots and reverting (or directly accessing) a snapshot. In the example below, I'm using Linux LVM snapshots to provide my backup option. Specturm Scale would also be a good choice. After the upgrade Spectrum Protect is smart enough to realise that it has been upgraded and will do a one time schema update.
> * As always, make sure you have a back-out plan, in case things go pair shaped. Docker itself provides a great back-out option, especially if you use it with persistent Docker Container storage that can exploit snapshots and reverting (or directly accessing) a snapshot.
In the example below, I'm using Linux LVM snapshots to provide my backup option. Spectrum Scale would also be a good choice.
## Configuration ## Configuration
Our configuration will use these defaults Our configuration will use these defaults
```bash Container Variables ```bash Helpful Variables
TSM_DB=/srv/sp/database TSM_DB=/srv/sp/database
TSM_USER=/srv/sp/user TSM_USER=/srv/sp/user
TSM_DATA=/srv/sp/data TSM_DATA=/srv/sp/data
@ -33,50 +34,70 @@ TSM_DATA=/srv/sp/data
# Upgrade Steps # Upgrade Steps
## Backup SP ## Backup Spectrum Protect's Database
`backup db type=FULL devclass=<YOUR BACKUP DEVCLASS> protectk=y pass=<PASSWORD> wait=yes` `backup db type=FULL devclass=<YOUR BACKUP DEVCLASS> protectk=y pass=<PASSWORD> wait=yes`
```plain Expected Output ```plain Expected Output
Protect: SERVER1>backup db t=f devclass=dbb protectk=y pass=password wait=y Protect: SERVER1>backup db t=f devclass=dbb protectk=y pass=password wait=y
ANR0984I Process 8 for Database Backup started in the FOREGROUND at 15:59:21. ANR0984I Process 143 for Database Backup started in the FOREGROUND at 14:48:02.
ANR4559I Backup DB is in progress. ANR4559I Backup DB is in progress.
ANR2280I Full database backup started as process 8. ANR2280I Full database backup started as process 143.
ANR4626I Database backup will use 1 streams for processing with the number originally requested 1. ANR4626I Database backup will use 1 streams for processing with the number originally requested 1.
ANR4550I Full database backup (process 8) completed. Total bytes backed up: 2,232,418,304. ANR4550I Full database backup (process 143) completed. Total bytes backed up: 2,215,641,088.
ANR0985I Process 8 for Database Backup running in the FOREGROUND completed with completion state SUCCESS at 16:02:16. ANR0985I Process 143 for Database Backup running in the FOREGROUND completed with completion state SUCCESS at 14:52:45.
ANR2394I BACKUP DB: Server device configuration information was written to all device configuration files. ANR2394I BACKUP DB: Server device configuration information was written to all device configuration files.
Protect: LEENOOKS>q volh type=dbb Protect: SERVER1>q volh type=dbb
... ...
Date/Time: 08-03-2018 15:59:22 Date/Time: 14-03-2018 14:48:04
Volume Type: BACKUPFULL Volume Type: BACKUPFULL
Backup Series: 246 Backup Series: 253
Backup Operation: 0 Backup Operation: 0
Volume Seq: 100,001 Volume Seq: 100,001
Device Class: DBB Device Class: DBB
Volume Name: /data/dbb/20485162.DBV Volume Name: /data/dbb/20999284.DBV
``` ```
Make sure your backup is successful and make a note of where it is, in case you need it. Make sure your backup is successful and make a note of where it is, in case you need it.
```bash On the Host ```bash On the Host
[root@host]# ls -aln $TSM_DATA/dbb/|grep 20485162 [root@host]# ls -aln $TSM_DATA/dbb/|grep 20999284
-rw------- 1 201 201 2232419730 Mar 8 16:01 20485162.dbv -rw------- 1 201 201 2215642514 Mar 14 14:51 20999284.dbv
``` ```
## Stop SP ## Stop Spectrum Protect
Stop your SP server. Stop your SP server.
`halt` ```plain Stop the SP Server
Protect: SERVER1>halt
ANR2234W This command will halt the server; if the command is issued from a remote client, it may not be possible to restart the server from the remote location.
<small>**NOTE**: Dont use `docker stop` to stop your SP server. Do you wish to proceed? (Yes (Y)/No (N)) y
By default docker sends a SIGTERM signal to the containers process 1 and then waits 10 seconds for it to die. If it hasnt died, it will then send it a SIGKILL.
While you can control the timeout between the SIGTERM and SIGKILL signal and you can even control which kill signal is sent, I havent explored what signals SP will act on. So as a recommendation, always use SP `halt` command to stop an SP server.</small> ANS8002I Highest return code was 0.
[root@host]#
```
<small>**NOTE**: Don't use `docker stop` to stop your SP server.
By default docker sends a SIGTERM signal to the containers process 1 and then waits 10 seconds for it to die. If it hasn't died, it will then send it a SIGKILL.
While you can control the timeout between the SIGTERM and SIGKILL signal and you can even control which kill signal is sent, I haven't explored what signals SP will act on. So as a recommendation, always use SP `halt` command to stop an SP server.</small>
## If you can, snapshot your persistent storage. ## If you can, snapshot your persistent storage.
If you can snapshot your persistent storage that you provide to the SP Docker Container, that will save you doing a "restoredb" if something goes pair shaped. If you can snapshot your persistent storage that you provide to the SP Docker Container, that will save you doing a "restoredb" if something goes pair shaped.
```bash Using LVM Snapshot
[root@host]# df /srv/sp
Filesystem 1K-blocks Used Available Use% Mounted on
/dev/mapper/vgroot-lvprotect 20511356 10031500 9431280 52% /srv/sp
[root@host]# lvcreate -s -L5G -n lvspsnapshot vgroot/lvprotect
Using default stripesize 64.00 KiB.
Logical volume "lvspsnapshot" created.
```
<small>**NOTE**: If you are using IBM Spectrum Scale, you can use `mmcrsnapshot`</small>
## Delete your container ## Delete your container
`docker rm -v spectrumprotect` `docker rm -v spectrumprotect`
@ -85,16 +106,67 @@ If you can snapshot your persistent storage that you provide to the SP Docker Co
### You need to do a one-time start of the container with the option `upgrade`. ### You need to do a one-time start of the container with the option `upgrade`.
`docker run --detach=true --hostname=tsm --interactive=false -v $TSM_USER:/tsm -v $TSM_DB:/database --privileged=true --rm=true --tty=false --name=spectrumprotect ibm/spectrumprotect:8.1.4 upgrade` `docker run --detach=true --hostname=tsm --interactive=false -v $TSM_USER:/tsm -v $TSM_DB:/database --privileged=true --rm=false --tty=false --name=spectrumprotect ibm/spectrumprotect:8.1.4 upgrade`
<small>**NOTE**: You can see the console by running `docker logs -f spectrumprotect` to make sure the upgrade works as expected. Once you see success, you can delete the container with `docker rm -v spectrumprotect`.</small>
```bash Expected Output
[root@host]# docker logs -f spectrumprotect
DB2 installation is being initialized.
Total number of tasks to be performed: 4
Total estimated time for all tasks to be performed: 309 second(s)
Task #1 start
Description: Setting default global profile registry variables
Estimated time 1 second(s)
Task #1 end
...
The execution completed successfully.
For more information see the DB2 installation log at "/tmp/db2iupdt.log.5".
DBI1446I The db2iupdt command is running.
DBI1070I Program db2iupdt completed successfully.
```
### Normal Run ### Normal Run
Spectrum Protect is ready to run on the new version as normal. The first time it runs at a later release, you should see some `ANR0276I` messages.
`docker run --detach=true --hostname=tsm --interactive=false -v $TSM_USER:/tsm -v $TSM_DB:/database -v $TSM_DATA:/data --memory=16g -p 1500:1500 -p 1543:1543 --privileged=true --rm=false --restart=no --tty=false --name=spectrumprotect ibm/spectrumprotect:8.1.4` `docker run --detach=true --hostname=tsm --interactive=false -v $TSM_USER:/tsm -v $TSM_DB:/database -v $TSM_DATA:/data --memory=16g -p 1500:1500 -p 1543:1543 --privileged=true --rm=false --restart=no --tty=false --name=spectrumprotect ibm/spectrumprotect:8.1.4`
```plain Expected Output ```plain Expected Output
show the upgrade ANR7800I DSMSERV generated at 16:03:41 on Nov 17 2017.
IBM Spectrum Protect for Linux/x86_64
Version 8, Release 1, Level 4.000
Licensed Materials - Property of IBML
(C) Copyright IBM Corporation 1990, 2017.
All rights reserved.
U.S. Government Users Restricted Rights - Use, duplication or disclosure
restricted by GSA ADP Schedule Contract with IBM Corporation.
ANR7801I Subsystem process ID is 7.
...
ANR0275I Detected schema change for table SD.Pool.Clusters, attempting to
reconcile.
ANR0276I Schema reconciliation for table SD.Pool.Clusters succeeded.
ANR0275I Detected schema change for table SD.Containers, attempting to
reconcile.
ANR0276I Schema reconciliation for table SD.Containers succeeded.
ANR0275I Detected schema change for table SD.Replicated.Stgpools, attempting to
reconcile.
ANR0276I Schema reconciliation for table SD.Replicated.Stgpools succeeded.
...
IBM Spectrum Protect:SERVER1>
``` ```
I did this on the lab server, and while admittedly it doesn't have much data - the whole "upgrade" process took 12 minutes after the DB backup was complete - and that was with me documenting it for this wiki. So it could have been quicker!
Happy days :) Happy days :)
> If something did go pear shapped, then revert your snapshot (or if it is read only, copy the files back to the original path) and then start your container with the original image (the old version) of SP. It should start up as normal as if it was only shut down temporarily. > If something did go pear shaped, then revert your snapshot (or if it is read only, copy the files back to the original path) and then start your container with the original image (the old version) of SP. It should start up as normal as if it was only shut down temporarily.
> If you really have problems, you have that backup to recover from! Start the container with the `restore <PASSWORD>` option - you'll need a DBDIRS.TXT file as well in the instance home directory. More info on recovering [here](recover). > If you really have problems, you have that backup to recover from! Start the container with the `restore <PASSWORD>` option - you'll need a DBDIRS.TXT file as well in the instance home directory. More info on recovering [here](recover).