本文遵循BY-SA版权协议，转载请附上原文出处链接。

本文作者: 黑伴白

本文链接: http://heibanbai.com.cn/posts/f000a388/

DataStage11.7集群部署

此文档为IBM InfoSphere Information Server 11.7.1.3版本的集群搭建步骤

步骤较为简略，仅供参考，有问题可留言交流

官网安装说明: Installing IBM InfoSphere Information Server by using the wizard (graphical mode) - IBM Documentation

进程服务: Processes used by InfoSphere Information Server - IBM Documentation

增加计算节点: Adding computers to share engine processing - IBM Documentation

官网非root化说明: Starting IBM InfoSphere Information Server node agent as a non-root user - IBM Documentation

启动服务说明: Starting services (Linux, UNIX) - IBM Documentation

环境信息

# 数据库信息
199.188.166.109:60000/dasdb

# 数据库用户密码
dasuser/heibanbai
wasuser/heibanbai
dsouser/heibanbai

# 数据库表空间 page_size 32kb
XMETA
TBS_DASDB_DAT

# ETL服务器信息
199.188.166.110
199.188.166.111
199.188.166.112

# 安装目录
/app

# 安装完成后非root化用户
app

软件包

# 安装包目录
/app/soft/datastage_11713/

# 主安装包
IS_V11713_Linux_x86_multi.tar.gz

# lic认证文件包
IS_V11713_bundle_spec_file_mult.zip

主节点安装

解压安装包

cd /app/soft/datastage_11713/

# 解压主安装包 解压后为is-suite目录
tar -zxvf IS_V11713_Linux_x86_multi.tar.gz

# 解压lic认证包 解压文件至is-suite目录中
unzip IS_V11713_bundle_spec_file_mult.zip -d is-suite/

执行安装程序

# 需使用root用户执行
cd /app/soft/datastage_11713/is-suite
./setup

# 从浏览器打开输出的最下方的地址 若本机hosts没有配置则将主机名替换为IP地址
heibanbai:/app/soft/datastage_11713/is-suite # ./setup
INFO: Installation program will be running in this session, please do not close installation window until installation program completes.
The installation program started at 2/2/23 10:53 AM.


======> Enter the following URL to your web browser to begin the installation process:
https://heibanbai:8446/ISInstall

选择安装语言

安装说明

防火墙要求说明

前期环境检查

选择安装路径

选择安装方式

选择安装组件

选择安装产品

许可声明

服务器集群高可用

选择是否安装数据库

点击next后如果出现如下页面（元数据库配置），可能是由于安装过程的网络交互等原因，需要点击左边树状进度条High Availablity Server Cluster Configuration 将安装进程往回切换一下，然后在HA配置页面点击NEXT：

是否安装WAS

选择WAS安装目录

WAS服务端口分配

配置WAS管理员用户

配置InformationServer管理员用户

配置数据库信息

配置WAS控制台数据库信息

CAS端口选择

ABS代理程序端口配置

操作系统管理员用户设置

作业监视器端口配置

为Engine层指定ITAG以区别于同服务器上的其它版本Datastage实例

是否安装全球化支持

WebsphereMQ插件选择

SAS设置

预建工程配置

操作数据库配置信息，用于存储操作控制台使用的监控信息

选择安装版本

安装环境检查

Error:

Ensure that the user app can read from directory /app/soft/datastage_11713/is-suite.

heibanbai:/app/soft/datastage_11713 # chmod +r -R is-suite

Waring：

Ensure kernel parameters are valid for DSEngine.

Update kernel parameter ‘kernel.sem’. Its current value is 1250 32000 100 256. The recommended value is 250 128000 32 1024.

# 查看当前值
cat /proc/sys/kernel/sem
1250    32000   100     256
# 修改值
vi /etc/sysctl.conf
# 添加
kernel.sem = 1250    128000   100     1024
# 生效
sysctl -p

响应文件设置

安装情况总览

安装进度

完成安装

计算节点安装

SSH免密登录配置

• 生成秘钥，三次回车即可

  heibanbai:~ # ssh-keygen -t rsa
  Generating public/private rsa key pair.
  Enter file in which to save the key (/root/.ssh/id_rsa):
  Enter passphrase (empty for no passphrase):
  Enter same passphrase again:
  Your identification has been saved in /root/.ssh/id_rsa.
  Your public key has been saved in /root/.ssh/id_rsa.pub.
  The key fingerprint is:
  SHA256:5WLS91GgGV/YnJMNLT+gNhJY76x6UaCvN1BEKwYIkas root@heibanbai
  The key's randomart image is:
  +---[RSA 2048]----+
  | o+ ..  +o. .+o* |
  | . .  .. +o=.+B o|
  |  .    oo.*.o o+ |
  | .    .o.=o= . ..|
  |.     . S *oo   .|
  |E      + =.. .   |
  |        o.. .    |
  |       ..+       |
  |       .o .      |
  +----[SHA256]-----+

• 复制秘钥

  heibanbai:~ # ssh-copy-id -i ~/.ssh/id_rsa.pub 199.188.166.110
  /usr/bin/ssh-copy-id: INFO: Source of key(s) to be installed: "/root/.ssh/id_rsa.pub"
  /usr/bin/ssh-copy-id: INFO: attempting to log in with the new key(s), to filter out any that are already installed
  /usr/bin/ssh-copy-id: INFO: 1 key(s) remain to be installed -- if you are prompted now it is to install the new keys
  Password:
  
  Number of key(s) added: 1
  
  Now try logging into the machine, with:   "ssh '199.188.166.110'"
  and check to make sure that only the key(s) you wanted were added.

共享及目录配置

将Datasets、Projects、Scratch三个目录移动至共享存储上，同时建立链接

# 共享目录配置
cd /data/etlds
mkdir datastage_mds
mv /app/IBM/InformationServer/Server/Datasets /data/etlds/datastage_mds/
mv /app/IBM/InformationServer/Server/Projects /data/etlds/datastage_mds/
mv /app/IBM/InformationServer/Server/Scratch /data/etlds/datastage_mds/

cd /app/IBM/InformationServer/Server
ln -s /data/etlds/datastage_mds/Datasets Datasets
ln -s /data/etlds/datastage_mds/Projects Projects
ln -s /data/etlds/datastage_mds/Scratch Scratch

组件配置

# 主节点压缩下面三个目录然后cp至至计算节点
tar -zcvf /data/etlds/tmp/ds_agent.tar.gz /app/IBM/InformationServer/ASBNode /app/IBM/InformationServer/jdk /app/IBM/InformationServer/Server

# 解压至计算节点目录
cd /app/IBM/InformationServer
tar -zxvf ds_agent.tar.gz

目录链接

# 保持和之前老环境DS目录一致 主节点及计算节点均操作
cd /home
mkdir dsinst
cd dsinst
ln -s /app/IBM/InformationServer InformationServer

凭证用户设置

# wasadmin用户登录 https://199.188.166.110:9446/ibm/iis/console

非ROOT化

备份操作

# root用户操作
mkdir /app/IBM/no_root_backup

tar -zcvf  /app/IBM/no_root_backup/ASBServer.tar.gz  /app/IBM/InformationServer/ASBServer
tar -zcvf /app/IBM/no_root_backup/InfoSphere.tar.gz /app/IBM/WebSphere/AppServer/profiles/InfoSphere
tar -zcvf /app/IBM/no_root_backup/ASBNode.tar.gz /app/IBM/InformationServer/ASBNode

cp /etc/profile  /etc/profile_20230202
cp /etc/security/limits.conf /etc/security/limits.conf_20230202
cp /etc/rc.d/ISFAgents /etc/rc.d/ISFAgents_20230202
cp /etc/rc.d/ISFServer /etc/rc.d/ISFServer_20230202

# app用户操作
cp /app/IBM/InformationServer/Server/DSEngine/dsenv /app/IBM/InformationServer/Server/DSEngine/dsenv_20230202
cp /app/.profile /app/.profile_20230202

配置修改

# root用户操作
echo "ulimit -n 10240">>/etc/profile

echo "*  soft  nofile  10240">>/etc/security/limits.conf
echo "*  hard  nofile 10240">>/etc/security/limits.conf

mv /tmp/*.lck /app/IBM/no_root_backup

chmod -R g+rwx /app/IBM/InformationServer/ASBServer/  
chmod -R o+rx /app/IBM/InformationServer/ASBServer/ 
chmod 775 /app/IBM/InformationServer/Server/*

chown -R app  /tmp/informationServer
chown -R app /app/IBM/InformationServer/ASBServer
chown -R app /app/IBM/WebSphere/AppServer/profiles/InfoSphere
chown -R app /app/IBM/InformationServer/ASBNode

# 修改 /etc/rc.d/ISFAgents
cd "/app/IBM/InformationServer/ASBNode/bin"
/bin/su - app -c "/app/IBM/InformationServer/ASBNode/bin/NodeAgents.sh" "$@"

# 修改 /etc/rc.d/ISFServer
cd "/app/IBM/InformationServer/ASBServer/bin"
/bin/su - app -c "/app/IBM/InformationServer/ASBServer/bin/MetadataServer.sh" "$@"

# /app/IBM/InformationServer/Server/DSEngine/dsenv 加入以下内容 app用户操作
DB2INST=/app
export DB2INST
. ${DB2INST}/sqllib/db2profile

ORACLE_HOME=/oracle/product/19.3.0/client64
export ORACLE_HOME PATH=${PATH}:${ORACLE_HOME}/bin
export PATH
LD_LIBRARY_PATH=${ORACLE_HOME}/lib:${LD_LIBRARY_PATH}
export LD_LIBRARY_PATH
export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK

# 修改app用户环境变量 /app/.profile 加入以下内容
export ORACLE_HOME=/oracle/product/19.3.0/client64
export PATH=$ORACLE_HOME/bin:$PATH
export LD_LIBRARY_PATH=$ORACLE_HOME/lib32:$LD_LIBRARY_PATH

export NLS_LANG=AMERICAN_AMERICA.ZHS16GBK
export APT_ORACLE_NO_OPS=1
LD_LIBRARY_PATH=/app/sqllib/bin:/app/sqllib/adm:/app/sqllib/misc:/app/sqllib/lib:$LD_LIBRARY_PATH
export LD_LIBRARY_PATH

DSHOME=/app/IBM/InformationServer/Server/DSEngine; export DSHOME
ASBHOME=/app/IBM/InformationServer/ASBNode; export ASBHOME

export PATH=$PATH:${DSHOME}/bin:${ASBHOME}/bin
LANG=en_US.utf8
export LANG

创建登录用户

# wasadmin用户登录 https://199.188.166.110:9446/ibm/iis/console

创建工程

• 运行Administrator Client，登录服务器

  

• 点击ADD进行新增

  

• 定义工程名字及路径

  

• 配置工程属性信息

  

  

  

  

  

Routines

Server Routine

checkWarning

• Routine Name

  checkWarning

• Type

  Before/After Subroutine

• Arguments

  1. InpurtArg
  2. ErrorCode

• Code

  $INCLUDE DSINCLUDE JOBCONTROL.H
  ErrorCode = 0
  JobHandle = DSJ.ME
  JobName = DSGetJobInfo(JobHandle,DSJ.JOBNAME)
  JobStats = DSGetJobInfo(JobHandle,DSJ.DSJ.USERSTATUS)
  call DSLogInfo(JobStats,"当前作业状态：")
  currentDate=OCONV(DATE(), "D4-YMD[4,2,2]")
  startTime = DSGetJobInfo(JobHandle,DSJ.JOBSTARTTIMESTAMP)
  endTime = currentDate:" 23:59:59"
  warnings =  DSGetLogSummary(JobHandle,DSJ.LOGWARNING,startTime,endTime,0)
  Warning:
  if warnings ne "" then
     call DSLogFatal("作业出现警告，失败！","JobCheckAfterRouting")
     ErrorCode = 1
     goto NormalExit
  end
  NormalExit:

启停服务操作

#!/bin/sh
usage(){
   echo "usage:"
   echo "serviceAdmin.sh {stop|start|restart}"
   echo "stop:stop the DS service"
   echo "start:start the DS service"
   echo "restart:restart the DS service"
   exit -1
}
stopService(){
   echo "start to stop the DS service......"
   echo "**********************************"
   cd /app/IBM/InformationServer/Server/DSEngine/bin
   ./uv -admin -stop
   cd /app/IBM/InformationServer/ASBNode/bin
   ./NodeAgents.sh stop
   cd /app/IBM/InformationServer/ASBServer/bin
   ./MetadataServer.sh stop
   echo "**********************************"
   echo "finish to stop the DS service!!!!!"
}
startService(){
   echo "start to start the DS service......"
   echo "**********************************"
   cd /app/IBM/InformationServer/Server/DSEngine/bin
   ./uv -admin -start
   cd /app/IBM/InformationServer/ASBNode/bin
   ./NodeAgents.sh start
   cd /app/IBM/InformationServer/ASBServer/bin
   ./MetadataServer.sh run
   echo "**********************************"
   echo "finish to start the DS service!!!!!"
}
if [ $# -ne 1 ]
then
   usage
fi
if [ "X$1" == "Xstop" ]
then
   stopService
else
   if [ "X$1" == "Xstart" ]
   then
      startService
   else
      if [ "X$1" == "Xrestart" ]
      then
         stopService
         startService
      else
         usage
      fi
   fi
fi

后台数据库密码修改后操作

停止服务

cd /app/IBM/InformationServer/ASBServer/bin

./AppServerAdmin.sh -db -user 数据库用户名 -password 新密码

启动服务

---

蚂蚁再小也是肉🥩！