Pre-Installation Tasks
- Jason Kinchen
Analytics
This section discusses the pre-installation tasks required before installing and building SciDB in either single or multi-server configurations.
Download SciDB Community Edition
To download SciDB CE, do the following:
- Access the 15.12 CE page and download the 15.12 source code tarball.
- cd <dev_dir>; tar -xvzf <the_tarball>
Optional: create a directory under the <dev_dir> called scidbtrunk and install the tarball there to match the installation instructions.
Installing Expect, and SSH Packages
To install expect, and ssh packages, run the following:
For CentOS:
sudo yum -y install expect openssh-server openssh
For Ubuntu
sudo apt-get -y install expect openssh-server openssh-client
Starting the SSH Server
To start the SSH server, do the following:
- For CentOS:
Check whether sshd is running:
sudo service sshd status
If sshd is not running, start the sshd server as follows:
sudo chkconfig --add sshd sudo chkconfig sshd on sudo service sshd start
If sshd is running restart sshd server as follows:
sudo service sshd restart
For Ubuntu:
Check whether ssh is running:
sudo service ssh status
If ssh is not running, start the ssh server as follows:
sudo /usr/sbin/update-rc.d ssh defaults sudo service ssh start
If ssh is running restart ssh server as follows:
sudo service ssh restart
Allowing SciDB to Communicate on all Ports
To ensure SciDB communicates on all ports, do the following:
For CentOS:
Turn off iptables.
sudo chkconfig --del iptables sudo chkconfig iptables off sudo service iptables stop
For Ubuntu there is no firewall by default.
Providing Passwordless SSH
SciDB requires passwordless ssh on all servers logging in on each server as:
- the <scidb_usr> user, for starting and stopping SciDB.
- the root user, for initializing SciDB on each server.
cd ~ ssh-keygen # DO NOT enter a passphrase; accept the default. chmod 755 ~ chmod 755 ~/.ssh cd <dev_dir>/scidbtrunk deployment/deploy.sh access root "" "" <hostIP0> <hostIP1> deployment/deploy.sh access <scidb_usr> "" "" <hostIP0> <hostIP1>
In the deploy access commands, for a single host, omit <hostIP1>. With more than two hosts, list all of them. This rule applies to all subsequent commands using <hostIP1>.
To confirm that as user scidb on <hostIP0>, you can ssh to all hosts without providing a password, run the following:
ssh <hostIP0> date ssh <hostIP1> date
For multi-server installations, SSH is required on all the hosts.
Fixing /etc/hosts on CentOS
You must remove the default /etc/hosts file on CentOS 6.5 (and possibly earlier versions) due to IPv6 aliases for localhost that are incompatible with SciDB. A good /etc/hosts file for workstation "frobnitz" might look like this:
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4 ::1 localhost6 localhost6.localdomain6 10.0.20.42 frobnitz frobnitz.localdomain frobnitz.local.paradigm4.com
Removing Old Versions of SciDB
If you've previously installed SciDB, remove the old version before installing the new version as follows.
Perform the following steps on all hosts.
To remove old versions of SciDB, do the following:
- Remove the following files:
For CentOS, remove scidb.repo, paradigm4.repo, and p4.repo from /etc/yum.repos.d.
For Ubuntu, remove scidb.list, paradigm4.list, and p4.list from /etc/apt/sources.list.d.
- Run the following:
For CentOS:
yum list | grep paradigm4 | awk '{print $1}' | grep '[a-zA-Z]' | xargs sudo rpm -e yum list | grep scidb | awk '{print $1}' | grep '[a-zA-Z]' | xargs sudo rpm -eFor Ubuntu:
dpkg --list | grep paradigm4 | awk '{print $2}' | xargs sudo dpkg --purge dpkg --list | grep scidb | awk '{print $2}' | xargs sudo dpkg --purge
Installing Build Tools
To install the various libraries, scripts, tools, debuggers, and compilers SciDB CE requires, run the following:
cd <dev_dir>/scidbtrunk deployment/deploy.sh prepare_toolchain <hostIP0>
Preparing a Chroot (Multi-Server Only)
For multi-server configurations, prepare a chroot by running the following:
cd <dev_dir>/scidbtrunk deployment/deploy.sh prepare_chroot <scidb_usr> <hostIP0>
Installing Postgres
If you have previously initialized Postgres, either uninstall it, or manually delete the Postgres data directory before doing this step or you will receive a Data directory is not empty![FAILED] error.
SciDB stores the metadata describing its arrays in a Postgres database on one of the server hosts. To configure a Postgres instance for this purpose, do the following:
Run:
cd <dev_dir>/scidbtrunk deployment/deploy.sh prepare_postgresql postgres postgres <netmask> <hostIP0>
This installs and configures the Postgres instance under username postgres with password postgres to accept client network connections using <netmask>.Change the above <netmask> as needed for your environment. For example, if <hostIP0> is: 10.0.20.31, and ifconfig shows:
$ ifconfig | grep Mask inet addr:10.0.20.31 Bcast:10.0.20.255 Mask:255.255.0.0 inet addr:127.0.0.1 Mask:255.0.0.0
then use a <netmask> of 10.0.20.0/16.
Common values for Mask (assuming your host IP is a.b.c.d) and the resulting <netmask> are:
255.255.255.0 a.b.c.d/24 255.255.254.0 a.b.c.d/23 255.255.0.0 a.b.c.d/16
Providing the postgres user Access to SciDB Code
You must complete this step to avoid a run.py install error that prevents the postgres user from reading/executing the scidb.py script.
To give the postgres user access to the SciDB source code, do the following:
Run:
sudo usermod -G <scidb_usr> -a postgres chmod g+rx <dev_dir>
Then run:
/usr/bin/sudo -u postgres ls <dev_dir> # the 'postgres' user should have access.
If this step generates a permission denied error, check the permission of <dev_dir> AND its ancestor directories. A common misconfiguration is when <dev_dir> resides inside the scidb user's home directory, while /home/scidb has 700 access mode. If you encounter this issue, try changing the permissions of the scidb user's home folder to 750. Do not enable group write permissions on the scidb home folder as that breaks passwordless SSH.